Welcome to Siva's Blog

~-Scribbles by Sivananda Hanumanthu
My experiences and learnings on Technology, Leadership, Domains, Life and on various topics as a reference!
What you can expect here, it could be something on Java, J2EE, Databases, or altogether on a newer Programming language, Software Engineering Best Practices, Software Architecture, SOA, REST, Web Services, Micro Services, APIs, Technical Architecture, Design, Programming, Cloud, Application Security, Artificial Intelligence, Machine Learning, Big data and Analytics, Integrations, Middleware, Continuous Delivery, DevOps, Cyber Security, Application Security, QA/QE, Automations, Emerging Technologies, B2B, B2C, ERP, SCM, PLM, FinTech, IoT, RegTech or any other domain, Tips & Traps, News, Books, Life experiences, Notes, latest trends and many more...

Sunday, May 1, 2022

Upgrade to the Modern Data Stack

 

Solution: The Modern Data Stack

There's already a tool perfectly suited to storing massive amounts of data, that can be queried easily, and is connected to everything. A database or data warehouse. You probably already have one running in your company that you can reuse so you don't need to buy another CRM, CDP, DMP, MAP, or any other acronym.

Building around a data warehouse has additional benefits such as:

  • You own your data. It helps you comply with different regulations.
  • Get value quicker. It is 10x easier to dump historical data in a DB than importing the data in yet-another-tool.
  • Easier to sync with other tools. Databases integrate with everything, contrary to SaaS Tools that have limited APIs (and please don't get me started on APIs like Marketo).
  • Reusability. Other teams in the company can use this trusted source of truth.

In addition to a data warehouse, you will need 4 other key components:

  1. An event tracking tool. You can continue using Segment here. It does the job well and allows you to collect events across all of your websites & apps.
  2. A data loader. I recommend Fivetran. It’s easy to set up in a couple of clicks and amazingly reliable.
  3. A data modeling tool. DBT is the new power tool here. It allows you to transform and model your data.
  4. An Integration Platform. I’m a 100% biased here, but I recommend using Census. We integrate well with DBT and enable you to sync your clean and unified data models back to all of your other tools.

As a bonus, you can replace Amplitude with a BI tool like Mode or Chart.io, which is cheap and as good as Looker.

Reference: https://www.getcensus.com/blog/graduating-to-the-modern-data-stack-for-startups