DataData Architecture

The modern ELT stack explained: dbt, ingestion, and orchestration working together

The shift from ETL to ELT reshaped how data teams build pipelines, but the real complexity lies in understanding how the three layers, ingestion, transformation, and orchestration, actually interact. This article breaks down the mechanics of the modern stack with concrete examples, and explains where the genuine tradeoffs sit for leaders making architecture decisions.

🎙️

Listen to the podcast

4 min

The concept at the center of this article is the modern ELT stack: specifically the combination of a cloud data warehouse, a transformation layer built on dbt (data build tool), an ingestion layer typically handled by tools like Fivetran or Airbyte, and an orchestrator such as Airflow or Dagster. Most data leaders can name these tools. Far fewer can explain precisely what each layer does, where one ends and the other begins, and why the sequencing matters so much for organizational outcomes.

That gap between naming and understanding is exactly where bad architectural decisions get made.

Why it matters for CDOs specifically

A CDO who cannot articulate the mechanics of this stack is permanently dependent on engineering teams to translate it. That creates a filtering problem: by the time a decision reaches your desk, it has been compressed, simplified, and often already made. Understanding the architecture means you can participate in the real conversation, catch the tradeoffs before they become technical debt, and speak credibly when allocating budget between vendors.

There is also a talent implication. The modern ELT stack is now the default curriculum for data engineers entering the workforce. Knowing the stack helps you evaluate whether your existing team is using it well, or just using it.

Finally, governance sits here. Column-level lineage, data contracts, freshness SLAs, and transformation logic documentation all live inside or adjacent to this stack. If you care about audit trails and regulatory exposure, you need to understand where in the pipeline those controls actually apply.

How the modern ELT stack actually works

The acronym ELT stands for Extract, Load, Transform. The shift from the older ETL model is significant: instead of transforming data before it reaches the warehouse, you load raw data first and transform it inside the warehouse using SQL. That became practical once cloud warehouses like Snowflake, BigQuery, and Databricks made compute cheap and scalable enough to run large transformations on demand.

Here is how the three layers break down in practice.

Ingestion is the extract-and-load portion. Tools like Fivetran (commercial, vendor-disclosed) or the open-source project Airbyte pull data from source systems, including Salesforce, Postgres databases, Google Analytics, Stripe, and hundreds of others, and deposit it as raw tables in your data warehouse. The ingestion layer does almost no transformation. It replicates. A Fivetran connector to Salesforce will land a table called `opportunity` that looks almost identical to what Salesforce stores internally. Your warehouse now has a timestamped copy of that data, refreshed on a schedule you define.

Transformation is where dbt lives. dbt (developed by dbt Labs, a commercial company) is a framework that lets data analysts and engineers write SQL SELECT statements, called models, that dbt compiles into CREATE TABLE or CREATE VIEW statements and runs against your warehouse. The key insight is that dbt treats transformation logic as code: it is version-controlled in Git, it produces a dependency graph (a DAG) of all your models, and it auto-generates documentation and lineage. A simple example: your raw Salesforce `opportunity` table has a column `amount` in USD. Your finance team needs it in EUR, joined with `account` and `user` tables, filtered to closed-won deals, and deduplicated. That logic lives in a dbt model called something like `fct_revenue`. When dbt runs, it materializes that model as a table in your warehouse that downstream BI tools like Looker or Tableau can query.

Orchestration is the layer that schedules and sequences everything. Tools like Apache Airflow (open source, maintained by the Apache Software Foundation) or the commercial offering from Astronomer, or the newer Dagster and Prefect, define when jobs run and in what order. An orchestrator ensures that your dbt transformation does not run before Fivetran has finished loading the latest data, and that your BI cache refresh does not happen before the transformation is complete. Without orchestration, you have a collection of jobs with no guaranteed sequence, and data freshness becomes unpredictable.

A concrete production scenario: a retail company runs Fivetran ingestion from Shopify and their ERP at 6am. Airflow detects that ingestion is complete, then triggers a dbt run that rebuilds twenty-odd models, including customer lifetime value calculations and inventory aggregates. By 7am, the executive dashboard in Looker reflects the previous day's data. The orchestrator handles retries if the Shopify connector is slow, and alerts the data team if anything fails.

When to use this stack, and when not to

This architecture fits well when your data sources are SaaS applications with existing connectors, your transformation logic is primarily SQL, your warehouse is already cloud-based, and your team has at least two or three engineers who are comfortable with Git. That describes a large share of mid-market and enterprise companies in 2026.

There are genuine cases where it does not fit as cleanly. If your transformation logic is heavily Python-based, for example because you are running ML feature pipelines or complex statistical models, dbt alone is insufficient. You will need to extend it with Python models or hand off parts of the pipeline to a different framework. dbt Labs has added Python model support (available on Snowflake and Databricks), but it is not as mature as the SQL layer and the documentation reflects that.

If your data volumes are small and your team is one analyst with no engineering support, this stack can be over-engineered. A single Airbyte instance, a few dbt models, and a cron job to run them is a legitimate architecture for an early-stage company. Airflow specifically has significant operational overhead and is a poor choice unless you have someone who can maintain it.

Cost is also non-trivial. Fivetran pricing scales with monthly active rows, which can become expensive as your data volume grows. This is vendor-disclosed pricing and the actual cost depends heavily on usage patterns. Several organizations have moved partially to Airbyte self-hosted or to newer tools like Estuary to manage that cost. Get your own numbers rather than relying on vendor case studies.

The stack works best when its three layers are treated as a system. When each layer is purchased or configured independently, without regard for how they hand off to each other, the operational burden multiplies. Orchestration is where that integration either holds together or falls apart.

Finished reading?

Validate your read to earn XP and feed your radar.