dbt State: lag_tolerance is now a budget decision

0:05Host:Leaders Insights. Today: lag_tolerance is a budget decision now, and most teams have not made it. Why are you paying to compute a number that didn't change?

0:14Expert:Because a cron expression told you to. I was in a review last month, a retailer, and we pulled their Tuesday morning query history. Something like 60% of their scheduled model builds that day produced a result byte-for-byte identical to the run before. Source tables hadn't refreshed. The SQL hadn't changed. The job ran because the clock said 6am.

0:37Host:And that's not an edge case, that's the warehouse.

0:40Expert:That's the warehouse. Open the history on any mature setup and a huge share of what you're paying for is recomputing yesterday's answer. dbt Labs claims more than 22 million models get built on their tooling every day, most of them identical to the previous run. That's their figure — they sell the tool, so take the number with a grain of salt — but the shape of it matches what I see in the wild.

1:04Host:So this episode is about the bill nobody's cutting. What actually changed to make this a story in October 2026 rather than a complaint we've had for years?

1:13Expert:dbt State went generally available in September. Announced at the dbt Summit in Las Vegas, the 16th. It turns "rebuild everything on a schedule" into "rebuild only what actually changed." And it runs where you live — Snowflake, BigQuery, Databricks, Redshift, under Airflow, Dagster, GitHub Actions. The tooling finally caught up to the obvious idea.

1:37Host:Obvious ideas that save money usually come with a catch. Where's yours?

1:42Expert:The catch is the whole job. dbt State can't decide how stale each model is allowed to be. That's a business decision wearing an engineering costume. There's a config called lag_tolerance — it's how long the system waits before rebuilding a model after its upstream data changes. Most teams set it once, globally, and walk away. Which means they've either overpaid or served stale numbers, and they don't know which.

2:08Host:Give me the number that makes a CFO sit up.

2:10Expert:RxBenefits — this is a dbt Labs customer result, published with the GA announcement, so again, vendor-sourced — their principal data engineer reported a 59% cut on scheduled-job warehouse costs on Snowflake. Worth $8,173 in the first sixty days. More than 700,000 models reused instead of rebuilt.

2:34Host:Run that forward for me.

2:35Expert:If the rate holds, you're looking at roughly $49,000 annualised. One team, one warehouse, mid-size pipeline. And the honest version is dbt's own range for early adopters was 15 to 30% compute savings. So don't anchor on 59. Anchor on your own history.

2:55Host:Which means step one isn't flipping a switch.

2:57Expert:Step one is metering the rebuild, not the invoice. On BigQuery you query INFORMATION_SCHEMA.JOBS — that's the system table that logs every query and what it scanned — filter on bytes processed, find the recurring jobs with no partition filter. On Snowflake you pull credit consumption split by job and by hour. You want exactly one number: credits spent rebuilding models whose upstream source didn't change. That's your addressable budget. Everything else is trying to move that number.

3:29Host:Say I've got the number. Now I have to decide staleness model by model. That sounds like a meeting that never ends.

3:36Expert:It's less philosophical than it sounds. The rule I put on the whiteboard: no scheduled rebuild without a decision waiting for it. For each model, name the decision it feeds and how often someone acts on it. A finance close model someone touches monthly does not need an hourly rebuild. A fraud screening feature table that gets acted on continuously does. Set the tolerance to the decision cadence, never shorter.

4:01Host:And the models where nobody can name the decision?

4:03Expert:Those are the fun ones. Either you give them a long tolerance window or you flag them for deletion. The models worth arguing about are the ones running hourly to feed a weekly decision. That's pure waste dressed up as diligence. Find those this week.

4:18Host:Here's the uncomfortable one. You turn on reuse, you stop rebuilding, and somewhere a source system quietly changes what a column means without changing the row count. You're now serving garbage that passes every volume check. How does this not blow up?

4:32Expert:That's the real risk, and I won't pretend it isn't. When a model gets reused, its tests don't run — the logic being they already passed. Defensible, until semantics drift under you. dbt added two guards earlier this year: a model that fails a data test gets rebuilt next run instead of reused, and a model whose table got deleted rebuilds even with no code change. But those don't save you from silent semantic drift.

4:59Host:So what does?

5:00Expert:Keep your source freshness checks running on their own schedule, regardless of reuse. They're the only thing standing between "efficient" and "confidently wrong." And before you trust any of it — audit mode. dbt shipped an Explain tab that shows why each resource was rebuilt, reused or cloned. Turn State on for one non-critical job, keep tolerances conservative, read the explanations for a week. You're checking one thing: did it reuse a table your controller expected to be fresh? If so, you just found a freshness promise nobody ever wrote down.

5:36Host:Let's say it works. Savings land. In my experience that money evaporates by the next quarter.

5:41Expert:Every time. Compute reductions vanish if nobody owns them. The 2026 State of FinOps survey — that's the FinOps Foundation, the practitioner community for cloud cost management, roughly a thousand responses covering over $83 billion in spend — found 78% of those teams now report into the CTO or CIO, not finance. Good. Put the recovered credits on a named budget line, review credits per domain and per job monthly. Otherwise it's a slide, not a saving.

6:14Host:There's a trap I can smell. If I'm on a committed-spend contract, my invoice doesn't even drop.

6:20Expert:Correct, and this bites people. On a Snowflake capacity contract you've pre-bought credits at a discount — vendor pricing guides put that at 15 to 40% off per-credit rates. Cut consumption 30% and the invoice stays flat. You're now under-consuming against money you already spent. So bank the efficiency, document the new consumption curve, and walk it into your renewal. The saving shows up at negotiation, not on next month's bill. Ask your account team what happens if consumption falls 25% — get the answer in writing before renewal season.

6:57Host:One more thing that keeps me up. You've been talking about scheduled pipelines, the predictable part. But everyone's wiring autonomous agents into their warehouse now. Those don't run on a clock.

7:09Expert:They don't, and that's the part of your bill that's about to get ugly. An agent decides what to query based on context, fans out in parallel, retries on failure with broader scope — all at 3am with nobody watching the meter. Flexera's 2026 cloud report — vendor survey, 750-odd decision-makers, read it alongside independent work — put self-estimated waste at 29%. First increase in five years. They blame AI workloads outpacing governance. I believe that part.

7:42Host:So how do you cap an agent that writes its own queries?

7:45Expert:Resource monitors on Snowflake with suspend triggers. maximumBytesBilled on BigQuery jobs — that's a hard ceiling on how much a single query is allowed to scan. Both cost nothing to set. They won't make your agents smarter, they'll just stop one runaway loop from printing you a five-figure surprise overnight.

8:03Host:Give me the honest framing before we close. Is this the end of the cost programme or the start?

8:08Expert:It buys you credibility, not permission to stop. That same FinOps survey is blunt that optimization is table stakes now, not a differentiator — the harder work is governance, forecasting, organizational alignment. A one-time 20% compute cut earns you the right to do that work. It doesn't replace it.

8:28Host:So if someone listening does exactly one thing Monday morning, what is it?

8:33Expert:Pull last month's scheduled transformation runs and calculate the share that rebuilt models whose upstream sources hadn't changed. One query. One number. That's your baseline, and until you have it, every vendor percentage you've heard today is somebody else's story. Get your own number first. Everything else follows from it.

8:52Host:The rebuild you're paying for that changed nothing. Measure that, and the rest is arithmetic. Sources for today's episode: dbt Labs (vendor — data tooling), MIT Technology Review, The New Stack, MIT Sloan Management Review. That's it. The CDO decision tools are live at mba-training.com.