Okta moved its AI agents' context into dbt pull requests, and wrong answers became one-merge fixes

Okta's first AI analytics agents worked, and that was the problem. According to an account published by dbt Labs (a vendor whose tooling Okta chose, so treat this as a customer story rather than an audit), Okta's early AI experiments ran on semantic views built directly inside Snowflake's UI, wired straight to Snowflake Cortex agents. It worked, and it worked fast, but those semantic views were completely disconnected from the governed dbt models that actually defined the business logic. The team behind them is small. Crahen's team, about thirteen analytics engineers embedded across delivery, go-to-marketgo-to-marketThe strategy defining how you'll launch a product: target segments, channels, value proposition and coordinated action plan.View full definition →, product, and finance, sits inside a broader technology, data, and intelligence org that runs on Tableau for reporting and Snowflake as the central warehouse.

Pooja Crahen, Okta's senior manager of analytics engineering, explained the problem at dbt Summit 2026: "A couple years in, the realization came in where building the agent was never really the hard part," Crahen said. "What it could see and trust was the hard part." Her team found three cracks in that foundation: context fragmentation between Snowflake and dbt, no auditability since everything lived in UI click history, and all of an agent's instructions flattened into one undifferentiated blob.

Okta's position is what makes this case useful. The company sells identity governance for AI agentsAI agentsAgentic AI refers to AI systems that pursue goals autonomously by planning, taking actions through tools, and adapting based on results, with minimal step-by-step human direction.View full definition →. CEO Todd McKinnon told investors that among customers "over 90% of them have agents in production and only 22% of them are confident that they have them governed" (a vendor's own customer read, so not a neutral benchmark). Even so, Okta's internal data team didn't break down on access. It broke down on meaning. My thesis: an agent can hold exactly the right permissions and still give a confident wrong answer. For a CDO, governing AI risk means putting the context agents read under the same change control as the code that produces the numbers, keeping it in one repository, and publishing it where every agent can read it.

How Okta moved agent context out of the Snowflake UI and into dbt

The three cracks turned into concrete failures. Nobody could discover what existed, because semantic views had no catalog exposing them outside Snowflake; nobody could trust what they found, with no shared system of record confirming it was current; and every team ended up re-deriving its own definitions from scratch. That third failure is the one that should worry a CDO. When every team writes its own definitions, you end up with several versions of each KPIKPIKey Performance Indicator, a measurable value that shows how effectively you're achieving a specific objective, tracked over time against a target.View full definition →, and no one is responsible for any of them.

Three layers of context, one pull request workflow

The fix was a three-layer agent context framework, built entirely inside dbt and shipped through normal pull requests.

• At the asset level, model and column-level metadata gets a new "agent" sub-block in the YAML, flagging what a model is safe to use for, what to avoid it for, and any filters that need to apply automatically.
• At the company level, context docs, in markdown, capture company-wide facts (Okta's fiscal year starts February 1) and domain-specific routing rules.
• At the metric level, semantic views, migrated out of the Snowflake UI into dbt's own semantic layersemantic layerA shared business definitions layer that sits between raw data and reports, so everyone uses the same numbers for revenue, churn or margin.View full definition → spec, become the governed, PR-reviewed source of facts, dimensions, and metrics.

The migration was automated too. Getting semantic views into dbt took a custom skill wired to a slash command that pulls the DDL off an existing Snowflake semantic view and converts it into a dbt semantic view spec automatically. If you want to see how metric definitions turn into a single contract that every consumer queries, our lesson ondbt's semantic layer spec and governed metrics goes deeper.

Publishing context through Agents SchemaSchemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →

Keeping context in one repository only matters if agents actually read from it. At Okta, all three sync automatically into Agents Schema, an open, warehouse-native metadata standard, so any agent with SQLSQLSales Qualified Lead: a prospect the sales team has validated as ready for direct outreach and a proposal, having passed clear qualification criteria.View full definition → access, whether it's Cortex or Claude, reads the same definitions. In the open-source repository maintained by dbt Labs, after the first run, your warehouse has queryable metadata tables such as AGENTS.DBT_MODEL, AGENTS.LOOKML_VIEW, AGENTS.OMNI_VIEW, or AGENTS.OSI_DATASET. Agents can use those tables to understand which models and semantic objects exist. The standard is commercially backed: Fivetran and dbt Labs, now one company, announced that their new Context Layer uses Agents Schema, an open source standard, that centralizes context in a structured, extensible format directly in the data warehousedata warehouseA central repository that consolidates data from many source systems into a structured, query-optimized store designed for analytics, reporting, and business intelligence.View full definition →.

Okta also took the design question to the dbt community. It opened a public GitHub discussion on dbt Core asking whether something like a Cortex agent should just be a normal dbt model materialization, or deserves to be a first-class node type of its own, the way an exposure is. That question is worth watching. If agents become nodes in the DAG, lineage tools could show which agents a schema change would break.

Did governing context in dbt make Okta's agents more accurate?

Okta hasn't published an accuracy figure, so the honest answer is that we don't know by how much. What the company reports is consistency and faster fixes. According to Crahen, fixing a wrong answer no longer means editing an opaque agent config and redeploying through multiple cycles. An analytics engineer merges a change to the context layer in dbt, and every agent that reads from it gets smarter immediately. Her summary: "The result is the 100th question gets the same quality answer as the first," and "It's not about the agent. It's more about the context as infrastructure."

The roadmap shows what's still missing. Next up for the team: automated regression tests that run verified queries against the agents every time context changes, extending context beyond a single domain, and eventually checking agent answers against ground truth. So today, a context merge reaches every agent at once, but no automated test yet confirms that the merge improved anything. That gap is the next thing to close, because the same mechanism that spreads a fix instantly will spread a mistake just as fast.

Data Culture, a consultancy, measured what Okta hasn't yet. Its results also come through dbt Labs and its own blog, so they're vendor-adjacent. Data Culture started with a set of 13 golden questions, real business questions that stakeholders would ask, each with a verified correct answer, then added context one layer at a time. With the agent pointed at raw tables, the result was SQL generation: 0% | Accuracy: 0%. Later, their evaluation turned out to be the problem. Answers were marked wrong because the agent named a column TOTAL_APPOINTMENTS when we expected APPOINTMENT_COUNT. Or it returned 33.30% when we had written 33.3%. Once they fixed the comparison logic, accuracy jumped into the low 90s. We hadn't touched the agent at all. The figure dbt Labs reports for that deployment is 94.7%.

For a neutral baseline, Gartner predicts that through 2026, organizations will abandon 60% of AI projects unsupported by AI-ready data, and advises organizations to ensure continued investment and ongoing maturity in metadata management, data observabilitydata observabilityThe practice of continuously monitoring the health of your data so you know when it breaks, drifts or goes missing before it reaches a report or a model.View full definition →, and D&A and AI governance. Okta's setup is one concrete version of that advice.

What CDOs can copy from Okta's context-as-code model

A sorting rule for agent context

Okta's three layers give you a simple rule for deciding where each piece of context belongs. Put it on the whiteboard:

If it's true for the whole company, it's a context doc. If it computes a number, it's a semantic definition. If it limits how a table may be used, it's a model-level guardrailguardrailRules and controls that keep an AI system inside safe, legal and on-brand boundaries, blocking outputs and actions that cross the line.View full definition →. If it lives anywhere else, it's a risk.

Then check each item with four questions. Where does it live? Who approved it? When did it last change? Can every agent read it? Answers like "in a UI", "nobody", "not sure" or "only one agent" point to the cracks Okta found.

A worked example: Okta's fiscal calendar

Okta's fiscal year starts on February 1, and the company reports on that calendar. Its most recent results were for the second quarter fiscal year 2027, which reported total revenue of $805 million. Here's the arithmetic. If Q1 runs February to April, Q2 runs May to July 2026. An agent that doesn't know this and is asked for "Q2 revenue" will pull April to June. Only two of the three months overlap, so one-third of the period is wrong, and the agent will still return a precise-looking number. With a single line in a reviewed context doc, every agent gets the quarter right. Without it, each team has to remember to tell its own agent, and Okta found that teams don't reliably do that.

Where your situation differs from Okta's

Okta runs on one warehouse and has a mature dbt practice. If your metrics are spread across Snowflake, Databricks and a BIBITechnologies and processes that turn raw data into actionable insights via reporting, dashboards and analysis, so teams can decide based on facts rather than intuition.View full definition → tool's own semantic layer, the value of an open standard goes up. Agents Schema already lists dbt, Looker, Omni, OSI, Sigma, Markdown skills as sources. If you don't have dbt in production, start by getting your definitions version-controlled before you build agents on top of them.

The security point CDOs tend to skip is that a YAML "avoid" flag is guidance for the agent. It doesn't enforce anything. An agent that ignores its context can still query any table its role can reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition →. The real control is a warehouse grant, a row policy or an identity layer, and that's where Okta's commercial products come in: the company says each agent is treated as a "unique first-class identity" in Okta's Universal Directory, rather than as a user or service account. So pair every "avoid" flag with an enforced rule, followinga zero-trust approach to classification and access. Context governance decides whether an answer is right. Access control decides whether the agent was allowed to look in the first place.

This becomes more pressing as agents start writing transformation code. In news items dbt Labs published from its 2026 summit (vendor sources), Careem's agent writes and validates dbt models before a human reviews the PR, and DocuSign reported a 50% productivity gain from pairing Claude Code with dbt MCPMCPAn open standard that lets AI assistants connect to your company's tools and data in a consistent, governed way, instead of one custom integration at a time.View full definition →. Once agents can edit the context layer, review of context pull requests is a security control.

What to do on Monday about AI agentAI agentSoftware that pursues a goal on its own: it plans steps, uses tools and takes actions with limited human input.View full definition → context

• List every place agent instructions live today: warehouse UIs, system prompts, copilot configs, notebooks. Count them. Okta's problem started with that number being greater than one.
• Write a single markdown context doc for one domain. Include the fiscal calendar, customer definitions and routing rules, commit it to the repo, and assign a code owner.
• Add an agent metadata block to your ten most-queried models, stating what each is safe for, what to avoid, and which filters are mandatory.
• Write 13 golden questions with domain experts (the number Data Culture used) and run them by hand after every context change until you can automate the run.
• MapMapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.View full definition → each "avoid" flag to a real warehouse grant or row policy. Wherever no such rule exists, open a ticket with your security team.

Okta didn't solve agent trust with a better model or a new access tool. It made the definitions agents rely on reviewable, versioned and shared, so that one merge corrects every agent at once. The next step, which Okta hasn't built yet, is a regression test that proves each merge made answers better.