Data observability: catching bad data before it reaches decisions

The cost of bad data reaching a decision is rarely a single catastrophic failure. It usually compounds quietly: a revenue forecast built on incomplete pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → data, a churn model trained on mis-labeled customer records, a board report citing numbers that the finance team can no longer reconcile. Gartner has estimated the average cost of poor data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → at $12.9 million per year for large organizations, and that figure predates the proliferation of real-time ML pipelines, where stale or corrupted data can propagate through dozens of downstream systems within minutes.

The pressure point in 2026 is the speed at which data moves. When pipelines ran overnight batch jobs, a data analyst could spot anomalies in the morning before anything consequential happened. Now data flows continuously into dashboards, into AI inference pipelines, into automated pricing and credit decisions. The window between data entering a system and that data influencing an action has collapsed. Observability is the discipline that fills the gap.

Building a data observability practice: a working sequence

Step 1: mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.View full definition → the decision surfaces, not the data assets

Most organizations approach data quality by cataloging tables and assigning owners. That's necessary but insufficient. Start instead with the decisions that matter most to the business and trace backward to the data that feeds them. At a retail bank, the decision surface might be credit limit adjustments triggered by an automated model. At a logistics firm, it might be dynamic route optimization. List five to ten such surfaces. For each one, document which tables, fields, and transformations are in the critical path. This narrows the observability problem to where failures actually hurt.

Step 2: instrument the five dimensions of data health

The industry has largely converged on five measurable dimensions: freshness (is this data arriving on schedule?), volume (are row counts within expected bounds?), schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → (have column types or names changed?), distribution (are value ranges and null rates stable?), and lineage (can you trace a number back to its source?). Tools like Monte Carlo (a vendor, with the associated commercial framing on benchmarks they publish) and open-source options like Great Expectations and Soda Core let you define expectations across these dimensions and alert on violations. The point is not to install a platform on day one. The point is to define, for each critical pipeline, what "normal" looks like and write that definition down in a form a machine can check.

Step 3: set thresholds based on decision impact, not data volume

A common mistake is treating all anomalies equally. A 2% drop in daily transaction rows in a reporting table used for quarterly reviews is very different from a 2% drop in the feature storefeature storeA centralised repository managing ML features, ensuring consistency between training and serving environments.View full definition → that feeds a real-time fraud model. Prioritize alert thresholds by downstream consequence. For pipelines feeding automated decisions, set tight bounds and route alerts to on-call engineers. For pipelines feeding weekly dashboards, a daily digest is sufficient. This prevents alert fatigue, which is the fastest way to kill an observability program.

Step 4: close the loop with data SLAs

Observability without accountability is a monitoring dashboard nobody checks. Formalize data SLAs with the teams that produce and consume critical data. These do not need to be lengthy legal documents. A one-page agreement specifying expected freshness, acceptable null rates, and escalation paths is enough. Airbnb's data engineering team has written publicly about using internal SLAs to reduce the mean time to detect and repair pipeline failures. The SLA creates a shared language: when an alert fires, everyone knows who owns the fix and what the acceptable resolution window is.

Step 5: integrate checks into the pipeline, not alongside it

Tests run as post-hoc checks on a finished pipeline catch failures after data has already landed in a target system. When possible, instrument checks at transformation boundaries so that a failure blocks propagation. In dbt (data build tool), this means using its native test framework to halt a run when a uniqueness or not-null test fails, rather than letting bad rows reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → production tables. The earlier in the pipeline a failure is caught, the cheaper it is to remediate.

Pitfalls that sink observability programs

The first is buying a platform before defining the problem. Vendors in this space, including Monte Carlo, Acceldata, and Bigeye (all commercial products), sell sophisticated anomaly detection. Those capabilities are genuinely useful, but teams that implement them without first mapping decision surfaces end up monitoring everything and acting on nothing.

The second is treating observability as a data engineering project. If business stakeholders don't know that a data SLA exists, they will not report failures, they will just lose trust in data silently and revert to spreadsheets. The CDO's job is to make data quality visible as a business metric, not a technical metric. Put failure rates and SLA compliance into the same review cadence as pipeline uptime.

The third pitfall is static thresholds on dynamic data. A retailer's transaction volume in December looks nothing like February. Anomaly detection built on fixed bounds will fire hundreds of false alerts in peak season. Use seasonality-aware baselines or, at minimum, review thresholds quarterly.

Quick wins to start this week

• Pick one high-stakes dashboard or model. Write down, in plain language, what the data feeding it should look like: expected row counts, key field null rates, refresh schedule. This is your first SLA draft.
• Add a Great Expectations suite or a dbt test block to a single production transformation. Even three or four basic tests create a detection layer that didn't exist yesterday.
• Pull the last six months of pipeline incident logs. Identify which three failures caused the most downstream confusion. Those pipelines are your priority instrumentation targets.
• Schedule a thirty-minute conversation between a data engineer and the business owner of a critical report to align on what "bad data" means for that specific use case.

Observability is not a product category. It is a set of practices that require deliberate design, stakeholder buy-in, and continuous calibration. A team that defines clear expectations, instruments the right checkpoints, and connects alerts to accountable owners will catch most failures before they reach a decision. That is the outcome worth building toward.