Data qualityData qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → failures have a geography: they almost always originate at the source.
A report shows wrong revenue numbers. The investigation traces back through the Data warehouseData warehouseA central repository that consolidates data from many source systems into a structured, query-optimized store designed for analytics, reporting, and business intelligence.View full definition →, the ETL pipeline, the staging database, the integration, and finally to a source system that started sending malformed data six weeks ago. Six weeks of bad data in production. Six weeks of decisions made on incorrect information.
The "shift-left" principle borrows from software engineering: catch defects as early as possible in the development process, because fixing a bug in production is 100x more expensive than catching it in code review. Applied to data: catch quality issues at the source, not after the fact.
At the source system: Validation rules built into the application that produces the data. If a field cannot be null, the application enforces it, the Data pipelineData pipelineETL (Extract, Transform, Load) is a data integration process that pulls data from sources, reshapes it into a consistent format, and writes it into a target system.View full definition → never sees null values because they never enter the system.
At the ingestion layer: Data qualityData qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → checks run immediately when data enters your infrastructure. If the schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → doesn't match the contract, the pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → stops. If completeness drops below the SLA, an alert fires. Great Expectations, Soda, and Monte Carlo all operate here.
At the transformation layer: dbt (data build tool) has built-in testing: not-null tests, unique tests, referential integrity tests, accepted-value tests, custom SQLSQLSales Qualified Lead: a prospect the sales team has validated as ready for direct outreach and a proposal, having passed clear qualification criteria.View full definition → tests. Every dbt model should have tests. A dbt run that includes failing tests should not deploy to production. Many organizations run their dbt tests in CI/CD pipelines, no untested transformation reaches the Data warehouseData warehouseA central repository that consolidates data from many source systems into a structured, query-optimized store designed for analytics, reporting, and business intelligence.View full definition →.
At the serving layer: Dashboards and reports that expose data to business users should include data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → indicators: "Last refreshed: 2 hours ago. Quality score: 94%. Known issues: 0."
Knowledge check
1. What is the core principle behind 'shift-left' data quality?
2. Why does the lesson argue that catching a data quality issue at the source is preferable to catching it in a production report?
3. In the transformation layer, what is the recommended best practice regarding dbt tests and deployment?
4. Select ALL statements that correctly describe where shift-left data quality controls operate.
Select all the correct answers.
5. Select ALL practices that reflect building data quality into a CI/CD pipeline.
Select all the correct answers.
The gold standard for shift-left data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition →: data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → tests run in CI/CD pipelines, failing builds are blocked from deployment, and quality metrics are tracked in the same dashboard as engineering metrics.
This requires:
1. Test coverage for data: Every critical pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → has documented quality tests. Tracked as a metric: "Percentage of data assets with quality tests: 67%." The CDO should set a target, say, 90%, and track it quarterly.
2. Automated validation on merge: When a data engineer submits a pull request that changes a pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition →, automated tests run against a sample of production data. SchemaSchemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → changes that would break downstream contracts fail the build before merge.
3. Quality gates for promotion: Data doesn't move from staging to production without passing quality checks. This is standard in software engineering (you don't deploy broken code). It should be standard in data engineering too.
Airbnb built an internal framework called Minerva to solve a specific problem: hundreds of analysts were defining the same metrics differently, creating inconsistency that undermined trust in data.
Minerva is a metrics layer, a central repository where business metrics are defined once (by the business, with the data team), and consumed consistently across all BIBITechnologies and processes that turn raw data into actionable insights via reporting, dashboards and analysis, so teams can decide based on facts rather than intuition.View full definition → tools, data science models, and experiments.
The key insight: Minerva shifts "what does this metric mean?" from ad hoc analyst judgment to a governed, version-controlled definition. When a business metric changes (new return policy changes how revenue is counted), the definition is updated in one place and flows to all consumers.
This is shift-left data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → applied to semantic consistency rather than technical quality. It's one of the highest-leverage data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.View full definition → investments Airbnb has made, and the pattern is being replicated by organizations using dbt Semantic Layer, Looker measures, and other metrics-layer approaches.
These four metrics, tracked monthly, tell you whether your shift-left program is working.
These actions are compiled in the role's Playbook.