Your data pipelines are breaking. Not because of bugs. Because nobody agreed on what the data should look like before it was built.
A data contract is the solution, and it's one of the most practical governance innovations of the last five years.
In a typical data organization, the journey of data from source to consumer looks like this:
1. A production engineer adds a new field to the checkout APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → response
2. They forget (or don't know) that a downstream data pipelinedata pipelineETL (Extract, Transform, Load) is a data integration process that pulls data from sources, reshapes it into a consistent format, and writes it into a target system. depends on the existing
3. The pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → breaks silently, it processes the data but produces incorrect results
4. Three weeks later, a business analyst notices revenue numbers look wrong
5. The data team spends two days debugging across six systems to find the source
This happens everywhere, every week, at every organization with a non-trivial data infrastructure. It's not a people problem. It's an architecture problem: there is no formal agreement between data producers (engineering teams that build source systems) and data consumers (data teams that build pipelines and analytics).
A data contract is that agreement. It specifies:
When the production engineer adds that new field, the data contract is updated. The change triggers automated tests. The pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → is updated before it breaks. The analyst's dashboard never sees the problem.
Knowledge check
1. According to the lesson, what is the root cause of data pipelines breaking in typical organizations?
2. In the scenario described in the lesson, how long did it take for the broken pipeline to be noticed by a business analyst?
3. Why is storing a data contract in a Git repository (version control) particularly valuable for governance?
4. Select ALL elements that a data contract specifies according to the lesson:
Sélectionnez toutes les réponses correctes.
5. Select ALL correct statements about the example data contract structure presented in the lesson:
Sélectionnez toutes les réponses correctes.
A data contract is typically a YAML or JSON file stored in version control (Git), alongside the code that produces the data. A simplified example would specify:
When this contract is violated, order_amount_eur has null values, or freshness exceeds 15 minutes, an alert fires. The producer team is notified. Consumers aren't surprised by degraded data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition →; they're notified of the violation.
Shopify implemented an internal data contracts program at scale across their data platform. Their approach: any dataset served through their internal data platform must have a contract. Teams that consume data without a contract cannot hold producers accountable for quality. Teams that produce data without a contract cannot claim downstream consumers are using it correctly.
The outcome: a significant reduction in pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → incidents caused by schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → drift, a clearer accountability model for data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition →, and faster root-cause analysis when issues do occur.
Don't try to contract everything at once. Identify the five most business-critical data flows, the ones where a failure causes the most pain. Contract those first. Use the experience to refine your contract template and your change management process. Then scale.
Tools in this space: Soda (contract testing), Great Expectations (data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → validation), Gable.ai (data contract management platform), dbt contracts (built-in to dbt for transformation layer). The tooling is maturing rapidly, choose based on where in your stack you want to enforce contracts.