# Interoperability and shared standards across agencies
A caseworker in a county child welfare office flags a 9-year-old for chronic absenteeism risk. The school district's attendance system has the data. The child welfare agency has the case file. Both records describe the same child. Neither system knows it.
Why? The school district identifies the student by a state education ID. The welfare agency identifies him by a case management number tied to a different name spelling his guardian used at intake. No shared key exists to link the two records. Multiply this by every school, hospital, court, and housing authority a vulnerable family touches, and you get the defining data problem of the public sector: rich data trapped in incompatible silos.
This lesson covers the identifier and schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → standards that solve this, the governance metrics that track interoperability health, and the benchmarks used to judge whether a data-sharing effort is actually working.
Government systems were built agency by agency, often decades apart, funded by different grant streams with different rules attached.
Each law was written to protect one type of sensitive data, without a common technical or legal bridge between systems. The result: legally-mandated silos that predate any conversation about data interoperability.
This matters for fluency. When you hear "our systems can't talk to each other" in a public agency, it's rarely pure incompetence. It's usually a mix of legal constraint, legacy IT, and absent standards.
You cannot join two datasets about the same person, place, or case without a shared identifier or a reliable matching method.
Deterministic matching joins records on an exact shared key (a Social Security Number, a state student ID, a National Provider Identifier for clinicians). Fast and precise, but only works if the key is populated consistently and correctly, which in practice it often isn't.
Probabilistic matching (also called fuzzy matching or record linkage) scores similarity across multiple fields (name, date of birth, address, phone) when no shared key exists. It produces a match confidence score rather than a certainty. This is the workhorse technique for connecting welfare, health, and education records across agencies that never agreed on a common ID.
A simplified illustration of how a match score gets built:
match_score = (
0.4 * name_similarity # e.g., Jaro-Winkler string distance
+ 0.3 * dob_exact_match # 1 if identical, 0 if not
+ 0.2 * address_similarity
+ 0.1 * phone_match
)
# Threshold: match_score > 0.85 = auto-link
# 0.6-0.85 = flag for human review
# < 0.6 = no linkAgencies like the Actionable Intelligence for Social Policy (AISP) network at the University of Pennsylvania publish open toolkits on building these cross-agency "integrated data systems" responsibly.
Even with a matched identifier, two systems need to describe the *content* the same way. This is what schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → and interchange standards do.
Key standards professionals should recognize:
The pattern across all of these: a shared schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → turns a data *transfer* into a data *integration*. Without it, agencies exchange files that require manual re-mapping every time, which is expensive, slow, and error-prone.
Once a data-sharing pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → exists, you need metrics to know if it's actually working. Core ones used in public sector data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.View full definition →:
| Metric | What it measures | Why it matters |
|---|---|---|
| Match rate | % of records successfully linked across systems | Low match rate means silos persist despite the pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → existing |
| False positive link rate | % of matches later found incorrect | Wrongly linking two different children's welfare and school records is a serious harm, not just noise |
| Data latency | Time lag between an event and its availability in the shared system | A school attendance flag arriving 3 months late is useless for prevention |
| Field completeness | % of required fields populated (e.g., DOB, address) | Matching algorithms fail silently on missing fields |
| Consent/authorization coverage | % of data-sharing instances with documented legal basis | Required under FERPA, HIPAA, and most EU data protection regimes |
Under the EU's GDPR (General Data Protection Regulation), cross-agency sharing must document a lawful basis (Article 6) and, for special category data like health or welfare status, an additional condition under Article 9. Governance dashboards in EU public bodies typically track "legal basis documented" as a compliance metric alongside technical match rates.
A worked example: if a county integrated data system attempts to link 10,000 child welfare case records to a state education database and successfully matches 7,400, the match rate is 74%. If a later audit finds 220 of those matches were incorrect (wrong child), the false positive rate is roughly 3% (220/7,400). Both numbers should be reported together. A high match rate with a hidden high error rate is worse than a modest match rate you can trust.
Knowledge check
1. In the child welfare example, why don't the school district and welfare agency records automatically link despite describing the same child?
2. According to the lesson, what is the primary reason data silos exist across government agencies?
3. Why is a shared identifier considered the 'core building block' for interoperability between agency datasets?
4. Select ALL correct answers about laws that contribute to legally-mandated data silos in the US public sector.
Select all the correct answers.
5. Select ALL correct answers that explain why 'our systems can't talk to each other' is often heard in public agencies.
Select all the correct answers.
There's no single global benchmark for interoperability, but reference points help calibrate expectations.
When evaluating an agency's interoperability claim, ask three fluency questions:
1. What identifier or matching method underlies the join?
2. What's the documented match rate and error rate, not just "we share data now"?
3. What is the legal basis, and who audits it?