# Running a privacy impact assessment before you launch
A county in Pennsylvania once built a predictive-risk model to flag which newborns were most likely to face maltreatment. It scored every birth against county records, welfare history, and jail data before a caseworker ever met the family. That model, the Allegheny Family Screening Tool, went live only after years of external review, precisely because someone asked the hard questions first: whose data is in here, who gets flagged disproportionately, and what happens if this leaks or errs.
That "asking first" process has a name: a Privacy Impact Assessment (PIA), a structured review of what personal data a system collects, why, and what could go wrong before it touches a single resident's record. In the US, PIAs are legally required for federal systems under the E-Government Act of 2002 whenever a new system handles personally identifiable information. In the EU, the equivalent is the Data Protection Impact Assessment (DPIA), mandated by Article 35 of the GDPR (General Data Protection Regulation) for any processing "likely to result in a high risk" to individuals, which explicitly includes large-scale profiling and automated decision-making about vulnerable groups like children.
If you are about to deploy a predictive-risk tool in child welfare, policing, benefits eligibility, or housing, a PIA is not paperwork. It is the last checkpoint before your model starts shaping real interventions in real families' lives.
A rigorous PIA walks through five concrete questions. Skip any one of them and you are flying blind.
1. What data goes in? Every source field: birth records, prior child welfare contacts, arrest records, Medicaid claims, school data.
2. Why do you need each field? This is the GDPR's data minimization principle (Article 5): collect only what's necessary for the stated purpose. Zip code might predict risk, but it's often a proxy for race and poverty, not causal risk itself.
3. Who can re-identify someone from this data? Even "anonymized" datasets often aren't.
4. What happens if the model is wrong, and for whom specifically? False positives and false negatives don't land evenly across groups.
5. Who is accountable when it fails? Name a human, not a committee.
Skipping straight to "let's build the model" without walking through these is the single most common failure mode in the public sector.
"Anonymized" is not a technical guarantee, it's a claim you have to test. A landmark study by Latanya Sweeney found that 87% of the US population could be uniquely identified using just ZIP code, birth date, and sex (Sweeney, 2000, Data Privacy Working Paper), an estimate still widely cited because the underlying combinatorics haven't changed.
For a child welfare tool, the practical test is a k-anonymity check: for any combination of fields you plan to release or share (age, zip, case type, referral source), does each combination match at least *kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →* people (commonly kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →=5 or kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →=10 as a working threshold, not a legal mandate)? If a query returns one household, you have a re-identification problem, even if you stripped the name and Social Security number.
A simple check you can run before any data leaves a secure environment:
import pandas as pd
# quasi-identifiers likely to enable re-identification
quasi_ids = ["zip_code", "age_bracket", "case_type"]
group_sizes = df.groupby(quasi_ids).size()
risky_groups = group_sizes[group_sizes < 5] # k=5 threshold
print(f"{len(risky_groups)} combinations have fewer than 5 people")
print(risky_groups.head(10))If that count is nonzero, you either suppress those rows, generalize the field (exact age to age bracket), or add noise before anyone downstream can query the dataset.
Re-identification protects individuals. Disparate impact protects groups, and in child welfare it's the higher-stakes risk.
Disparate impact is a legal concept from US civil rights law (originating in *Griggs v. Duke Power*, 1971, and applied to algorithmic decisions by the Department of Justice and HUD in fair housing contexts) meaning a facially neutral policy or model that produces significantly worse outcomes for a protected group, race, sex, disability, even without intent to discriminate.
The practical test: compute your model's flag rate and error rate by demographic subgroup, not just overall accuracy.
| Metric | Group A | Group B | Gap |
|---|---|---|---|
| Flagged as high-risk | 22% | 41% | +19 pts |
| False positive rate | 14% | 29% | +15 pts |
A 19-point gap in flag rates, if it tracks race or income and isn't explained by legitimate risk factors, is a red flag serious enough to halt launch. This is exactly the critique independent researchers raised against Allegheny's tool: family poverty and prior contact with the system (itself shaped by biased reporting) were baked into "risk" scores, meaning poor families got surveilled more, which generated more data, which justified more surveillance. A well-run PIA names this feedback loop explicitly rather than discovering it after deployment.
The Brookings Institution has a clear plain-language explainer on how these feedback loops form in predictive policing and welfare tools: Brookings: Algorithmic bias detection.
Vérification des acquis
1. What is the primary purpose of a Privacy Impact Assessment (PIA) when deploying a predictive-risk tool?
2. A predictive-risk model developer wants to include a family's zip code as an input feature because it correlates with the outcome being predicted. What concern does the data minimization principle raise about this?
3. Under GDPR Article 35, when is a Data Protection Impact Assessment (DPIA) specifically mandated?
4. Select ALL correct answers about why a PIA/DPIA should be conducted before launch rather than after.
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers about the legal frameworks requiring privacy assessments described in the lesson.
Sélectionnez toutes les réponses correctes.
A PIA is only as good as the governance structure behind it. Three things need to be in place before launch, not after:
For US state and local agencies, the NIST AI Risk Management Framework (nist.gov/itl/ai-risk-management-framework) gives a free, non-regulatory but widely-adopted structure for exactly this: mapping risks, measuring them, and assigning governance ownership.
Before any predictive-risk tool touches a resident's record, confirm:
🎬 [VIDEO: "Algorithms in Child Welfare: Risk, Bias, and Accountability" - youtube.com - search for panel discussions from the Data & Society or AI Now Institute on predictive tools in child welfare systems, which walk through the Allegheny case in detail]