# Building a de-identification and re-identification risk workflow
A researcher published an "anonymized" oncology dataset. Zip code, admission date, and rare tumor subtype were all still there. Within days, a graduate student cross-referenced it against a public obituary and a hospital press release, and named one patient. The data was technically stripped of names. It was not anonymous.
This lesson shows you how to avoid that outcome: how to de-identify a real oncology cohort under US and EU rules, then attack your own dataset to prove the anonymization holds.
Under the US HIPAAHIPAAHealth Insurance Portability and Accountability Act, loi américaine imposant la protection des données de santé (PHI). Violations : amendes jusqu'à 1,9M$ par catégorie de violation. (Health Insurance Portability and Accountability Act, the 1996 federal law governing protected health information), the HHS (Department of Health and Human Services) recognizes exactly two ways to de-identify data. This is set out in the Privacy Rule, and the reference document is worth bookmarking: HHS De-identification Guidance.
Remove 18 specified identifier types. Examples relevant to an oncology dataset:
Safe Harbor is mechanical. You either removed the 18 categories or you did not. No statistician required.
The catch: it can destroy the science. Oncology research often needs precise dates (time from diagnosis to progression) and granular geography (cancer cluster analysis). Safe Harbor strips both.
A qualified statistician certifies that the re-identification risk is "very small." This lets you keep more detail (for example, exact intervals between events instead of just years) in exchange for a documented, defensible risk analysis.
This is where re-identification risk math lives. And it is the method serious biotech and medtech teams use for high-value clinical datasets.
Under the EU GDPR (General Data Protection Regulation), the standard is higher. GDPR distinguishes:
The key regulatory reference is the Article 29 Working Party opinion on anonymization techniques (the Working Party is now succeeded by the EDPB, the European Data Protection Board). GDPR judges anonymization against three risks: singling out, linkability, and inference. Your workflow must address all three, not just remove names.
Practical takeaway: HIPAAHIPAAHealth Insurance Portability and Accountability Act, loi américaine imposant la protection des données de santé (PHI). Violations : amendes jusqu'à 1,9M$ par catégorie de violation. Safe Harbor is NOT automatically GDPR-compliant. A dataset legal to release in Boston may still be personal data in Berlin.
k-anonymity is the core metric. A dataset is kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →-anonymous if every record is indistinguishable from at least kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →-1 other records, based on its quasi-identifiers.
Quasi-identifiers are fields that are not direct IDs but combine to fingerprint a person: age, gender, zip code, admission year, diagnosis. A famous cited estimate (Latanya Sweeney's work, widely referenced) suggests roughly 87 percent of the US population was uniquely identifiable in 1990 census data using just ZIP + birth date + gender. Treat that as an often-cited estimate, not a current figure, but the lesson stands: three "harmless" fields can be a fingerprint.
If kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →=1 for a record, that record is unique on its quasi-identifiers. That is your re-identification risk.
Suppose an oncology cohort has these quasi-identifiers per patient: age_band, sex, state, cancer_type.
Count how many patients share each combination:
| age_band | sex | state | cancer_type | count (kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →) |
|---|---|---|---|---|
| 50-59 | F | CA | Breast | 42 |
| 60-69 | M | NY | Lung | 18 |
| 40-49 | F | WY | Ovarian, BRCA1 | 1 |
The first two rows are safe. The third row has k=1: one 40-something woman in Wyoming with a rare BRCA1 ovarian case. Even with her name gone, she is a unique record. Combine that with a local news story and she is re-identifiable.
The fix: generalize (state becomes region, cancer_type becomes Ovarian) or suppress the row entirely until kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète → reaches your threshold. A common target is k=5 or higher for released health data, though the right threshold depends on your risk assessment, not a universal rule.
Here is a minimal kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →-anonymity check. This is the "audit" step: you attack your own dataown dataData collected directly from your own customers and prospects through your own channels: your most reliable and privacy-compliant source.Voir la définition complète → before anyone else does.
import pandas as pd
quasi = ["age_band", "sex", "state", "cancer_type"]
# group by quasi-identifiers, count records per combination
group_sizes = df.groupby(quasi).size().reset_index(name="k")
# flag high-risk records where fewer than k=5 people share the profile
risky = group_sizes[group_sizes["k"] < 5]
print(f"Vulnerable equivalence classes: {len(risky)}")
print(f"Patients at risk: {risky['k'].sum()}")
print(risky.sort_values("k").head(10))If risky is non-empty, you are not done. You generalize or suppress, then re-run until every class meets your threshold.
kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →-anonymity alone is not enough. Two attacks defeat it:
For most oncology releases, aim for kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →-anonymity PLUS l-diversity on the sensitive outcome field.
🎬 [VIDEO: "Data Anonymization: kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →-anonymity, l-diversity and t-closeness explained" - youtube.com - a concise walkthrough of the three privacy models with worked examples]
Vérification des acquis
1. In the opening example, the oncology dataset had names removed but a patient was still identified by cross-referencing zip code, admission date, and rare tumor subtype. What core concept does this failure illustrate?
2. A cancer research team needs precise dates to measure time from diagnosis to progression and granular geography for cluster analysis. Which de-identification approach best fits their needs, and why?
3. Why is Safe Harbor described as 'mechanical' compared to Expert Determination?
4. Select ALL correct answers about how an oncology dataset must be handled to satisfy HIPAA Safe Harbor.
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers about the reasoning behind building a re-identification attack against your own de-identified dataset.
Sélectionnez toutes les réponses correctes.
The math is half the job. The other half is documentation and process, because regulators and IRBs (Institutional Review Boards, the ethics committees that approve human-subjects research) will ask for it.
A defensible workflow has these steps:
1. Data inventory. List every field. Classify each as direct identifier, quasi-identifier, or non-identifying clinical variable.
2. Method choice. Safe Harbor for low-value routine releases; Expert Determination for research datasets needing granular dates or geography.
3. Risk measurement. Run the kkThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.Voir la définition complète →-anonymity and l-diversity checks. Record thresholds and results.
4. Transformation. Generalize, suppress, or add controlled noise. Log every transformation.
5. Expert sign-off. For Expert Determination, the statistician documents assumptions and certifies "very small" risk. Keep this document; it is your audit trail.
6. Data Use Agreement (DUA). Even de-identified data usually ships with a contract barring re-identification attempts. HIPAAHIPAAHealth Insurance Portability and Accountability Act, loi américaine imposant la protection des données de santé (PHI). Violations : amendes jusqu'à 1,9M$ par catégorie de violation.'s Limited Data Set pathway (which keeps dates and some geography) REQUIRES a DUA.
7. Re-assessment trigger. Anonymization decays. New public datasets (genealogy databases, leaked breaches) can re-enable linkage later. Schedule periodic re-review.
Genomic data is the hardest case. A genome is inherently identifying: you cannot "de-identify" DNA the way you redact a zip code, because the sequence itself is unique. This is why controlled-access repositories like the NIH's dbGaP exist: the data is not released openly at all, it is gated behind approved-researcher agreements. When someone tells you their genomic dataset is "anonymized," be skeptical.
Medtech adds device telemetry. A pacemaker or continuous glucose monitor produces a data stream with timing patterns that can act as a behavioral fingerprint. Treat device serial numbers and high-resolution timestamps as quasi-identifiers.
This lesson is educational and not legal advice. Consult qualified counsel and a certified statistical expert before releasing any patient-level dataset.