# Data qualityData qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → metrics that matter: completeness, latency and lineage
A health plan's claims warehouse shows a 74-day average lag between when a patient fills a prescription and when that record is usable for analysis. A drug safety team pulling data for a signal detection exercise doesn't notice, runs the query anyway, and concludes an adverse event rate is declining. It isn't declining. The most recent 10 weeks of claims simply haven't arrived yet. This is how real-world evidence (RWE, data generated outside of clinical trials, from claims, electronic health records, or registries) goes wrong, not through bad science but through unmeasured data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI..
This lesson gives you three metrics to check before trusting any pharma dataset: completeness, latency, and lineage.
Most enterprise data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → frameworks (think Six Sigma-style defect counts or generic "null rate" dashboards) were built for retail transactions or financial ledgers. Pharma data has different failure modes:
Each source needs quality dimensions tailored to how it breaks.
Completeness measures the proportion of expected data fields, records, or time periods that are actually populated.
Example: in an oncology EHR dataset, a common weak point is biomarker test results (e.g., HER2 status in breast cancer). Industry estimates suggest structured biomarker fields are missing in roughly 30 to 50% of EHR-derived oncology datasets as of recent published RWE studies (estimate, varies widely by data vendor and cancer type). If your analysis depends on biomarker status to define a patient population, that missingness isn't a rounding error, it can invalidate your cohort definition entirely.
Worked example:
Say you pull 10,000 patient records for a lung cancer RWE study. You need EGFR mutation status (a biomarker guiding targeted therapy choice).
If the missingness is random, you might reweight or impute. If missingness correlates with something meaningful, say, community oncology clinics test less often than academic centers, then your remaining 62% is a biased sample, not a smaller version of the truth. Always check missingness *patterns*, not just missingness *rates*.
Latency is the lag between an event occurring and the data reflecting it in your dataset.
In US claims data, a commonly cited benchmark is that 70 to 90% of medical claims are submitted and adjudicated within 90 days, but full "run-out" (all claims fully processed, including late submissions and corrections) can take 6 to 12 months (estimate, based on typical payer claims lag patterns). European sources, such as pharmacy dispensing registries in the Nordics, often have shorter lag (weeks, not months) because of centralized national health systems, but hospital-based EHR data across the EU varies enormously by country and system maturity.
Why this matters concretely: if you're monitoring drug safety signals for a newly launched product, a 3 to 6 month reporting lag means your "current" dashboard is describing a market that existed two quarters ago. Regulatory bodies like the FDA (Food and Drug Administration) and EMA (European Medicines Agency) explicitly account for this in pharmacovigilance (drug safety monitoring) protocols by defining minimum observation windows before drawing conclusions from claims-based signals.
Lineage is the documented trail of where data came from, what transformations it underwent, and who touched it before it reached your table.
Without lineage, you can't answer basic audit questions: Was this diagnosis code mapped from ICD-9 to ICD-10 (International Classification of Diseases, the standard coding system for diagnoses) correctly? Did a vendor's "data refresh" silently change a field definition last quarter?
Regulatory scrutiny makes this non-optional. The FDA's guidance on real-world evidence (part of the framework under the 21st Century Cures Act) explicitly asks sponsors to document data provenancedata provenanceData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition → when submitting RWE to support regulatory decisions. In the EU, the European Health Data Space (EHDS) regulation, entering into force in phases through 2025 to 2027, sets requirements for traceability of health data used in secondary research. No lineage record, no credible submission.
Here's a lightweight way to score a dataset before you rely on it. Not a full audit, but a gate check.
Dataset: Regional EHR extract, diabetes cohort, n=45,000
1. Completeness (key fields: HbA1c, medication list, BMI)
populated_fields / expected_fields = 38,000 / 45,000 = 84%
2. Latency (event date vs. data availability date)
median_lag_days = 21
Benchmark: acceptable for chronic disease monitoring (<30 days)
Flag: NOT acceptable for acute safety signal detection
3. Lineage
- Source system documented? Yes (Epic EHR, single health system)
- Coding standard documented? Yes (ICD-10, RxNorm for meds)
- Transformation log available? No <- FAIL
Overall gate decision: CONDITIONAL PASS
Do not use for regulatory submission until transformation log is obtained.This kind of gate, run before analysis starts, catches the mistake from our opening scene before it becomes a wrong conclusion in a slide deck.
For a deeper reference on structured RWE data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → frameworks, the Sentinel Initiative's data quality documentation (an FDA-funded active surveillance system) is a genuinely useful public resource showing how a major regulator-grade system codifies these checks at scale.
Knowledge check
1. In the opening scenario, why did the drug safety team wrongly conclude that an adverse event rate was declining?
2. Why do generic enterprise data quality frameworks (e.g., Six Sigma-style defect counts) often fail when applied to pharma data?
3. A researcher wants to use a claims dataset to identify patients' primary diagnoses for a clinical accuracy study. What is the key risk to consider?
4. Select ALL correct answers about why EHR data quality varies significantly across hospital sites.
Select all the correct answers.
5. Select ALL correct answers describing what 'completeness' measures as a data quality metric in pharma datasets.
Select all the correct answers.
Once you score a dataset, your conclusions have to shrink to match what the data supports.
This is the core discipline: data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → metrics don't just grade a dataset, they define the boundary of what you're allowed to say with it.
🎬 [VIDEO: "Real-World Evidence: What It Is and Why It Matters" - youtube.com/results?search_query=real+world+evidence+fda+explained - search for FDA or ISPOR-produced explainer videos covering RWE data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → fundamentals and regulatory use cases]