Measuring data quality with sector-specific metrics

The consequence: you cannot trust a dataset because it "has all the columns." You measure it.

The three dimensions, defined

These three come directly from the widely cited Kahn framework for EHR data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition →, adopted by research networks like OHDSI (Observational Health Data Sciences and Informatics). See the open-access paper on harmonized data quality assessment terminology.

1. Completeness

Are values present where they should be? A field can be absent (never collected) or missing (expected but blank). These are different problems.

2. Conformance

Do values follow the required format, type, and vocabulary? A HER2 field should hold a controlled value ("Positive," "Negative," "Equivocal"), not "positve" or "3+ maybe."

3. Plausibility

Are values believable given clinical reality? An adult patient weighing 6 kg fails plausibility even though the field is present and numeric.

A record can pass one dimension and fail another. That is the whole point of measuring them separately.

Scoring the messy oncology dataset

Let's work a concrete example. Our dataset: 4,200 breast cancer records with these key fields.

| Field | Issue found |

|---|---|

| tumor_stage | 38% blank |

| her2_status | free-text typos, 12% non-conforming |

| patient_weight_kg | 3% implausible values |

| diagnosis_date | 1% after death_date |

Completeness score

Completeness for one field:

For tumor_stage: 62 percent of 4,200 records have a value.

That is 62 percent. Below most analytics thresholds (research networks often target 90 percent or higher for core clinical variables, though there is no single universal standard).

But be careful: some blanks are legitimately "not applicable." For a patient whose cancer was caught pre-invasive, a full stage may not apply. So split completeness into "expected but missing" (a real problem) versus "not applicable" (fine). If 400 of those blanks are genuinely not applicable:

Now 68.5 percent. Still weak, but honest.

Conformance score

her2_status should mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.View full definition → to a controlled vocabulary. Twelve percent fail.

88 percent. The 504 non-conforming records are often recoverable: "positve," "POS," and "positive" can be normalized to one canonical value. Conformance failures are frequently the cheapest to fix because they are mechanical.

Plausibility score

Two checks here:

• Range check: patient_weight_kg between roughly 30 and 250 for adults. 3 percent fail.
• Temporal logic check: diagnosis_date must precede death_date. 1 percent fail.

Weight plausibility is 97 percent, date logic 99 percent. The date failures are red flags: they usually signal a data merge error, not a real outlier.

Turning scores into a readiness verdict

You do not average blindly. Weight the fields by how much your downstream analytics need them.

Suppose you are building a model to predict treatment response, and tumor_stage and her2_status are essential predictors. A field-level score of 68.5 percent completeness on tumor_stage is a blocker, not a footnote. No amount of clean weight data compensates for a missing key predictor.

A simple readiness rubric:

• Green (ready): all essential fields at or above 90 percent on every dimension.
• Amber (conditional): fixable conformance issues, or missingness that can be imputed or excluded transparently.
• Red (not ready): essential field completeness below threshold, or systematic plausibility failures suggesting a broken pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition →.

Our dataset is Red, driven by tumor_stage completeness. The her2 conformance issue is Amber (mechanically fixable). The date-logic failures need investigation before anything else, because they hint the merge itself is faulty.

That verdict is the deliverable. "Red because of stage completeness and unresolved date-logic errors" is far more useful than a single blended percentage that hides the fatal flaw.

🎬 [VIDEO: "Data QualityData QualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → in Healthcare: The OHDSI Data QualityData QualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → Dashboard" - youtube.com - walkthrough of an open-source tool that runs thousands of automated conformance and plausibility checks on health data]

Automating the checks

You would not eyeball 4,200 records. Here is the logic of a minimal conformance and plausibility check in Python-style pseudocode.

VALID_HER2 = {"Positive", "Negative", "Equivocal"}

def score_record(r):
    flags = []
    # Conformance
    if r["her2_status"] not in VALID_HER2:
        flags.append("her2_nonconform")
    # Plausibility: range
    if not (30 <= r["patient_weight_kg"] <= 250):
        flags.append("weight_implausible")
    # Plausibility: temporal logic
    if r["diagnosis_date"] > r["death_date"]:
        flags.append("date_logic_fail")
    return flags

Run this across the dataset, count flags per rule, and you have field-level dimension scores automatically. Tools like the OHDSI Data Quality Dashboard do exactly this at scale, running thousands of checks against a standardized data model.

Sector-specific benchmarks and standards

A few reference points for 2026, all to be treated as context, not hard rules:

• Common Data Models matter. Mapping to OMOP (Observational Medical Outcomes Partnership) or FHIR (Fast Healthcare Interoperability Resources, the HL7 standard for exchanging health data) lets you reuse published quality checks instead of inventing your own.
• Regulatory expectations exist. For data supporting drug or device submissions, the US FDA (Food and Drug Administration) and the EMA (European Medicines Agency) expect data integrity governed by ALCOA+ principles: Attributable, Legible, Contemporaneous, Original, Accurate, plus Complete, Consistent, Enduring, Available. Note the overlap: completeness and accuracy are regulatory expectations, not just analytics niceties.
• Real-world data scrutiny is rising. The FDA's Real-World Evidence framework explicitly discusses data reliability, which maps directly to conformance and plausibility.

There is no single legally mandated completeness percentage. What regulators require is that you document your quality assessment and your handling of defects. A transparent Amber is defensible. A hidden Red is not.

A note on medtech device data

The same three dimensions apply to device-generated data (a continuous glucose monitor, an infusion pump log).

• Completeness: gaps in a sensor time series (dropped readings).
• Conformance: timestamps in the expected timezone and unit.
• Plausibility: a glucose reading of 900 mg/dL that likely reflects a sensor fault, not a patient.

For high-frequency device data, plausibility often becomes the dominant concern, because sensors produce present, well-formatted, but physiologically impossible values.

Key Takeaways

• Measure three dimensions separately. Completeness, conformance, and plausibility catch different failures. A record can be complete but implausible, or well-formatted but empty where it matters.
• Score fields, not just datasets. A 68.5 percent completeness on tumor_stage (an essential predictor) makes the dataset Red regardless of how clean everything else is.
• Split "missing" from "not applicable." Adjusting the denominator turned 62 percent into an honest 68.5 percent and avoided punishing legitimate blanks.
• Conformance is often cheap, plausibility failures are often warnings. Typos normalize easily; a diagnosis_date after death_date usually signals a broken data merge.
• Document, do not hide. Regulators (FDA, EMA) and research networks (OHDSI) expect a transparent quality assessment. A defensible Amber beats a concealed Red.