# Data qualityData qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → metrics that fintechs actually track
A single duplicated customer record cost a European neobank an estimated €300,000 in a 2023 regulatory remediation exercise, according to industry postmortems circulating at data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.View full definition → conferences. Not because the record was malicious. Because nobody caught it before it hit production.
This lesson puts you in that seat. You'll work through a sample onboarding dataset, calculate the four metrics every fintech team lives by, and make the call: ship this feed, or block it.
Fintechs run on data pipelines that feed KYC (Know Your Customer, the identity verification process required before opening a financial account), credit decisioning, fraud scoring, and regulatory reporting. Bad data doesn't just look sloppy on a dashboard. It triggers false declines, wrong credit limits, and compliance failures reportable to regulators like the CFPB (Consumer Financial Protection Bureau, US) or under the EU's DORA (Digital Operational Resilience Act, effective 2025).
Unlike a retail analytics team where a dirty field means a bad chart, a fintech data error can mean onboarding a sanctioned individual or miscalculating someone's affordability. The stakes reshape which metrics matter.
Completeness: the percentage of required fields that are actually populated.
Formula: (filled required fields / total required fields) × 100.
Freshness (also called timeliness): how current the data is relative to when it's needed. Measured as the lag between an event happening and the data reflecting it, often in minutes or hours for fraud data, days for credit bureau refreshes.
Accuracy: the percentage of records that match a trusted source of truth (a government ID database, a bank's core ledger, a credit bureau file).
Duplication rate: the percentage of records that are exact or near duplicates of another record in the same dataset, often the same customer with a typo'd email or a re-submitted application.
Imagine a digital bank's new-customer onboarding feed for one day. Sample size: 1,000 records.
Completeness check. Required fields: legal name, date of birth, government ID number, address, email. Audit finds 940 records with all five fields populated.
Completeness = 940 / 1,000 × 100 = 94%
Freshness check. KYC data should sync with the identity verification vendor within 2 hours of submission. Log analysis shows the average lag today is 3.5 hours, with 150 records exceeding 6 hours.
Freshness breach rate = 150 / 1,000 × 100 = 15% of records missed SLA (service level agreement, the target response time agreed with the vendor or internal team)
Accuracy check. Sampling 200 records against the national ID registry (a common audit method since checking all records is costly), 8 have a mismatched ID number.
Accuracy = (200, 8) / 200 × 100 = 96%
Duplication check. Fuzzy matching (comparing name, DOB, and email with tolerance for typos) flags 35 likely duplicate applicants.
Duplication rate = 35 / 1,000 × 100 = 3.5%
These thresholds vary by firm and are not universal regulatory mandates, but the following are commonly cited estimates from fintech data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.View full definition → practitioners as of 2025:
| Metric | Typical target | Notes |
|---|---|---|
| Completeness | ≥ 98% for KYC-critical fields | Lower tolerance than marketing data |
| Freshness | Fraud data: minutes; credit bureau data: 24 to 48 hours | Real-time payments (like FedNow in the US or SEPA Instant in the EU) push this toward near-zero lag |
| Accuracy | ≥ 99% against source of truth for identity fields | Below this, false-positive/negative KYC decisions rise sharply |
| Duplication | ≤ 1% | Higher rates suggest a broken dedup pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → at signup |
Our sample dataset breaches every single one of these. Completeness is 4 points under target, freshness has a 15% SLA miss rate, accuracy is 3 points short, and duplication is 3.5x the target.
Not all breaches carry equal weight. This is the judgment call a data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → team makes daily.
Duplication (3.5%): annoying, costs marketing spend and creates messy dashboards, but rarely blocks a launch on its own. Fixable downstream with a merge job.
Completeness (94%): concerning if the missing fields are KYC-critical (government ID, address). If the 6% gap is concentrated in optional fields like a middle name, it's tolerable. If it's missing ID numbers, this alone should block the feed, because you cannot legally onboard without verified identity under BSA/AML rules (Bank Secrecy Act / Anti-Money Laundering, US) or the EU's AMLD (Anti-Money Laundering Directive).
Freshness (15% SLA miss): matters enormously for fraud scoring feeds, less for a monthly reporting extract. Context-dependent.
Accuracy (96%): this is usually the hardest blocker. A 4% mismatch rate against the national ID registry means roughly 1 in 25 applicants has a wrong identity field in production. That's a direct KYC/AML exposure.
The call: block the feed on accuracy first, completeness second (if concentrated in ID fields), and treat freshness and duplication as monitored-but-shippable with a remediation ticket. This mirrors how real fintech data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.View full definition → teams triage: legal/compliance exposure beats operational tidiness.
Knowledge check
1. Why does a duplicated customer record carry higher stakes for a fintech than for a typical retail analytics team?
2. A fraud detection model needs transaction data within minutes of an event, while a credit decisioning process only needs bureau data refreshed every few days. This difference in acceptable lag time is best described by which metric?
3. A dataset has every required field filled in for 100% of records, but 15% of the values don't match the government ID database used as the source of truth. Which metric captures this specific problem?
4. Select ALL correct answers about how completeness and accuracy differ as data quality metrics.
Select all the correct answers.
5. Select ALL correct answers about why fintechs treat data quality metrics as a compliance issue, not just an engineering one.
Select all the correct answers.
Fintechs pull from a specific set of sources you should recognize:
Each source has different native quality characteristics. Bureau data is highly accurate but stale by days. Open banking feeds can be fresh but inconsistent across banks. Knowing the source's inherent quality profile shapes what threshold is realistic, not just aspirational.
Data qualityData qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → teams often automate these checks. A minimal example in pandas-style pseudocode:
def completeness(df, required_cols):
filled = df[required_cols].notna().all(axis=1).sum()
return round(filled / len(df) * 100, 1)
def duplication_rate(df, match_cols):
dupes = df.duplicated(subset=match_cols).sum()
return round(dupes / len(df) * 100, 1)
completeness_score = completeness(onboarding_df, ["name","dob","id_number","address","email"])
dup_score = duplication_rate(onboarding_df, ["name","dob","email"])Real production systems (using tools like Great Expectations or Monte Carlo) run these checks on every batch and alert automatically when a threshold is breached, rather than relying on a manual daily audit like our worked example.
For a deeper open-source reference on setting up these checks, see the Great Expectations documentation, a widely used open-source data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → framework.
🎬 [VIDEO: "Data QualityData QualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → Fundamentals" - youtube.com/results?search_query=data+quality+fundamentals+fintech - search this term for current practitioner walkthroughs on completeness, accuracy, and monitoring pipelines in financial data contexts]