# Measuring data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → with banking dimensions and DQDQThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → scorecards
A single wire transfer arrives with a blank IBAN field. That one gap can freeze a payment, trigger a sanctions screening exception, and end up as a red cell on a scorecard the Chief Data Officer (CDO) presents to the board next quarter. Data qualityData qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → in banking is not abstract. It is the difference between a payment that clears and one that sits in a suspense account.
This lesson shows you how to measure data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → using six standard dimensions, apply them to real payment records, and roll the results into the pass-rate scorecards that regulators and boards expect.
Banks run on data they did not always collect cleanly. A customer onboarded in 2009 may have a record missing fields that only became mandatory later. When that data feeds regulatory reports, the errors surface where it hurts.
The anchor here is BCBS 239, the Basel Committee on Banking Supervision's "Principles for effective risk data aggregation and risk reporting" (2013). It requires large banks to demonstrate accuracy, completeness, and timeliness in the data behind their risk reports. Supervisors in the EU (the European Central Bank) and the US (the Federal Reserve, OCC) actively check compliance. You can read the original principles for free at the Bank for International Settlements.
The practical takeaway: banks must *measure* quality, not assert it. That means dimensions and scorecards.
The industry converged on six dimensions, formalized in frameworks like the one from DAMA (Data Management Association). Here they are, each with a banking example.
Is the data present where it should be?
Example: On a SEPA (Single Euro Payments Area) credit transfer, the IBAN (International Bank Account Number) and BIC (Bank Identifier Code) fields must be populated. A record missing the IBAN fails completeness.
Does the data match reality?
Example: A counterparty name says "Acme Trading Ltd" but the registered legal entity is "Acme Trading GmbH." The field is complete but inaccurate. Accuracy is the hardest to measure because it needs a trusted reference (a "golden source"), such as a validated LEI (Legal Entity Identifier) registry.
Does the value follow the required format or rule?
Example: An IBAN has a country-specific length and a checksum. A German IBAN is 22 characters. If the checksum fails, the value is invalid even if the field is filled.
Does the same fact agree across systems?
Example: The customer's country of residence is "FR" in the core banking system but "France" spelled out in the AML (Anti-Money Laundering) screening tool, and "DE" in the CRMCRMCustomer Relationship Management: software and strategy to manage and analyse customer interactions throughout their lifecycle.View full definition →. Same customer, three answers.
Is each real-world entity represented once?
Example: The same corporate client appears as three separate records because of a merger and two spelling variants. Duplicate counterparties distort exposure calculations.
Is the data current and available when needed?
Example: A sanctions list update lands at 09:00 but the payments engine only refreshes at 18:00. For nine hours, screening runs on stale data.
A dimension is a concept. A DQ rule is the testable version. You cannot measure "completeness"; you measure "percentage of payment records where IBAN is not null."
Let's apply this to a payments dataset. Assume a batch of 100,000 outboundoutboundProactive outreach that pushes your message to targeted audiences through advertising, email, or direct prospecting, initiated by the seller rather than the buyer.View full definition → SEPA payments (illustrative figures for a worked example, not real bank data).
| Dimension | Rule | Records failing | Pass rate |
|---|---|---|---|
| Completeness | IBAN populated | 1,200 | 98.8% |
| Validity | IBAN passes checksum | 450 | 99.55% |
| Accuracy | Counterparty name matches LEI registry | 3,100 | 96.9% |
| Consistency | Country code matches across core and AML | 800 | 99.2% |
| Uniqueness | No duplicate counterparty ID | 600 | 99.4% |
| Timeliness | Record ingested within SLA window | 250 | 99.75% |
Completeness pass rate for IBAN:
Pass rate = (Total records - Failing records) / Total records
= (100,000 - 1,200) / 100,000
= 98,800 / 100,000
= 0.988 = 98.8%That is the atomic unit of a scorecard. Every green, amber, or red cell a board sees traces back to a fraction like this.
Validity is one of the few dimensions you can check with pure logic, no external source needed. The IBAN uses the ISO 13616 standard with a mod-97 checksum. Here is the core check in Python.
def is_valid_iban(iban):
iban = iban.replace(" ", "").upper()
# Move first 4 chars to the end
rearranged = iban[4:] + iban[:4]
# Convert letters to numbers: A=10, B=11, ... Z=35
digits = ""
for ch in rearranged:
if ch.isdigit():
digits += ch
else:
digits += str(ord(ch) - 55)
# Valid if the big number mod 97 equals 1
return int(digits) % 97 == 1
print(is_valid_iban("DE89370400440532013000")) # TrueRun this across a payments table and you have your validity failing-record count directly. No manual review needed. This is why validity rules are usually the cheapest quality wins.
A pass rate means nothing without a threshold. Banks set DQ thresholds per rule, often on a red / amber / green (RAG) basis. A common (illustrative) pattern:
Critical fields get stricter thresholds. A sanctions-relevant field like counterparty name might demand green at 99.9%, because a single missed match can mean a regulatory breach. A marketing-preference field can tolerate amber.
Applying this to our table: accuracy at 96.9% lands in amber. That is the story the CDO must explain, and the LEI reconciliation project that goes into next year's budget.
Knowledge check
1. Why does the lesson argue that data quality in banking must be measured rather than simply asserted?
2. A wire transfer arrives with a blank IBAN field. Which data quality dimension does this most directly violate?
3. Why is a customer record onboarded in 2009 a useful illustration of a data quality challenge?
4. Select ALL correct answers about the role of DQ scorecards and dimensions in banking.
Select all the correct answers.
5. Select ALL correct answers about why data quality is treated as a board-level topic in banking.
Select all the correct answers.
A scorecard aggregates rule-level results into something a board can absorb in 30 seconds. It typically rolls up in three layers.
Layer 1: Rule level. Individual pass rates, as in the table above.
Layer 2: Dimension level. Weighted average of rules within a dimension. If completeness has five rules, you average their pass rates (often weighted by field criticality).
Layer 3: Domain level. A single score for a data domain like "Payments" or "Customer." This is what appears on the executive dashboard.
Take the six pass rates above and apply equal weights for simplicity:
Domain score = (98.8 + 99.55 + 96.9 + 99.2 + 99.4 + 99.75) / 6
= 593.6 / 6
= 98.93%The Payments domain scores 98.93%, which is amber under our thresholds. Notice how the weak accuracy dimension (96.9%) drags the whole domain below green. Real banks weight critical rules more heavily, so a failing sanctions-relevant rule can pull the score down harder than an equal-weight average would.
🎬 [VIDEO: "Data QualityData QualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → Dimensions Explained" - youtube.com - a clear 10-minute walkthrough of the six dimensions with worked examples]
Boards do not want 300 rules. They want:
1. Domain scores with RAG status and the trend versus last quarter.
2. The top failing rules driving red status.
3. Remediation status: what is being fixed and by when.
4. Regulatory exposure: which failures touch BCBS 239 reports.
The scorecard is a governance instrument. It creates accountability by naming a data owner for each domain, usually a senior business leader, not IT. When Payments goes red, a named executive answers for it.
Averaging away the risk. A 98.9% domain score can hide a sanctions field at 92%. Always surface critical-field failures separately.
Counting the wrong denominator. If you measure IBAN completeness only on records where the field was submitted, you miss the ones dropped upstream. Measure against the full expected population.
Rules with no owner. A failing rule that nobody is accountable for stays red forever. Every rule needs a business owner and a remediation path.
Static thresholds. As regulatory expectations tighten, yesterday's green becomes today's amber. Review thresholds at least annually.