Running a data audit before a regulatory filing

The core audit checklist

A pre-filing data audit is not a spot check. It is a structured walk from raw source to submitted number. Six steps matter most.

1. Confirm data lineagedata lineageData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition → end to end

Data lineageData lineageData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition → means the documented path a data point takes from its origin (a smart meter reading) through every transformation to its final form in the filing. For DR filings, lineage typically runs: meter → utility meter data management system (MDMS) → aggregator platform → baseline calculation engine → regulatory report.

Ask: can you point to the exact system and timestamp where each number was created, and every place it was touched afterward? If the answer requires guessing, the lineage is broken.

2. Reconcile source counts against filed counts

A simple but powerful check: count the number of meters, intervals, or customer accounts at the source and compare to what appears in the final filing.

Forty-four missing meters is not automatically a problem, but it must be explained (decommissioned meters, opted-out customers, data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.View full definition → exclusions). An unexplained gap is a lineage failure waiting to be flagged by a regulator's own reconciliation.

3. Flag undocumented transformations

Estimation, interpolation, and outlier removal are normal in meter data pipelines, smart meters occasionally miss intervals due to communication failures. The problem is not the transformation itself, it is the absence of a record.

Every transformation should log: what rule was applied, when, by what system or person, and why. A common failure mode: a vendor's estimation algorithm silently fills missing intervals using a default method, and the utility's internal team never sees a flag distinguishing "measured" from "estimated" data. Regulators increasingly ask for this distinction explicitly. PJM's Demand Response Operations manual, for example, requires baseline methodology disclosure precisely because estimation choices materially change settlement amounts.

4. Test baseline methodology consistency

DR baselines are typically calculated using methods like the "10-in-10" (average of the highest 10 usage days out of the prior 10 similar days) or regression-based weather-normalized baselines. Confirm the same methodology version was applied consistently across the entire filing period.

A frequent error: a mid-year software update changes a rounding rule or a weather-adjustment coefficient, and half the filing period reflects the old method while the other half reflects the new one. This is a governance issue, not just a technical one, someone approved the change without triggering a data recheck.

5. Validate against independent data

Cross-check filing data against an independent source: interval data from the wholesale market operator's settlement system, weather station data from a public source like NOAA's Climate Data Online, or third-party M&V reports. Discrepancies above a defined threshold (commonly 2 to 5%, though this varies by program) should trigger investigation before submission, not after a regulator's audit finds it.

6. Document the audit trail itself

The audit needs its own record: who ran which checks, when, what was found, what was remediated, and who signed off. This is the artifact that demonstrates good faith compliance if a regulator later questions the filing. Under frameworks like REMIT, the ability to reconstruct not just the data but the review process is itself part of demonstrating compliant conduct.

Governance roles: who owns what

A clean audit depends on clear ownership, not just a checklist.

• Data stewards (often within IT or a dedicated data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.View full definition → office) own the technical lineage documentation and metadata standards.
• Compliance or regulatory affairs teams own the interpretation of what the regulator requires and sign off on the final filing.
• Business unit analysts (DR program managers, market operations) own the domain judgment: does this number look right given what we know about the program.

Utilities that separate these roles cleanly tend to catch errors earlier. Utilities that collapse them into one overworked analyst tend to find the 12%-backfilled-data problem three weeks before deadline, or worse, after submission.

A practical governance artifact: the data dictionary

One underrated tool: a maintained data dictionary, a document defining every field in the filing (source system, unit of measure, calculation method, owner, last review date). When a regulator or auditor asks "how was this number derived," the answer should be a lookup, not a scramble. FERC and European regulators alike have increasingly cited the absence of clear field-level documentation as a contributing factor in enforcement actions, because it signals the filer itself may not fully understand its own submission.

🎬 [VIDEO: "Data LineageData LineageData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition → Explained" - youtube.com - a concise walkthrough of what data lineagedata lineageData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition → means in practice and why it matters for audit and compliance work]

What penalties actually look like

Penalties vary widely by jurisdiction and severity, and figures should be treated as illustrative rather than universal. FERC has authority to impose civil penalties for market manipulation and reporting violations under the Federal Power Act, with maximum statutory penalties in the range of roughly $1.4 million per violation per day (this figure is periodically adjusted for inflation, so treat it as an estimate and check FERC's current guidance). ACER and national European regulators can refer REMIT breaches for penalties determined at the member-state level, which vary considerably by country. The exact dollar or euro figure matters less than the underlying lesson: penalties scale with the number of days and the number of affected data points, so a lineage gap discovered late is far more expensive than one caught in a pre-filing audit.

Key Takeaways

• A pre-filing data audit means tracing every reported number back to its source system and confirming every transformation in between is documented, not just checking that totals look reasonable.
• Undocumented estimation or interpolation (like a vendor silently backfilling missing meter intervals) is the single most common trigger for compliance findings in demand-response and energy market filings.
• Reconcile source counts against filed counts, validate against an independent dataset (weather, market settlement data), and confirm baseline methodology was applied consistently across the full filing period.
• Clear ownership across data stewards, compliance teams, and business analysts, plus a maintained data dictionary, turns "can you explain this number" from a scramble into a lookup.
• Regulatory penalties under frameworks like FERC's Federal Power Act authority or the EU's REMIT scale with the severity and duration of the issue, so catching lineage gaps before submission is materially cheaper than after.