Scoring data quality: completeness, accuracy and timeliness in FMCG feeds

# Scoring data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.Voir la définition complète →: completeness, accuracy and timeliness in FMCG feeds

A national account manager at a mid-size snack brand pulls up her retailer's point-of-sale (POS) dashboard on a Monday morning and sees zero units sold for a top SKU (stock keeping unit, a unique product-and-package identifier) across 40 stores over the weekend. The product is on shelf. Shoppers are buying it. The scan data simply never arrived. This is a "late scan-out," and it is one of three data failures that quietly cost FMCG (fast-moving consumer goods, also called CPG or consumer packaged goods) companies millions in phantom stockouts, wrong replenishment orders, and misfired trade promotions every year.

This lesson builds the scorecard data teams use to catch these failures before they wreck a forecast.

The data that matters: core FMCG feeds

Three feed families dominate FMCG data operations:

1. Point-of-sale (POS) / scan data. Retailer registers capture unit sales, price, and timestamp per UPC (Universal Product Code, the barcode number identifying a specific product and pack size). Aggregators like NIELSENIQ and Circana (formerly IRI) compile this across retailers into syndicated panels.

2. Retailer/distributor inventory feeds. On-hand units by store or warehouse, usually sent via EDI (Electronic Data Interchange, a standardized format for B2B transactions) using formats like the EDI 852 (product activity data) or EDI 846 (inventory inquiry/advice).

3. Shipment and order data. EDI 850 (purchase order), 856 (advance ship notice), and 810 (invoice) transactions that tell a manufacturer what was ordered, shipped, and billed.

A fourth, growing category: loyalty and e-commerce clickstream data from retailer media networks (Walmart Connect, Kroger Precision Marketing, Tesco Clubcard data via Dunnhumby), which links purchase to shopper identity and browsing behavior.

Each feed has a different failure mode. POS feeds go stale or drop stores. Inventory feeds diverge from physical reality ("phantom inventory": the system says 12 units on shelf, the actual count is zero). Shipment data mismatches when a UPC gets remapped after a pack-size change and the old code lingers in one system.

Why this matters commercially

Phantom inventory triggers a specific, expensive loop: the system believes stock exists, so it doesn't trigger a replenishment order, the shelf stays empty, and the sale is lost. Industry research on out-of-stocks (a long-running stream from Gruen-era studies through more recent retail analytics) has repeatedly estimated retail out-of-stock rates around 5 to 8% of SKUs on any given day as an industry benchmark estimate; phantom inventory is a recognized contributor, though the exact share attributable to data error versus shelf execution varies by category and is not precisely quantified industry-wide.

The three-pillar quality scorecard

FMCG data teams score incoming feeds on three dimensions. Each gets a threshold; breach the threshold and the feed is flagged before it touches a forecast or replenishment system.

Completeness

Percentage of expected records actually received.

Formula: Completeness % = (records received / records expected) × 100

Worked example: A distributor should report inventory for 850 stores daily. Today's file contains 803 stores.

803 / 850 = 0.9447 → 94.5% complete.

Typical threshold (industry practice estimate): flag anything below 98% completeness for POS feeds used in demand forecasting; below 95% for secondary inventory feeds is a hard stop that blocks auto-replenishment triggers. These thresholds vary by company and are not universal regulation, they're operational norms set by each data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.Voir la définition complète → team.

Accuracy

Whether the values match ground truth. In FMCG this usually means:

• UPC match rate: percentage of scanned UPCs that correctly mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.Voir la définition complète → to the manufacturer's master product catalog (GS1's GDSN, the Global Data Synchronization Network, is the industry backbone for this mapping).
• Inventory variance: difference between system-reported on-hand units and physical cycle count.

Worked example: Cycle count finds 40 units on shelf; the inventory feed says 58.

Variance = |58 − 40| / 40 = 45% overstatement. That's phantom inventory, and it's severe.

Typical threshold: UPC match rate below 99% is commonly treated as a red flag requiring catalog reconciliation. Inventory variance tolerance is tighter for high-velocity SKUs (often under 5%) and looser for slow movers.

Timeliness

How fast data arrives relative to when the event happened.

Common benchmarks (estimates, vary by retailer agreement):

• POS scan data: expected within 24 to 48 hours of the sale (T+1 or T+2).
• EDI 852 inventory feeds: daily, often overnight batch.
• EDI 856 advance ship notices: within hours of shipment departure.

Worked example: A retailer's scan-out data for Saturday sales arrives Wednesday instead of Monday. That's a 2-day lag against a T+2 standard, a straightforward SLA (service level agreement) breach that should suppress that store's data from the current forecast run rather than let it silently skew the model.

A simple flagging rule in practice

Here's a minimal pseudocode pattern data teams use to auto-flag a feed before it enters a demand-planning model:

for each store_feed in daily_batch:
    completeness = records_received / records_expected
    variance = abs(system_qty - cycle_count_qty) / cycle_count_qty
    lag_days = today - feed_event_date

    if completeness < 0.98:
        flag("INCOMPLETE", store_feed)
    if variance > 0.05 and sku_velocity == "high":
        flag("PHANTOM_INVENTORY_RISK", store_feed)
    if lag_days > 2:
        flag("STALE_FEED", store_feed)

    if any_flag:
        exclude_from_forecast(store_feed)
        route_to_data_steward(store_feed)

This is deliberately simple. Real production systems (often built on platforms like Snowflake, Databricks, or retailer-specific EDI middleware) add weighting, store-level history, and statistical outlier detection, but the logic tree is the same: measure, threshold, quarantine, escalate.

Governance: who owns the fix

Scoring is useless without an escalation path. Most mature FMCG data organizations run a data governance council (cross-functional, usually sales operations, IT, and category management) that:

1. Owns the master data (product catalog, UPC assignments) in line with GS1 standards.

2. Reviews weekly quality scorecards by retailer/distributor partner.

3. Escalates chronic offenders (a retailer consistently below 95% completeness) to the commercial team for a data-quality conversation, since this is a trading-partner relationship issue, not just an IT ticket.

GS1's data qualitydata qualityThe degree to which data is fit for purpose: accurate, complete, consistent, timely, valid and unique. Poor quality data undermines analytics, reporting and AI.Voir la définition complète → guidance and the Voluntary Interindustry Commerce Solutions Association (VICS, now largely folded into GS1 US initiatives) historically provided shared standards for EDI transaction sets that make cross-retailer scorecards comparable at all. Without a shared standard, a "complete" feed from Retailer A and Retailer B might not even mean the same thing structurally.

Benchmarking your scorecard

There's no single global regulator setting FMCG data-quality thresholds (unlike, say, financial reporting under IFRS or GAAP). Benchmarks come from: