Where real estate data actually comes from, and where it breaks

# Where real estate data actually comes from, and where it breaks

A single office building in Dallas can exist as five slightly different "facts" at once: one square footage in the county assessor's file, another in the CoStar listing, a third in the property management system, a fourth in the tenant's lease abstract, and a fifth in the loan file sitting at a bank. None of these are necessarily wrong. They were just measured differently, updated at different times, and never reconciled. That gap is where real estate analytics quietly goes wrong.

This lesson follows one asset's data trail end to end, then gives you the vocabulary to spot where it degrades.

The trail: one asset, five records

Picture a 150,000 square foot suburban office building. Here is how its data gets created and re-created.

1. County assessor and recorder records. Local governments record deeds, ownership transfers, and tax assessments. In the US this is county-level and wildly inconsistent in format; some counties still use PDF scans of paper forms. Europe's equivalent is the land registry (for example, HM Land Registry in England and Wales, or the Grundbuch in Germany), generally more standardized but still national, not pan-European. Square footage here often reflects assessed "gross building area," a tax definition, not a leasing definition.

2. MLS feeds (Multiple Listing Service). MLS systems were built for residential brokers to share listings cooperatively. Commercial real estate has no single equivalent; coverage is fragmented across regional commercial MLS boards, brokerage-proprietary systems, and marketplaces like Crexi or LoopNet. Data freshness depends entirely on whether a broker remembers to update status after a deal closes.

3. CoStar and other commercial data platforms. CoStar aggregates listings, comps, ownership, and space availability through a mix of licensed feeds, broker submissions, and a large field research team that calls buildings to verify occupancy. This human-verification layer is CoStar's competitive moatmoatA lasting edge over competitors: a resource, capability or position they cannot easily replicate, letting a firm earn above-average returns over time.View full definition →, but it means the "truth" you see is really "what a researcher last confirmed," timestamped, not necessarily current. Competing platforms (Cushman & Wakefield's internal tools, JLL's, or newer entrants like Reonomy and Cherre) pull from overlapping but non-identical source sets.

4. Lease abstraction. When a lease is signed, someone (increasingly an AI tool, historically a paralegal or outsourced abstraction service) extracts key terms: rentable square footage, rent steps, renewal options, expense pass-throughs. This becomes the operational record the landlord's asset management team relies on. It is frequently the *most* accurate square footage figure, and it is the one least visible to outside data platforms.

5. The loan file. Lenders and appraisers pull from all of the above, then add their own measurement (often per ANSI/BOMA standards; BOMA is the Building Owners and Managers Association, which publishes the standard method for measuring floor area) for underwriting.

Five records, five plausible square footage figures, no single source of truth.

Where duplication enters

Duplication happens when the same underlying event gets recorded independently by systems that do not talk to each other.

• A lease renewal gets entered into the property management system *and* separately re-reported to CoStar by the broker, sometimes with a different effective date.
• A sale gets recorded at the county recorder, then again days or weeks later when CoStar's research team confirms it by phone, then again when a broker manually logs it as a "comp" in their own tracker.
• Address matching creates phantom duplicates: "500 N Main St" and "500 North Main Street" can register as two separate assets in a dataset unless normalized.

The fix in professional data teams is entity resolution: matching records that refer to the same real-world asset despite different spellings, IDs, or formats, usually using a unique identifier. In the US, some firms use the parcel number (APN, Assessor's Parcel Number) as an anchor; there is no universal building ID equivalent to a stock ticker.

Where lag enters

Lag is the gap between when something happens and when your dataset reflects it.

• County assessment rolls in the US typically update annually, sometimes with a 6 to 18 month lag between a physical change (like a renovation) and its appearance in tax records.
• CoStar's occupancy and availability data is only as fresh as the last research call; a common estimate is that meaningful commercial vacancy shifts can take one to two quarters to fully surface across the market.
• MLS residential data is comparatively fast (often within 24 to 48 hours of a status change) because it is transaction-driven and brokers have contractual obligations to update it promptly, enforced by local MLS rules.

Lag matters most in benchmarking. If you are calculating a submarket vacancy rate, you are averaging records with different "as of" dates dressed up as one number.

Where silent errors enter

Silent errors are the dangerous ones: no error message, no red flag, just a wrong number treated as correct.

• Unit mismatches. Rentable square footage (includes a share of common areas) versus usable square footage (just the tenant's space) get swapped without anyone noticing. A 10 to 15% gap between the two is common in office buildings.
• Stale status flags. A listing marked "available" for a space that was leased three weeks ago but never updated.
• Currency and standard differences in cross-border portfolios. A European asset measured under IPMS (International Property Measurement Standards, promoted by the RICS, the Royal Institution of Chartered Surveyors) compared against a US asset measured under BOMA standards without conversion.
• Survivorship bias in comp sets. Distressed sales or off-market deals often do not enter CoStar or public comp databases at all, quietly skewing average price-per-square-foot upward.

A quick governance framework

Data governanceData governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.View full definition → in real estate analytics usually rests on four checks:

1. Lineage: can you trace a number back to its original source and timestamp?

2. Freshness: how old is this record relative to when the underlying event occurred?

3. Completeness: what percentage of expected fields are populated (a common data-quality metric is "field completeness rate")?

4. Reconciliation: when two sources disagree, is there a documented rule for which wins?

A simple completeness check in practice:

# crude field completeness metric for a lease abstract dataset
required_fields = ["rsf", "start_date", "end_date", "base_rent", "tenant_name"]
completeness = df[required_fields].notna().mean().mean() * 100
print(f"Average field completeness: {completeness:.1f}%")

Portfolio teams often set an internal threshold (commonly cited informally around 90 to 95% completeness) below which a dataset is flagged for manual review before it feeds into valuation models.

Benchmarks worth knowing

A few widely cited, order-of-magnitude figures (treat all as estimates, subject to change):

• US commercial real estate transaction data is estimated to have 90%+ coverage for institutional-grade assets in major metros through platforms like CoStar and Real Capital Analytics, but coverage drops materially for smaller, private, off-market deals.
• Lease abstraction accuracy from manual human review is often cited around 95 to 98% field-level accuracy; AI-assisted abstraction tools report similar or better accuracy on standard fields but still require human QA on complex clauses (renewal options, co-tenancy provisions).
• Property-level ESG (Environmental, Social, Governance) data, like energy benchmarking under US municipal disclosure laws (for example, New York City's Local Law 84) or the EU's Energy Performance Certificates, has patchy compliance; reporting rates well below 100% are commonly documented in city audits, since enforcement varies by jurisdiction.

For a public, free reference point on US benchmarking law and city-level energy disclosure data, see the US Department of Energy's Building Energy Asset Score program and comparable city open-data portals.

🎬 [VIDEO: "How CoStar Collects Its Data" - youtube.com - search for CoStar's own explainer content on its research and field-verification process, useful for seeing the human layer behind "big data" real estate platforms]

Key Takeaways

• Every real estate asset has multiple parallel data trails (assessor, MLS, commercial platforms, lease abstracts, loan files) that are rarely reconciled into one authoritative record.
• Duplication comes from independent re-entry of the same event; lag comes from infrequent update cycles (annual tax rolls, quarterly research verification); silent errors come from unit mismatches (rentable vs. usable square footage, BOMA vs. IPMS) and stale status flags.