# Where AI models fail in property decisions
In March 2020, an automated valuation model (AVM) used by a major U.S. iBuyer kept pricing homes as if the market from January still existed. By the time the model's underlying data caught up, the company had bought hundreds of homes above where the market had already moved. That model was not broken in any obvious way. It was confidently, precisely wrong. That gap, between confidence and accuracy, is the subject of this lesson.
Real estate has three structural features that make AI failure modes distinct from, say, fraud detection in payments.
First, transactions are infrequent and illiquid. A residential AVM might be trained on comparable sales ("comps") that are months old, because that is the freshest data available in a given ZIP code. Second, real estate markets are hyperlocal: a model trained nationally can systematically mismatch a neighborhood-level shock (a factory closing, a new transit line). Third, real estate carries a long history of embedded discrimination (redlining, the practice of denying services to residents of certain areas, historically based on race, outlawed by the U.S. Fair Housing Act of 1968), so historical data used to train AI is not neutral. It carries the fingerprints of past policy.
Understanding these three features (data staleness, hyperlocal drift, historical bias) gives you a framework for spotting failure before it becomes a governance crisis.
AVMs estimate a property's value using recent comparable sales, tax records, and property characteristics. Zillow's "Zestimate" and Redfin's valuation tool are consumer-facing examples; institutional versions power mortgage underwriting and iBuying (companies that use algorithms to buy homes directly from sellers, renovate, and resell).
The failure: comps lag reality. In a fast-moving market, or right after a shock, the most recent "comparable" sale might be three or six months old, effectively describing a market that no longer exists.
The most cited real-world case is Zillow's own iBuying unit, Zillow Offers, which the company shut down in November 2021 after its pricing algorithm overpaid for thousands of homes as the model failed to anticipate how quickly home price appreciation would decelerate in late 2021. Zillow publicly attributed the failure partly to the unpredictability of forecasting home prices at scale (Zillow's own statement and reporting, as of Nov 2021). This is a useful case precisely because it was a sophisticated, well-resourced team and the model still drifted.
The lesson: any AVM has a "freshness window." Ask vendors directly: how old is the comp data driving today's valuation, and how does the model behave when transaction volume drops (which is exactly when accuracy matters most)?
Tenant-screening tools use criminal records, credit scores, eviction history, and sometimes social media or rental payment data to generate a "risk score" for prospective renters. These are widely used by property managers across the U.S.
The failure: these inputs correlate with race and income in ways that reproduce historical discrimination, even when race is never an explicit input. Eviction records, for instance, are known to disproportionately reflect enforcement patterns rather than actual payment risk. A 2023 lawsuit and subsequent settlement against SafeRent Solutions (a tenant-screening company) alleged its scoring algorithm disproportionately harmed Black and Hispanic rental applicants who used housing vouchers, illustrating how a facially neutral score can produce disparate impact (a legal standard where a policy is neutral on its face but produces discriminatory outcomes), which is actionable under the Fair Housing Act even without intent to discriminate.
This is the sharpest governance risk in the sector: a model can be "accurate" at predicting historical outcomes and still be illegal, because the historical outcomes it learned from were themselves shaped by discriminatory practices.
The lesson: predictive accuracy is not a defense against disparate impact claims. Governance requires testing outcomes across protected classes (race, national origin, familial status, disability, and others under the Fair Housing Act), not just testing overall model accuracy.
Dynamic pricingDynamic pricingAutomatically adjusting prices in real time based on demand, competition or user behaviour to optimise revenue, margin or conversion.View full definition → engines, used heavily in multifamily rental (apartment) pricing, recommend rents based on real-time demand signals, competitor pricing, and occupancy targets. RealPage is the most prominent name here.
The failure and the controversy: RealPage has faced a U.S. Department of Justice (DOJ) antitrust lawsuit, filed in August 2024, alleging that its algorithm enabled landlords to effectively coordinate pricing by pooling competitively sensitive data, potentially violating the Sherman Antitrust Act. RealPage disputes the allegations. Multiple U.S. cities and states have separately proposed or passed legislation restricting algorithmic rent-setting tools, including ordinances in San Francisco and Philadelphia (as of 2024 to 2025; check current status, as this area is moving fast).
Separately from the antitrust question, there is a pure model-risk question: pricing engines trained on "normal" supply-demand dynamics can behave unpredictably during shocks. During COVID-era eviction moratoriums, for example, occupancy and payment data no longer reflected genuine market demand, but algorithms optimizing on that data did not automatically know to discount it.
The lesson: algorithmic pricingalgorithmic pricingAutomatically adjusting prices in real time based on demand, competition or user behaviour to optimise revenue, margin or conversion.View full definition → tools sit at the intersection of two distinct risks: model accuracy risk (does it price correctly in a shock) and legal/competition risk (does its design facilitate coordination among competitors, even unintentionally).
Here is the mental model to carry forward. A model outputs a prediction and, often, a confidence interval or score. The failure modes above share one pattern: the model's stated or implied confidence did not shrink even as real-world accuracy degraded.
Simple check before trusting a model output:
1. Data recency: how old is the training/reference data (in days)?
2. Regime check: has anything structurally changed since (rate shock,
policy change, local supply shock)?
3. Confidence calibration: does the model's confidence score actually
correlate with historical error rates in similar conditions?
4. Subgroup test: does accuracy or error rate vary meaningfully across
protected classes or submarkets?
If #2 is "yes" and #3 has not been re-tested, treat the model's
confidence score as unreliable, regardless of what it displays.This is not a technical audit you need to run yourself as a manager, but you should be able to ask a vendor or internal data team these four questions and expect coherent answers.
Knowledge check
1. The iBuyer AVM example illustrates a specific kind of AI failure. What made it dangerous from a governance perspective?
2. Why does real estate's infrequent, illiquid transaction pattern create a distinct AI failure mode compared to a sector like payments fraud detection?
3. A national valuation model performs well on average but badly mispriced homes in a neighborhood after a factory closure. Which structural feature of real estate best explains this failure?
4. Select ALL correct answers about why historical real estate data is described as 'not neutral.'
Select all the correct answers.
5. Select ALL correct answers describing the three structural features that make real estate AI failure distinct from other sectors.
Select all the correct answers.
Given these failure modes, a minimal pre-deployment checklist for real estate AI tools should include:
For a practical overview of U.S. federal thinking on this, the Federal Trade Commission's guidance on AI and consumer protection is a useful, free, plain-language starting point.
🎬 [VIDEO: "How Zillow's Home-Buying Algorithm Lost Half a Billion Dollars" - https://www.youtube.com/results?search_query=zillow+offers+algorithm+failure - a walkthrough of the Zillow Offers shutdown as a case study in AVM overconfidence during a market shift]