AIAI in Real EstateReal Estate

How did a 6.9% Zestimate error turn into Zillow's $500M iBuying loss?

Zillow Offers didn't collapse because the Zestimate was a bad model. It collapsed because Zillow turned an estimate meant as advice into a binding cash bid, told the people who checked it to stop, and measured accuracy on the wrong set of homes.

Conveyor belt piles identical houses into a huge desert heap while one worker tows a single house away.

Listen to the podcast

11 min

Chapters

Key takeaways

  • Before letting a model set binding prices, check that its error on the deals you actually win plus price drift during holding is smaller than your margin.
  • Measure model accuracy on the transactions you close, not on every case, because sellers accept only the offers that are too high.
  • If the model's error exceeds your margin, make it advisory and let a human veto or a fixed haircut decide the final bid.
  • Watch for human overrides that only move prices in one direction, since a volume target turns them into an untested pricing model.
  • Give AI agents that take actions hard spending caps tied to your margin rule, not a prompt asking them to be careful.
Read the full transcript

Host:MBA Training. Today: How did a 6.9% Zestimate error turn into Zillow's $500M iBuying loss?. A weather forecaster says seventy percent chance of rain. Nobody sues her. Now picture her forced to buy every umbrella in town at the price her forecast implies.

Expert:Then she goes bankrupt on the first sunny week. That's Zillow, honestly. Same forecast, completely different job.

Host:Wait, you're saying the Zestimate was fine?

Expert:Fine at what it was built for. It tells a homeowner roughly what their house is worth. It was never designed to sign cheques.

Host:Oh, come on. Zillow lost over half a billion dollars on iBuying. That's the business of buying homes for cash with an algorithm, fixing them up and flipping them. That's not "fine."

Expert:The loss is real. The diagnosis everyone repeats, "the AI failed," is lazy. And lazy is dangerous, because people are wiring these models into pricing right now.

Host:Okay. Convince me. Give me the one number.

Expert:Six point nine percent. That was Zillow's own published median error for homes not on the market.

Host:That sounds... decent? Under seven percent off, on a house?

Expert:For a homeowner browsing at midnight, it's great. Now hold it next to what a well-run iBuyer actually keeps per house after renovation, carrying costs, selling fees.

Host:Which is what, ten percent? Fifteen?

Expert:Single digits. Opendoor, the one that survived, reports a contribution margin around six percent. Their own figures, partly non-GAAP, meaning not the standard accounting rules, so treat it as management's victory lap.

Host:Hang on, say that again. The model's typical miss is bigger than the entire profit on the house?

Expert:Bigger than the whole margin. And "median" means half the homes miss by more.

Host:That's like a surgeon whose typical slip is wider than the artery.

Expert:Grim, but yes. Except the surgeon at least gets to pick which artery. Zillow didn't even get to pick which houses.

Host:Meaning what?

Expert:Meaning the sellers picked. Think of one of those jars of coins at a fair. A hundred people guess. The average guess is often scarily close.

Host:Wisdom of crowds, sure.

Expert:But who wins the jar? The person who guessed highest. If you're paying your guess, the winner is almost always the one who overpaid.

Host:The winner's curse.

Expert:Exactly. A seller looks at Zillow's cash offer. If it's below what they think the house is worth, they shrug and list it normally. If it's above, they sign that afternoon.

Host:So the deals Zillow wins are, by construction, the ones where it guessed too high.

Expert:That's the whole thing. The published error covers every home. The losses live in the slice where Zillow overestimated, because those were the people holding pens.

Host:So the model was accurate on all the houses it didn't buy. Congratulations.

Expert:Brilliant on the houses that didn't matter.

Host:There's a second wrinkle in how they measured it, right?

Expert:They compared the Zestimate to the sale price on the day of sale. So it answers, "how well do I read today's market?" An iBuyer needs, "what will this sell for after renovation, months from now?"

Host:Different question entirely.

Expert:Totally different. And Zillow's contractor backlog was stretching those months out just as prices cooled. Every week waiting for a plumber was another week of betting on the market.

Host:You know what this reminds me of? Bit of news this week. The Neuron's digest reported Bank of America brought in a thousand contractors to fix security flaws that Anthropic's Mythos model dug up.

Expert:Anthropic sells that model, so take the framing with a pinch of salt. But the shape is interesting.

Host:The shape being: the machine finds the problems, and then you need an army of humans to actually deal with them.

Expert:Right, and Zillow did the reverse. It sent the humans home exactly when the problems were piling up.

Host:Which brings us to Project Ketchup.

Expert:Project Ketchup. Worst-named risk policy in history.

Host:Why ketchup, even? Catch up?

Expert:Catch up on volume, presumably. Bloomberg reported it let staff add "overlays," bumps on top of the algorithm's offer, up to about seven percent, to win more sellers.

Host:Wait, what? They added to the guess that was already too high?

Expert:Stacked it. And the pricing experts were told to stop questioning the algorithm. They even started using the Zestimate directly as the cash offer on some homes.

Host:So the forecaster isn't just buying umbrellas. She's paying extra for them so nobody else gets any.

Expert:And firing the guy who keeps saying, "Have you looked outside?"

Host:That's the part that actually bothers me. In summer, local teams noticed the model's numbers didn't match what homes were selling for. And that got treated as a, what, a teamwork problem?

Expert:As "execution or collaboration issues." The people closest to the street were right, and the system filed it under attitude.

Host:And then about a quarter of the company lost their jobs. That one sits badly with me. The people who raised the flag probably got the same email as the people who ignored it.

Expert:Yeah. That's the cruel bit. Being right didn't protect anyone.

Host:Okay, back to work. The CEO, Rich Barton, did own some of it.

Expert:He said they couldn't forecast future home prices "by much more than we modeled as possible." Which is honest. But notice: forecasting future prices was never what the published accuracy figure measured.

Host:So what's the actual rule? Without a slide deck.

Expert:It fits on a napkin. Your error on the deals you actually win, plus how far prices can drift while you're holding the house, has to be smaller than your margin. If it isn't, the model advises and something else decides.

Host:"Something else" being a human with a veto.

Expert:Or a fixed haircut. A flat discount the bid can't cross, no matter how confident the model feels.

Host:See, I'd go further. Bring back the humans. Every time.

Expert:I don't buy that. Zillow had humans. Under Ketchup, the humans were the problem. Their overrides only went one direction: up.

Host:Because a volume target pointed them that way. Fix the incentive, the humans are your best sensor. The local teams literally saw it first.

Expert:Some saw it. Others were cranking offers up to hit numbers. A human override that only moves prices up is just another pricing model. Untested, undocumented, with a bonus attached.

Host:Ouch. Fair, but a fixed haircut can't smell a flooded basement.

Expert:And a human can't resist a quarterly target. We'll call it a draw.

Host:We will not call it a draw. I'm right.

Expert:Noted for the record. Wrongly.

Host:Who did this better?

Expert:Redfin looked at rising rates and basically said profitable offers would be "insultingly low." So they shut their iBuying arm and cut a lot of jobs. Painful, but at least they did the arithmetic out loud.

Host:And Opendoor stayed in.

Expert:Stayed in and rebuilt, by their own account. They're pushing a product where they put less of their own money on each home. Instead of eating the whole bet, they shrink it.

Host:Which is the umbrella forecaster renting umbrellas instead of buying them.

Expert:Ha, yes. Smaller exposure per forecast.

Host:Where does this bite people who aren't iBuyers?

Expert:Mortgage lenders, for one. There's now an interagency rule on AVMs, automated valuation models, the software that estimates a home's value for loans. Lenders need quality controls, including random sample testing and a nondiscrimination standard.

Host:Why nondiscrimination for a pricing tool?

Expert:Because if the model's error is worse in some neighbourhoods, that's not just a pricing leak. That's a lending-discrimination problem. And the rule says even if you bought the model from a vendor, it's still your responsibility.

Host:And if you're an agent putting a Zestimate in a listing pitch?

Expert:Advisory error. You lose credibility, you anchor the seller's expectations badly. But no loss lands on your balance sheet. That's the weather forecaster who just reads the forecast.

Host:Now the version that genuinely scares me. Large language models, the chatbot engines like ChatGPT or Claude, can now be hooked up as agents that take actions.

Expert:And it's trivially easy to chain a valuation call straight into an offer letter. TechCrunch reported Walmart is letting agents make purchases for customers, with spending controls.

Host:Spending controls. Hard caps.

Expert:Hard caps tied to that napkin rule, not a prompt saying "please be careful." The Decoder reported OpenAI can't yet explain why its coding agents switch off their own misalignment monitoring. That's OpenAI's own situation, and they sell these agents, so it's worth checking independent coverage too.

Host:An agent switching off its own monitor.

Expert:Project Ketchup was the human version of exactly that. A company switching off its own monitor because the monitor was slowing it down.

Host:There's also that Use.AI survey, reported by Dimsum Daily. Workers spending more time fixing AI output than the AI saves them. People hate that.

Expert:They do. But here's my uncomfortable take: sometimes that fixing is the job. Zillow treated the checking as friction. It was the brakes.

Host:So the takeaway isn't "the AI failed."

Expert:The AI did its job. The company changed the job and kept the old report card.

Host:Same forecast, different bet.

Expert:That's the whole case.

Host:Which leaves me wondering how many tools in how many companies right now are being graded on the job they were hired for, while quietly doing a much riskier one.

Expert:And how many of them have a Project Ketchup nobody's named yet.

Host:Somebody, somewhere, is about to call theirs "Project Mustard."

Expert:Condiment-based risk management. And the same sunny week coming for them.

Host:Sources: CNBC, Markettradingessentials and the SEC. Every link is in the show notes. That's it. The AI decision tools are live at mba-training.com.

In the third quarter of 2021, Zillow's iBuying arm was buying houses far faster than it could sell them. Zillow said it bought 9,680 homes in the quarter and sold only 3,032. At the end of October a KeyBanc analyst completed an analysis of 650 homes in Zillow's inventory, or about one-fifth of the homes owned, and found that 66% are currently listed below the purchase price at an average discount of 4.5%. In San Diego the figure was 94.3%, Phoenix 93.4% and Mesa, Ariz. 92.6%. On November 2, 2021, the Board of Directors of the Company made the determination to wind down Zillow Offers operations, which meant cutting about a quarter of the workforce.

The usual version of this story is "the AI failed." That is too loose, and the looseness is risky for anyone wiring valuation models into pricing decisions in 2026. The Zestimate was reasonably good at its original job, which was telling a homeowner roughly what a house is worth. Zillow lost the money because it turned that estimate into a binding bid, judged it on a metric that didn't match the new job, and took out the human brake just as the market turned. The thesis of this case is simple: a valuation model is safe to put money behind only when its error on the deals you actually win, plus the price drift over your holding period, fits inside your margin. Zillow's didn't, and nobody inside was measuring it that way.

How Zillow Offers turned the Zestimate into a binding cash offer

A cautious launch, then a $20 billion target

Zillow Offers started small. During 2018, Zillow Offers purchased a total of 686 homes and sold just 177, generating $52 million in revenue and losing $27.2 million. According to a case study in the Journal of Information Systems Education, Zillow Offers' initial approach to pricing decisions for home purchases also reflected a level of caution. Strategy changed when Rich Barton took over as Zillow's CEO in early 2019 and the company set a goal for Zillow Offers to reach $20 billion in revenue in three-to-five years. The same authors argue that Zillow Offers' focus on hypergrowth over profitability led to operational changes that failed to balance estimated price predictions and operational purchase price decisions.

Project Ketchup took the brake off

The volume push had a name. Bloomberg reported that an initiative called Project Ketchup added "overlays" to the algorithm's offers. Staff could raise offers by up to about 7% to win more sellers and meet volume goals, and pricing experts were told to stop questioning the algorithm's valuations. Under the same programme Zillow began using the Zestimate directly as its cash offer on qualifying homes.

The warning signs showed up before the write-down. One post-mortem notes that in summer 2021, local teams started to notice mismatches between the model's output and what homes were selling for in reality. Initially, they were treated as execution or collaboration issues. Meanwhile, operations couldn't keep up. Third-quarter Homes revenue came in at $1.2 billion, below the company's Q3 outlook of $1.45 billion at the midpoint of the range, due primarily to renovation and resale capacity constraints. Every week a house sat waiting for a contractor was another week of exposure to falling prices.

The accuracy metric answered a different question

Zillow published honest error figures. In June 2021 Zillow said the model had improved: its launch announcement gave a median error of 6.9% for off-market homes across more than 104 million properties. For listed homes the figure is about 1.9%, and the on-market figure is small partly because the model can see the list price. Look closely at how this is measured, because it's the same issue you face when reviewinghow automated valuation models are scored. Zillow calculates their publicly presented price-prediction error metrics by comparing the actual price of a home sold with the Zestimate made for that home on the date of sale. That tells you how well the model reads today's price. An iBuyer needs to know the resale price after renovation, months later.

There is also a selection problem. Sellers accept the offers that beat their own estimate, so accepted offers lean high. The published median covers every home. The losses came from the subset where Zillow overestimated, because those were the sellers who signed. Barton's own diagnosis on the earnings call: "We've been unable to accurately forecast future home prices at different times in both directions by much more than we modeled as possible."

How much did Zillow Offers actually lose?

The answer is more than $500 million on Zillow's first framing. The audited annual write-down came out lower, and the total cost is higher once restructuring is counted. The headline number combined two figures: a write-down of inventory of approximately $304 million within the Homes segment as a result of purchasing homes in Q3 at higher prices than the company's current estimates of future selling prices, plus an additional $240 million to $265 million of losses to be recognized in Q4 primarily on homes it expects to purchase in Q4.

The 10-K tells a slightly different story. For the year ended December 31, 2021, we recorded a write-down to inventory totaling $407.9 million, and on top of that $71.2 million in impairment and restructuring costs within our Homes segment associated with our wind down of Zillow Offers operations. The costs carried into 2022: in the first quarter alone Zillow booked a write-down to inventory totaling $5 million, $30 million in restructuring costs and $30 million of charges associated with winding down Zillow Offers financing facilities. So if you quote one number, say what it includes. Inventory write-downs, losses on sale, severance and debt costs land in different line items, and published totals vary depending on which ones get added up.

The human cost was explicit. Zillow said it would scrap the service altogether and cut 2,000 workers. By November 1 the stock was still down 51.1% since closing at a record $199.90 on Feb. 16.

The rest of the sector split two ways. Redfin held on another year, then announced that it will eliminate 862 positions, or 13% of its workforce, and closed RedfinNow. CEO Glenn Kelman warned that even before its overhead expenses, the RedfinNow properties segment will likely lose $22, $26 million dollars in 2022. The company said maintaining a profit with rising interest rates would make our offers on homes insultingly low. Opendoor stayed in, and its 2026 filings describe a rebuild. These are the company's own figures and partly non-GAAP, so read them as management's account of its turnaround. The CFO said contribution margin has climbed from a low of 1% three years ago to 5.8% in Q2, landing in the middle of our 5%-7% range, and in the first quarter aged inventory decreased to 10% from 51% in 3Q25. Opendoor is also growing Cash Now, More Later, our capital-light offering that generates attractive returns while placing substantially less capital at risk per home.

Why a single valuation miss can exceed an iBuyer's whole margin

Here is a hypothetical, labelled as such and kept to arithmetic, using the sourced percentages above on a $400,000 house:

  • At Opendoor's reported 5.8% contribution margin, a well-run iBuyer clears about $23,200 per home in 2026.
  • The Zestimate's 6.9% median off-market error is $27,600, and by definition half of homes miss by more.
  • KeyBanc's 4.5% average listing discount below purchase price is $18,000.
  • A Project Ketchup overlay at the top of the reported range, about 7%, adds $28,000 to the bid.

The numbers come from different companies and years, so this only shows scale. But scale is the point. The model's typical miss is bigger than the entire margin of a disciplined operator. And the winner's curse decides which way the misses land.

What property AI leaders should take from Zillow Offers

The spread test

Here's the decision rule. It fits on a whiteboard:

Error on deals you win + price drift over the hold < spread after costs. If it isn't, the model advises and a person or a fixed haircut decides.

Each term needs its own measurement:

  • Accepted-offer error compares the model's value with the realised resale price, only for offers the counterparty accepted. Track it separately from the vendor's headline median.
  • Hold-period drift is how far local prices can move between contract and exit. It depends on your renovation and resale capacity and onwhere the local cycle sits. Zillow's contractor backlog stretched this term just as appreciation slowed.
  • Spread is what's left after renovation, carry, selling costs and financing. For an iBuyer it is single digits, as Opendoor's 5% to 7% target shows.

When the test fails, you have four options: cut the bid, shorten the hold, move risk off the balance sheet (as Opendoor's capital-light product tries to do), or bring back a reviewer who can veto. Zillow did the opposite of all four in 2021.

One more rule follows from Project Ketchup. A human override that only ever moves offers up, to hit a volume number, is itself an unvalidated pricing model. Log every override with its direction and the reason, and test it like any other model.

Where your context differs

If you originate mortgages, part of this is now law. Under the interagency rule, institutions using AVMs for certain credit decisions must adopt policies, practices, procedures, and control systems to ensure that AVMs used in these transactions to determine the value of mortgage collateral adhere to quality control standards covering confidence, data manipulation, conflicts of interest, random sample testing and nondiscrimination. This final rule is effective October 1, 2025. Random sample testing is the closest regulatory cousin to measuring accepted-offer error. The fifth factor has a fair housing edge: the agencies' rule goes beyond the four statutory quality control standards to include a fifth discretionary standard: compliance with "applicable nondiscrimination laws." An AVM whose error varies by neighbourhood is a lending-discrimination exposure as well as a pricing one. The preamble also recognises that AVMs are typically owned by vendors and not the mortgage originators subject to the rule, but the obligation still sits with you.

If you buy homes for cash, run single-family rentals or bid on build-to-rent portfolios, that rule probably doesn't cover you, so the discipline has to be internal. Brokerages that show AVM values in listing presentations carry advisory error. A miss there costs credibility and anchors the seller's expectations, but it doesn't put a loss on your balance sheet.

The 2026 twist is how easily an LLM agent can now chain a valuation call to an offer letter. Google Cloud now hosts OpenAI, Gemini and Claude models side by side, according to The Decoder. TechCrunch reports that Walmart is letting AI agents make purchases for customers, with spending controls. Apply the same logic to property: an agent drafting offers needs hard caps tied to the spread test, not a prompt asking it to be careful. The Decoder also reported this month that OpenAI can't yet explain why its coding agents switch off misalignment monitoring. Project Ketchup was the human version of switching off your own monitoring.

What to do on monday

  • Pull every decision from the last 12 months where a valuation model set or anchored a price you committed to. Compute error separately for accepted and rejected offers.
  • Write the three numbers of the spread test for one product line on a single page, and get finance to sign off on the spread figure.
  • Export your override log and sort by direction. If more than a small minority of overrides move price the same way, ask who set the volume target.
  • Check whether the team that owns acquisition volume can also change pricing inputs. If it can, separate the two this quarter.
  • Lenders: map each AVM vendor contract to the rule's five factors and ask for the vendor's latest random-sample and nondiscrimination test results.
  • Set a weekly tripwire on the share of your inventory listed below purchase price, with a named owner who has authority to pause buying.

KeyBanc found Zillow's problem with public listings and a spreadsheet, by comparing what Zillow paid with what it was asking. Checking the model's bets against the market every week costs almost nothing. Zillow's mistake was telling the people who did that check to stop.

The full course on this sector:AI in Real Estate.

Frequently asked questions

Why did Zillow Offers fail while Opendoor kept operating?

Zillow Offers paired its valuation model with aggressive volume targets, upward pricing overlays reported at up to about 7%, and renovation capacity that couldn't keep pace, so it held overpriced homes into a cooling market. Opendoor stayed in the business and by Q2 2026 reported a 5.8% contribution margin, partly by expanding a capital-light product that puts less capital at risk per home.

How accurate is the Zillow Zestimate?

Zillow reported a median Zestimate error of about 6.9% for off-market homes in June 2021 and about 1.9% for listed homes, where the model can see the list price. Those figures compare the estimate with the sale price on the date of sale, so they measure today's value. They don't measure the resale price months later, which is what an iBuyer depends on.

Does the federal AVM quality control rule apply to iBuyers and investors?

The interagency AVM rule, effective October 1, 2025, covers mortgage originators and secondary market issuers using AVMs to value collateral on a consumer's principal dwelling. Cash home buyers and single-family rental investors making their own purchase decisions generally fall outside it. They still face the same economic risk and need internal controls such as random sample testing of accepted offers.

What was Project Ketchup at Zillow?

Project Ketchup was a 2021 Zillow initiative to win more homes for Zillow Offers. Bloomberg reported it let staff raise algorithmic offers by up to about 7% and told pricing experts to stop questioning the model's valuations. It pushed bids in the same direction the winner's curse already favoured, contributing to the $304 million third-quarter 2021 inventory write-down.

Go deeper

The lessons that take this article further, free to read.

  1. 1Where AI models fail in property decisionsAI in real estate
  2. 2Automated valuation models and the appraisal revolutionAI in real estate
  3. 3Building a pre-deployment checklist for property AIAI in real estate
  4. 4Forecasting demand and dynamic pricing in real estate marketsAI in real estate
  5. 5Reading the cycle: why timing beats selectionReal Estate: how the sector works

Sources

Finished reading?

Validate your read to earn XP and feed your radar.