Opinion:

Models Must Be Wielded With Care

Zillow’s instant Buying (aka Zillow Offers) business spectacularly imploded this month. After 3.5 yrs, Zillow made the tough decision to shutter their Zillow Offers business. The final result[1]:

• 25% workforce reduction

• 7,000 unsold homes bought for $2.8B (with an expected $304M write-down in Q4).

• -30% stock drop

• -$245M in losses for Q3

The scale of this disaster is impressive. Misjudgingly, most of the blame has been pinned on the Data Scientists; those “book-smart know-it-alls” that blindly create and trust their models /s. It’s lazy to mock ZEstimate and the idea that Zillow trusted it with with billions of dollars[2]. It’s also easy to mock the science of forecasting and the notion that Zillow’s data scientists simply used “pip install prophet” (profit) to get a production model[3].

All of this is wrong. Zillow Offers and iBuying is an intrinsically risky business that was recklessly executed by Zillow’s business leaders who wanted to take over the real estate market in just a few years. As evidenced by Zillow’s competitors, Opendoor and Offerpad, iBuying does work! But it needs patience, experience, and a healthy respect for risk. Instead Zillow’s business leaders dove head-first into the shallow-end.

What is iBuying

iBuying is an emerging and somewhat unproven industry. Opendoor (2014) and Offerpad (2015) have been at it for years with growing success. The concept is simple: Allow people to sell their home, instantly at a slight discount while saving them the hassle of a tedious, painful, and risky-selling process. Anyone who has bought or sold a home has had the same realization: The home buying process badly needs modernation and innovation. Even in 2021, it’s still largely a handshake and paper business.

iBuying at its core is not about market speculation nor reality TV-style overnight renovations. Zillow themselves spent less than 2% of their operating costs on renovations (not far from the cost of a paint job). Nor is this an automated business; Zillow sent home inspectors and conducted in-person assessments for each purchase.

Here’s the rub: iBuying has exceptionally slim profit margins. On the $1.2B in homes Zillow sold in Q3, they only made $21.5M in profit… a measly 1.8%. Competitors faired much better, between 7–9% margins during the same period. It’s no coincidence that these are close to total real estate agent’s commissions. And for Zillow, a multibillion dollar tech company, these razor-thin margins are only attractive if it could scale this to a $20–100B business.

Zillow believed iBuying was the future of real estate, and they were going to dominate. They expected this to be a $20B business in just a few years, and needed to aggressively expand in all markets, nationwide. While competitors continued to take a methodical, and cautious approach, Zillow’s executives took on all the risks.

The Data Science Challenge

To be successful, Zillow needed hyper accurate forecasting models which could predict a home’s sale price in the next 3–6 months, within +/- 3% [4]. An overestimate could wipe out profits, or worse, result in >$100M losses. An underestimate would result in rejected lowball offers and cripple Zillow’s ability to purchase homes (ie. low buyer conversion rates). While machine learning-based predictive models are standard practice in tech, housing is also subjected to significant and unpredictable macroeconomic trends.

A forecasting rule of thumb says to predict X months into the future, you need 2X months of historical data. Yet that rule only works if your historical data is representative of the future. The ongoing global pandemic, rapid inflation, shifts in consumer behaviors and major swings in home prices ensure that this is not possible: We’re in unprecedented territory.

Image used with permission from Lighter Side of Real Estate

Aggressive Growth and Business Goals Killed Zillow Offers

To aggressively scale the Zillow Offers business, Zillow executives intentionally adjusted their algorithm estimates upwards, which accomplished the goal of increasing buying conversion rates but at higher offer prices [5]. Zillow Offers, coming off a terrific Q2 with 15% gross margins thanks to generous price appreciations was feeling pretty confident and continued to expand.

Unfortunately, the market in Q3 reversed and instead of +12% growth, the housing market saw -5–7% drops, resulting in $300M in losses and an expected $245M in write-downs. Is this a failure in algorithms? No, Opendoor contrastingly reported a successful Q3 with $170M in profits and 7.3% in gross margins. As Zillow exits the iBuying business, Opendoor will hold the course and continue to refine their models while treating volatility with great respect.

The failure lies not in the models, but those who wield them. Models are not omnipotent, and those who choose to base a business on them need to understand their nuance and limitations.

References

1. Zillow Inc. (2021). Q3 Financial Report.

2. https://twitter.com/ryxcommar/status/1456079677957804038

3. W.D. “Zillow, Prophet, Time Series, & Prices.” 6 Nov 2021.

4. Vashishta, Vin. “Zillow Just Gave Us A Look at Machine Learning’s Future.” 4 Nov 2021.

5. Clark, Patrick, Natarajan, Sridhar, Perlberg, Heather. “The decision came after the company tweaked the algorithms that power the business to make higher offers, leaving it with a bevy of winning bids just as home-price appreciation cooled off a bit.” — Zillow Seeks to Sell 7,000 Homes for $2.8 Billion After Flipping Halt, Bloomberg. 1 Nov 2021.

6. Opendoor Technologies Inc. (2021) Opendoor announces third quarter 2021 financial results.

Special thanks to Mike from Lighter Side of Real Estate for the Bob Ross meme.