Zillow, Prophet, time series, and prices | Hacker News Reader

Zillow, Prophet, time series, and prices

ryxcommar.com

213 pointszorked143 comments

While there are some decent points in this post, it ignores another possibility for why Zillow Offers got shut down.

The first explanation, when it was just being paused, was that there was a shortage of labor and materials to do the renovating. Probably true, but not the real reason. It is entirely possible that the second explanation, that it was too hard to model accurately the price 6 months out, is also true but not the real reason.

The real reason might be something the Zillow CEO doesn't want to say out loud. Like, we are near a top in house prices, and it looks like it might fall a lot, or for a long time, or both, before we get back to reasonable house prices that have a chance of going up again. Zillow does not want to say "real estate is about to bust, as an investment strategy", because they are still tied to the real estate market. But, flawed as their analysis might have been, they might have been able to see enough to know that no matter how good their algorithm could be made, they're not going to be able to buy low and sell high if the prices keep falling.

Now, maybe Zillow is wrong again, and housing is not in for a rough ride in the very near future. But maybe they're not wrong, and shutting down Zillow Offers (rather than fixing it) will look like a quite prudent move in a year's time.

Median housing prices have been consistently rising [0] since Zillow started their property buying program in 2018. In their most recent earnings report, Zillow announced $422M in Q3 losses from their home buying program and expects $240M to $265M in losses for homes they plan to buy in Q4 [1]. Rising sale prices coupled with big losses implies Zillow has been systematically paying above-market rates for homes, and there's a lot of evidence corroborating that hypothesis [2]. Maybe we're near a peak in home prices, but prices haven't started falling yet, and if Zillow was paying market rates, they wouldn't be losing money yet.

[0] https://fred.stlouisfed.org/series/MSPUS

[1] https://investors.zillowgroup.com/investors/news-and-events/...

[2] https://fortune.com/2021/11/03/zillow-house-flipping-overpai...

yeah where I live in AZ, Zillow overpaid massively for every home it purchased that I've seen. Like 15-20% over usual price.

And they've priced the homes they are selling quite a bit below what they bought them for already. Many aren't moving too.

I'm not too sure why they just don't hold onto the homes though. Even with the carrying costs, holding on would be profitable if market is to rise another 10%+ as some are predicting. The only reasonable explanation to me is that they are expecting a significant drop in 2022-2023, but maybe I'm wrong.

They can't do it without getting creamed by the market (which is exactly what's happening now). They're a public company, which means they need to report earnings quarterly. All of those carrying costs come straight out of earnings. Worse, they're a public tech company, which are usually valued on having consistently increasing revenues and high margins (Zillow had something like 80% margins before Zillow Offers). If they have a year of massive losses investors are going to lose confidence in their business model, tank the stock, and oust the CEO.

I suspect that Blackrock or whoever buys the portfolio is going to make massive amounts of money off of it, though. Financial firms are setup to hold assets for long periods of time and even out gains and losses, and their investors expect this.

There’s also the reality that unoccupied homes rot from the inside out. It turns a traditionally appreciative asset quickly into a depreciating asset that, when looking at Zillow, also creates a bit of a market trend since comps are also common in pricing a house.

Banks had this same issue with the foreclosure issues following 2008. And it was largely why some just….didn’t forclose or in other cases struck deals to allow the previous owner to stay on temp basis.

Especially because whoever buys their portfolio is going to get Zillow selling it at a loss, they're going to pay less than zillow did!

Carrying costs aren’t $0 though and it would be horrible PR to sit on empty houses when people are struggling to find affordable housing.

It's hard to say how much they were overpaying. If they "overpayed" by 20% - almost certainly someone else was willing to "overpay" by at least 10% if not 15%.

Unless it's a pretty strong buyers market (spoiler: 2020-now wasn't) - you're gonna have a tough time buying a lot of houses and not "overpaying".

Exactly this. We got a house in hot market last Spring at 17% over asking. Did we overpay? Almost certainly. Did we lose out on 5 bids prior by "only" overbidding 5-15%? Yes. In a sea of exuberant buyers, you have to be the most exuberant. Not a great feeling at the time, but we plan on this being a keeper, and the timing was right for us.

There were a few Reddit threads on people who sold to Zillow. They certainly overpaid.

We’re talking listing for $450k, get real human offers of $485k, asking Zillow for offer and getting $510k.

Absolutely Zillow overpaid, at least a lot of the time. However, part of that is because they apparently had a hard time getting people to accept their offer often enough, and so tweaked the algorithm to pay higher. This suggests that, if you pay the amount you can actually make money on in six months, you don't get very many people accepting your offer, because people don't know as much as Zillow knows about what's about to happen in the housing market.

Or, maybe, Zillow is just no good at predicting housing prices. But it's at least worth considering that, if they were actually reasonably good at it, they might have come to the conclusion that the only way to make money right now is to convince house sellers that their home is about to be worth less than it seems to be now, which isn't likely to be something they wanted to try to do. Maybe they're just incompetent, but they might be competent and have nonetheless come to the same conclusion that Zillow Offers needed to be shut down.

p.s. thanks for the great links, but I would like to point out that the St Louis Fed link goes up to July 1, which leaves open the possibility that Zillow saw something in the last 1-2 months different from what came before.

> if you pay the amount you can actually make money on in six months, you don't get very many people accepting your offer, because people don't know as much as Zillow knows about what's about to happen in the housing market.

I mean... wouldn't that be because people do know as much as Zillow knows about what's going to happen in the housing market, so they're happy to wait the six months to make more money too, instead of letting zillow make a profit by doing nothing more than sitting on it for six months?

> maybe Zillow is just no good at predicting housing prices.

You can spot check the historical “Zestimate” for a few homes over a few years and notice the drastic spikes and troughs where the inaccurate Zillow price was instantly corrected after a real market event. Overall their modeling is wrong, but still useful.

Zestimate is configured strangely. They build it on stale data like 85% of the time. I noticed that when a house is listed, they adjust the Zestimate to be equal to the list price, then revise it again after the sale is posted. It's a revisionist history, and so the question would be what would you validate it against? It's riddled with bias.

Thanks for this well cited information. Here in the US we also have a strange habit of describing housing as national instead of local. Even during the housing bubble pop in 08/09, there were some markets that were hit extremely hard like Las Vegas, and others that were barely impacted like Dallas.

I'm in a market that had insane growth over the last few years, and I am now seeing price cuts on lots of houses listed for sale.

One thing to note is that seasonally there are always less listings and less home shoppers at the end of the year. So seeing more price cuts and less new listings is actually pretty normal in October, November, and December. Personally I'd bet on continued growth in January.

Price cuts don't necessarily mean anything. Better to look at the Case-Shiller index if you're in one of the covered metropolitan areas.

https://realestatedecoded.com/case-shiller/

> ...expects $240M to $265M in losses for homes they plan to buy in Q4

This part of the sentence blows my mind.

We plan to do this thing. It's going to lose a bunch of money. And they're still going to do the thing...

Perhaps those contracts already signed

I would, personally, call something I've already signed for "something that I bought", not "something I'm planning on buying".

But maybe that's not standard in the business sphere.

How does the accounting work for those losses? ie is $422M = (Total $ from homes sold in Q3) - (Total cost of buying/renovating those homes), or is it (Total $ from homes sold in Q3) - (Total $ from homes purchased in Q3)?

But if we are "near the top", that doesn't really explain why Zillow Offers lost so much money. If we're still on the upslope, and the problem is just that Zillow expects a bust soon, wouldn't they have just been able to sell their houses for a profit? Like I don't think Zillow's model was to buy houses and sit on them for five years with the expectation that they continue to rise in value, so should an approaching downturn really cause them to lose hundreds of millions?

I don't think they lost $420 million on it because they knew what they were doing and were good at forecasting home prices and know things others don't about them. I mean, it's an interesting story you tell, but doesn't seem like the most obvious explanation to fit the data -- which would be, actually, they were bad at forecasting home prices, is why they lost $420 million trying to predict home prices.

As someone who's been on the buyer side of the market for the past few months, I think it's even worse for their objective: they might be good at forecasting home prices on average conditional on their statistics, but bad at forecasting home prices of the subset that would be willing to sell to Zillow. Every zillow-owned home I've seen, bar none, has had either a bad micro-location (eg: next to a railroad in an otherwise good location), or a bad interior (eg: a smoker home). The same was true for Opendoor, who are in the same business. IE conditional on Sq Footage, location, acreage, etc. they might be incredibly accurate, but they're only able to buy the 50% below that average (and the gap is large because the missing/intangible variables matter a lot).

That's a really good point. The people most likely to sell to zillow, after all, are going to be the people who think zillow's price is way more than it should be!

En masse buying of buildings based on algorithmic pricing may be doomed because of this. Even if you're on the mark 80% of the time, under-estimate 10% and over-estimate 10% -- the 10% where you over-estimated will be the people most eager to sell to you!

Which is true for anyone in the property business, but if they didn't have local people who understood the market actually looking at the individual property and being like "uh, that one is next to a railroad track which the algorithm doesn't seem to be accounting for..." those people realized they hit the jackpot when zillow offered them a price as if they were NOT next to a railroad track!

I agree that OpenDoor and Zillow bought really weird houses. My neighborhood is completely brick houses except for one that has some plastic looking siding and was in slight disrepair, guess which one Zillow bought and it sat for 6 months? There was another house OpenDoor bought down the street and at one point the roof was caving in, I dove by it for a decade wondering who lived there. A house flipper came along and repaired the cosmetics then sold it to OpenDoor for a premium. It sat for a year. There were also other weird houses like a fake barn and one’s, as you said, with bad micro-locations. Every single one within 5 miles of my house was a house with something weird.

This reminds me of this: https://en.wikipedia.org/wiki/The_Market_for_Lemons

The Indicator from Planet Money podcast just did an episode on Zillow with the same thesis: https://www.npr.org/2021/11/08/1053689886/ibuyers-zillow-and...

The brilliant Zillow AI got fooled by old fashioned American DIY home improvement catastrophes.

The only time I've seen house flipping work at scale is 2010-2012 when those with huge piles of cash could buy foreclosures in a few heavily affected metros, do nothing for 3-6 months then put them back on the market for a tidy profit.

After that brief window closed, you either had to become a landlord for a couple years or employ a crew of handymen on staff to fix all the surprises that come with being a homeowner. You're either a landlord or a builder if you want to actually make money outside of a recession.

The idea was not house flipping at scale. They wanted to "market make" for housing. Housing market is illiquid. People would leave decent money on the table if they could just sell their houses within hours without having to deal with realtors, open houses, listing and closing process. Of course Zillow would have made some repairs and improvements but I don't believe that they thought this was their upside.

Exactly. People sell cars to CarMax for less than they could otherwise get because it's easy, fast, and certain. People buy cars from Carvana over the lowest price they could pay elsewhere because it's easy, fast, and certain.

I think there's a lot of money to be earned by turning a slow, tense process into a fast, lower stress process.

Except, it never works that way for houses. Banks are reluctant to give mortgages when the roof is EOL, or the siding has dry rot or there is some black mold basically anywhere, etc. Giant Corp may be able to buy quickly and efficiently but Joe Homeowner can't get his loan approved until A, B, and C are fixed. Good luck getting 3 different contractors lined up quickly and under budget while in escrow.

Do banks even look at these things; I mean, beyond the appraisal?

I don't think my mortgage lender has ever inquired about the age of the roof on the purchase or during any refinances. For my first house, I bought it with a roof that needed work, had some small leaks, and was well beyond its design life. I got a mortgage without the bank bringing that issue up (albeit in 1996).

Home inspectors look at these things and buyers might choose to walk away based on the inspection report (which the bank never saw on either of my purchases).

In my experience, the level of underwriting is a function of how levered the purchase is. The LTV and your Debt to Income are both major factors in how thoroughly the underwriter is going to review your mortgage and how much they will require to prevent a loss.

That makes sense. Both times I was near the 25% front-end ratio that I considered to be a sensible limit, but was putting 20% down on a conventional conforming loan the first time and a conventional jumbo the second time.

My mortgage was contingent on getting insurance and every place offering a decent insurance premium wanted to at least know the condition of the roof. I live in tornado alley though with lots of major hail events so roofing is kind of a big deal. The sellers of the house had a previous contract fail because the buyer failed to get a mortgage approved because of the bad roof, in the end we made a deal with the seller to adjust the sales price a bit and get a new roof on the house (a bit of give and take, we really loved the house otherwise).

From what I've seen it depends a lot on the kind of loan. I bought a house with a conventional loan earlier this year and I don't think the appraisal even looked at any of the potential issues (Despite at least a few thousand in repairs identified by the inspection). But I also know someone who used an FHA loan to buy a house and they couldn't get the loan unless the house passed a pretty detailed inspection.

FHA loans are 0-5% down - conventional are 20%. 20% covers a lot of repairs if they have to repossess and sell at a loss; 0% covers way less.

Like seemingly everything in the modern world, it depends on your credit worthiness. Some mortgages also come partnered with homeowners insurance, and those policies are what often lead to sales being blocked by even minor issues.

I wonder if this is location dependent, because my bank barely cared about those things when they approved my refinance last year. The appraisal consisted of just driving by the house. They didn’t check for any condition of the house.

It probably helps that the entire house could just disappear and the price for the lot would only change 10% or so.

That's probably due to a favorable LTV. If they have to foreclose, not only would the bank not lose money it might actually be profitable!

> Banks are reluctant to give mortgages when the roof is EOL, or the siding has dry rot or there is some black mold basically anywhere, etc.

but used cars seem to do fine, even though they can be similarly damaged?

Used cars are more fungible and have a higher volume of comparable sales constantly underway.

Repair costs are also better known and less daunting.

Maybe because used cars cost/risk about 1% as much as houses?

Why? because the unit price is lower? Can't zillow just... buy less houses? I don't see anything different between zillow buying a $500k house and one hundred used cars worth $5000 each.

Easier to sell 50 cars than 1/2 of a house tho.

While I don't doubt this effect exists, I seriously doubt it's a significant factor when you're dealing with 7000 houses.

7000 figure from: https://news.ycombinator.com/item?id=29081118

Also, every car has scrap value. Houses don't have that.

The equivalent for houses is the land they sit on. In metros that could easily be worth more than the house, and in most cases it's worth more (in % terms) than the scrap value of a car.

Used cars loans tend to have higher interest rates though.

Also the downsides are vastly different between buying a lemon of a car (although I’m those two cases I bet the companies offer a money back guarantee) that you might lose $1-2k on in repairs vs tens or hundreds of thousands of dollars in buying a bad house.

What market making opportunities do you see that aren’t currently aggressively competed for?

I'm no expert, but as a homeowner and someone who as a result has to employ contractors to fix things, I'd love to see someone make home repairs easier, more certain, and lower aggravation.

I think there are probably thousands of other industries/segments where an aggregate hundreds of millions or billions of dollars are spent annually and where people would spend 1 to 2.5% extra if friction were noticeably reduced. That used to be flights, hotels, and cabs that have been made easier in the last 2 decades.

> I'm no expert, but as a homeowner and someone who as a result has to employ contractors to fix things, I'd love to see someone make home repairs easier, more certain, and lower aggravation.

While it's definitely possible to improve the experience around that, I don't think it really qualifies as market making. Specifially, look at wikipedia's definition:

>A market maker or liquidity provider is a company or an individual that quotes both a buy and a sell price in a tradable asset held in inventory, hoping to make a profit on the bid–ask spread, or turn.

You can't really hold "renovations to be done" as an "asset in inventory". It'll be something like uber, ie. a marketplace, not a market maker.

That's fair and I agree that it's friction-reducing but not market-making.

Except for really basic stuff like snaking a toilet, home repairs and remodeling are all different. It's not a standardized commodity service like flights, hotels, and cabs where the majority of customers all want pretty much the same thing.

Some would say the real estate brokerage market isn't that competitve. The national realtors association keeps prices artificially high, there is room for ibuyers or market makers to offer a better service, and charge less than 6%

All home repairs - it's a damn nightmare. Home building - complex, slow and expensive RE agencies, seem to compete... but it's an oligopoly market. (just to stay in the RE segment)

And all three have severe government monopoly powers and competition killing regulations.

I think you're probably right. And in order to bootstrap the necessary volume (and starve their competitors), they were willing to actually pay more than market (similar to Uber's investor-subsidized rides in their early years). So the problem isn't so much that they lost money (which they probably expected for the first phase of the program), but that they lost more money than expected (and could tolerate). The additional cost and carrying time associated with labor and supply shortages in the construction industry may have contributed in that case, though I agree with the article that the adversarial market aspect was probably the biggest factor: in a competitive market, particularly bad offers were particularly likely to be accepted.

>The additional cost and carrying time associated with labor and supply shortages in the construction industry may have contributed in that case

And property taxes. And security. I do not know who ran the numbers on this program, but anyone with even a little bit of real estate experience was wondering what the hell they were doing. Blackstone has been around and killing it in real estate for 20+ years, and you would figure something obvious like what Zillow was planning to do would be on their radar if it made sense.

Market makers also typically make money on their spread, paying slightly under market rate for an asset and selling for slightly over. They can do this in, say, stock trading because people want to make trades right now to capitalize on moment-to-moment conditions, so that's the price of convenience and liquidity.

Zillow SAID they wanted to be a market-maker, but they ACTED more like a flipper or investor, expecting to make a profit on the houses themselves, not just on the spread. When they started making above-market offers on homes they pretty much left any pretense of market-making behind: at that point, you need to either sell at a loss or hold inventory and go (a little) long on your investment. Zillow realized they didn't want to hold inventory.

It doesn't help that houses aren't nearly as liquid or fungible as stocks: buying a house, prepping it for sale, accepting and offer, and closing takes months, and all that time can eat into your spread. Zillow would probably always have ended up in a situation where they were more concerned with the sale price 2-4 months down the line rather than their spread.

In the bay area there seems to be some companies that do this. Every time I turn on the radio there is an ad for "John buys bay area houses".

I think they do exactly what you say, give a quick cheap offer, then do some fixing and sell the normal way for a profit.

Basically, if you can convince someone to sell for x% less than the real price today, you can make x% - some costs. Of course this has huge variability, but the company I noticed do it in one market they probably know extremely well so they can make good bets.

I inquired about selling to a few in a different region. They all expected you to pay for repairs they identify. That's far too open ended of an agreement for me. Perhaps it was limited to structural issues, but that wasn't clearly outlined. I also assume they complete their own repairs using their own staff. I expect that gives them the opportunity to quote me at $1000 for a repair while their cost is lower.

In a red hot market, I opted to let the market decide the price and terms.

I think if you can it is almost always better to do it on the open market.

The exception being that there is a lot of middleman involved, loan agents and purchase agents. So cutting out these should save at least 5%.

Agree, but there is a tiny bit little more in depth to it.

* If you got a lucky market you could’ve made a lot of money. But I had far less to do with the renovation part and just the I happen to buy low and sell high part.

* A LOT of flippers lied about it being their primary residence. That helps with loans, although there are more serious paperwork and requirements since 2012 or so. FBI paperwork iirc.

* The math changed if you were your own crew. If you have the time, and the skills, it definitely change the equation. However, you basically had to make this your full time job, and there aren’t a lot of competent non-professional “full stack” contractors flipping for fun. It’s a lot less stress to just be hired for jobs.

Rising interest rates will be a drag on the market. It’s clear those will eventually come, but maybe not for five years.

Pundits have called 23 (or more) of the last 2 housing market drops.

I'm looking at the current RedFin value of my house, the statement that "A lot of these properties are purchased by companies" and "Zillow is getting out of the flipping business" and thinking it was a problem of their own making. They've overheated the market to the point that they can't make any money, and a lot of people were having to spend more cash than necessary on a house that wasn't worth what they paid for it.

I don't have to move, I expect a 30% drop in value...it probably won't be as bad as 2008.

The post seems more like an explanation for how Zillow they lost a bunch of money going long housing in a real estate boom than why Zillow is shutting down Zillow Offers.

I think the issue is around labor.

When a contractor g team has the skills necessary to do a house renovation sufficient to flip, then Zillow is really just financing it. Financing is not really an issue for house renovations because homeowners provide the financing, or small-scale flippers self-finance. In any case a loan is really all that’s needed. I can’t imagine a contractor ever doing contracting-for-hire by a mega corp when there’s so much demand from homeowners flush with cash as-is, to the point where skilled contractors can basically name their price.

Yes. I had been thinking the same thing through the various zillow threads here recently: none of the construction contractors I know would ever work for a corporation. Unless they were starving, which they are not.

If their Zestimate is any indication of their algorithm's proficiency, then that might point to the problem. The range of values they give for a home are frequently ~30% wide. Then they just pick a midpoint. So, it's "we think it's worth $700K, give or take $100K. Not exactly rocket data science.

I previously just chalked that up to the whole estimate being a marketing thing anyway. And figured that if they did have some brilliant data science accurately predicting home prices, why would they give it away?

Now that I see they were consistently overpaying for houses in a rising market--even with a presumed premium baked in for the convenience of an easy sale process--it's looking like their algorithms are indeed suspect.

I doubt it. It would mean Zillow had some type of insider knowledge regarding interest rates or the macro economy. We aren’t in a 2008 scenario either regarding subprime. Perhaps they have an insight that no one else sees?

I think you should also consider first and foremost the CEO's primary stated reason, which is introducing too much volatility for the company. He's said that Zillow aimed to be a market maker with +/- 200 basis points, but that Zillow Offers ended up swinging +/- 700 basis points. This volatility makes Zillow less like a reliable publicly traded company with regular growth and earnings, and more like a housing hedge fund where they need to hold onto a huge pool of capital to cover liquidity swings.

> We have been unable to accurately forecast future home prices at different times in both directions by much more than we modeled as possible, with Zillow Offers unit economics on a quarterly basis swinging from plus 576 basis points in Q2 to an expected minus 500 to minus 700 basis points in Q4.

> Put simply, our observed error rate has been far more volatile than we ever expected possible and makes us look far more like a leveraged housing trader than the market maker we set out to be.

> We’ve got these new assumptions that we’d be naïve not to assume will happen again in the future, we pump them into the model and the model cranks out a business that has a high likelihood, at some point, of putting the whole company at risk, not just the business, but in the more normal case, just causes a ton of volatility in earnings, which is not a great look for a public company. That’s basically what it boils down to.

He also acknowledged the operational issues, but in reality it seems like the volatility + scale of the capital needed is too much for Zillow to risk on top of their existing business and status as a public company.

Stratechery had a good article on this which is where I pulled those quotes from, but it is behind a paywall.

I keep hearing this line that Zillow must have Cassandra-like visions of a housing collapse and that's why they're bailing. The obvious question is, if they have such good foresight, what possessed them to overpay for houses so much in the first place?

I believe we are approaching a top. A lot of otherwise-undesirable places to live have doubled and tripled in price recently (while homes in desirable cities haven't moved much at all), fueled by COVID-19 and remote work spiking demand temporarily. Basically, city-dwellers paying city prices for rural real estate. I don't think this is a sustainable state of affairs and I believe this has created a giant bubble in home prices outside of cities. Something has to give. Either the price of homes in cities doubles, or the price of rural/suburban homes falls back down to earth.

Real estate agent checking in here: the thing about real estate is that it's local. So sure, there will be geographic locations or even segments of a market that are in a bubble and there will still be locations that are undervalued. Everything about my rural market would indicate continued sustainable growth for the foreseeable future, but I wouldn't dare prognosticate about another market without consulting local experts.

Eh, I think we'll see the nationwide housing market basically do what the Bay Area market did in 2018. Buyers dried up because prices outran incomes, the overbid vanished, and houses basically sold at list, leading to a ~10% decline from what they were selling for at the beginning of the year. Then it stagnated for a couple years until the COVID reopening, and has since shot up ~30-40%. The increase appears to be durable; unlike in many other regions, Bay Area techworker total comp has gone up like 50% since COVID, so houses are actually more affordable.

> Bay Area techworker total comp has gone up like 50% since COVID [, so houses are actually more affordable.]

That's a pretty shocking claim, even if you're exaggerating to make a point; do you happen to know if anyone is tracking any data for this?

The only data that I can find (non-tech worker) seems to indicate that things are still significantly worse in California and the Bay Area specifically, versus the country as a whole. This is dated Nov 2021, but it's not tech-focused, and, well, things don't look great: https://www.bls.gov/regions/west/summary/blssummary_sanfranc...

It's based on stock prices and levels.fyi (which itself has pretty shocking numbers, eg. new grad comp at any of the FANGs is pushing $200K/year). Stock prices of most of these companies have doubled in the last year; some recent IPOs (eg. Coinbase, Roblox) have gone up 10x. Someone who gets paid 50% in stock at a FANG, which is pretty typical from the levels.fyi data, has seen their comp go up 50%. Someone at one of the recent IPOs with a pre-IPO grant is getting paid in the multi-millions per year.

Things are significantly worse if you are not in tech, but then you are not buying houses. Things are significantly worse if you are not in tech largely because you are not buying houses - everybody who is not in tech is largely getting squeezed out of homeownership here. This too is part of my prediction for the rest of the country.

> Someone at one of the recent IPOs with a pre-IPO grant is getting paid in the multi-millions per year.

someone with existing grants should not use that grant when considering their compensation _this year_, even tho it's vested this year. Only new grants granted this year should be considered your compensation.

Otherwise, you're not taking into account the risk of holding equity - a priced risk. Imagine if you were paid in lottery tickets - if you happened to win with those tickets, you still would not consider the old tickets that won as this year's compensation.

Eh, depends on context.

When figuring out your personal financial budget, you're right. There's risk involved in being paid in equity, and stocks can go down as well as up. Plus, people paid in cash can also receive an equivalent deal by buying futures when they start their job, effectively paying a premium to take on that same market risk.

When explaining why houses are priced the way they are, the market value of today's stock compensation absolutely matters, because that's the resources that your competition has available to buy houses. You can look at it as an asset swap: you are trading 1000 shares of GOOG, and the rest of the homebuying demographic also has 1000 shares of GOOG. If GOOG is trading at $1500/share, that's worth $1.5M; if GOOG is trading at $3000/share, that's worth $3M.

To use your analogy - imagine that you're paid in lottery tickets, but everybody else in your town is also paid in lottery tickets with the same numbers. If your lucky number does not come up, you all remain poor and houses cost $100K. If your lucky number does come up, you all get $100M, but suddenly houses cost $100M too because new houses have not magically appeared for everybody to buy.

If 50% of your income is RSUs, and they increase in value by 50%, your total compensation rises 25%.

Ex: $100k base, $100k stock. 50% RSU appreciation = $150k. TC rises from $200k to $250k (100+100 => 100+150).

Yes. Stock price has doubled (sometimes more). If 50% of your income is RSUs and the RSUs increase in value by 100%, your total compensation rises 50%.

I feel that there are lots of contractors that can flip houses, with a better profit margin than Zillow. Contractors have the capital to outright purchase a home, no middle person as they do the upgrades, they also know the local market.

Perhaps with large enough capital, the money is in building smaller condos, commercial units. As the initial capital layout is greater than what a small contractor can afford to do.

What markets was zillow operating in? If it was in a particularly constrained market, there is no top over say a decade of ownership. Zillow could hold and rent properties and in 10 years, even if we had 2008.2 tomorrow, they'd be worth more than they are today. Then again that sort of longterm investment thinking doesn't click with modern shareholder controlled companies.

I feel like labor and materials would be included somewhere in the forecasting model for pricing. Fairly hard to ignore either in, well, any business.

Housing is a local game, even at the peak of the housing crisis some places did not fall by more than 10%.

Hi I'm the author of the Prophet article https://www.microprediction.com/blog/prophet which for better or worse is why many people have recently questioned Prophet as a time-series method. (There are better, and certainly more formal critiques before and after - I referenced several).

I just hope those who push back on your article read all the way to the last paragraph. I'll repeat it below. Very well put.

I'm happy to answer technical questions about Prophet from anyone here but again, this is somewhat beside the point, which is...

The requirement that people come to your company knowing how to use piss easy baby tools is an extremely dumb and lazy hiring practice. It is also, unfortunately, a common practice in data science job postings. The aggregate effect of this practice being widespread is that talented people with unusual backgrounds get gatekept out of good paying jobs that they’d be exceptional at. Making fun of the job posting and using Prophet has been compared to gatekeeping. To be clear, the Prophet prerequisite is an actual form of gatekeeping being undertaken by a major company that has actual material impacts on people’s careers. The job post excludes people not based on aptitude, but based on whether they have previous experience and familiarity with a tool they could be introduced to and then master in under 15 minutes. A tweet making fun of the job posting is not gatekeeping. Get over it, LinkedIn clout chasers

I would add that since posting my own less-well worded version of this astonishment I have received numerous DM's from people at large companies who are aghast at the way Prophet is a favorite of management. So whether or not this was a problem at Zillow beyond, say, 2015, it might well be the case elsewhere.

If claims to have good knowledge in TS, but had not played around with Prophet (which is dead simple, takes a few hours - or 15 minutes for you - to see some problems), how interested/knowledgeable are they in TS? (I assume that they look for candidates knowledgeable in that field, and just playing around with prophet dosnt make you knowledgeable)

Especially in TS is saw many PhDs, really focussed on one specific method, not looking left and right and neither are pragmatic about their choices. Which is essential to solve anything in real life. Filtering candidates out, based on criteria like "played around with different stuff" is a good indicator for open-minded people IMHO.

Your blogs are awesome !

I check on them every few weeks to see, What you wrote next :D

This was pretty stupid straw-man. I doubt anyone who has used Prophet is under any illusions as to what it is doing under the hood (after all, the only input is a single time series dataframe...). Companies use it to do quarterly/yearly goal setting and anomaly detection, not deploying it to production to power product features. This whole thing has the tone and substance of a giant "i am very smart" faff.

The author also seems annoyed that people get paid 200k for such little "technical" skill, to which I would ask why he cares?

>to which I would ask why he cares?

Do you think he didn't think enough about the reasons he stated in TFA for why he cares? Or they're not good reasons?

He says, from the employer's perspective,

"$200k a year can attract people who actually know what they’re doing. Maybe a math or econ PhD. Maybe a Microsoft Excel pros with 10 years of finance sector experience";

from the prospective employee's,

"The aggregate effect of this practice being widespread is that talented people with unusual backgrounds get gatekept out of good paying jobs that they’d be exceptional at. ... The job post excludes people not based on aptitude, but based on whether they have previous experience and familiarity with a tool they could be introduced to and then master in under 15 minutes."

I think reason he gives about gatekeeping is only weakly related to the bulk of the article. The point is that job listings shouldn't be arbitrarily specific is an obvious one that most already agree with, and was a thin excuse to show off his deep knowledge of modeling jargon.

If you read the article further it morphs from "you gotta have a better time series tool bro" to an article about "if you don't know what adverse selection is, no amount of highly-paid machine learning engineers will help you".

This, I think, it a more valid point (and it was not only made by the author).

Or, let's say, it is at least a fun implication: When companies like Amazon and Google hire hundreds of Econ PhDs to design platforms, market ops and auctions, it IS amusing to imagine that Zillow thought they could just "Machine Learn" their way through it and they fall flat on their noses.

Very likely this was not a real job post, but rather the statutory post required for a visa application (H1-B, etc). Zillow HR looked at the current employee's background for a set of skills that is rarely found in one person, to keep the number of qualified outside applicants at zero. "Must be a a senior person with recent hands on experience in this novice tool". What they didn't say is, their exist employee used Prophet in a class unrelated to work.

Similarly in Singapore - a lot of jobs require that, even if a direct candidate is ready to be hired, the job is advertised for a month (government mandate). As a result job ads are poor quality and extremely specific to get around visa constraints.

This is an excellent point, and whether or not it happened here it is often what happens, not only in developer jobs.

BTFD.

Zillow issued $4bn corporate bonds that they only have to pay 2% on annually while they pay it back over 10 years. This is way below their annual revenue which was already $2.7bn annually. The bonds were for the expansion of Zillow Offers, but based on their prior revenue. And now, they are trying to sell a bundle of homes for $2.8bn all at once “at a loss”, with no information about the percentage of loss. -5%? -25%? Does anyone else see that it doesn't matter even if it was 100% loss which it clearly isnt?

People out here acting like this is the great financial crisis they’ve missed 12 years of alpha just waiting for.

This is the least leveraged market participant in real estate lol. They flipped a few (thousand) houses maybe at an overall profit, made some of their developer and overleveraged home owner friends happy, and found an excuse to fire a bunch of their workforce!

This game is obvious

> The main reason why competitive market price movements tend to be stochastic and I(1), and tend to exhibit little highly profitable predictable seasonality, is because if that wasn’t true, then people would be able to make free money and markets would no longer be weak-form efficient.

Oh no! Someone invoked weak-form efficiency! I guess every market maker should shut down now?

OP is trying to criticize the hypothetical Prophet-trader because Prophet seems to rely on trivial seasonalities for forecasting. But he is ignoring that the bare minimum configuration for Prophet requires several injections of domain expertise (or at least bias):

- Prophet predicts event frequencies, not prices/events. So presumably it must be used in conjunction with another price model.

- The Prophet user must acquire and group data for their forecast -- which is itself a form of locale-sensitive regularization, and is often >50% of the challenge of calibrating a predictive model.

- Prophet users configure market sizes, change points and event streams ("holidays" are a special case, not the only example) based on external data.

Obviously none of these things saved Zillow, but they are all outside the conditions supported by weak-form efficiency, which gives you a good idea of how useless that concept is in practical trading.

I have zero investment in Prophet and wouldn't use it for a trading model, but it's annoying to see Data Twitter's culture of assuming others are dumb in order to quote grad school stats classes getting rebroadcast elsewhere.

I see some decent points in the post, but they are mostly about modelling.

What I'd like to say is even though you have a good model, that doesn't tell you what to do.

If you know it's going to rain next week, should you go corner the umbrella market?

If you are going down that route, there are a bunch of non-weather things that you'll need to know. Perhaps where to find a cheap source of umbrellas, likely places where people will be the most susceptible to buying, and so on. Basically domain expertise.

The major issue you will have is whether your own presence causes your model to be inapplicable. If a guy sees you at the umbrella factory, maybe he will decide that's a signal that the weather will rain next week? Maybe people will just stay home in that case?

The main thing I see with Zillow is a lack of due diligence. There seems to have been an idea that you can just have some data people, pour in data, and out comes money. In reality there's a lot of legwork in any business, and you pay for experience with money. In this case also your job.

Another ignored issue I see is that people choose the modeling methodology they use, the scoring methodology, and the data that goes into the model.

The reactions act as though the model has a life of its own and that some unlikely error led to these results.

Purchased data/models can frequently be wrong or unhelpful to one’s modeling objective.

Finance also has a number of risks vs rewards with every decision.

Is it possible that Zillow artificially inflated the housing market?

They bought up bunch of houses while having the most popular pricing tool? That's a strong incentive to predict ever increasing prices. Plus it whipped up a frenzy and site traffic. Dangerous incentives all around. You can see how this could be self reinforcing with systems that all want the numbers to go up.

Per this US Census and HUD report [0], there have between 600k (2018) to 1M (2020) single family home sales per year over the years since Zillow started Offers. Per this Barron's article [1], Zillow bought 9680 homes in Q3 2021 and less than half of that in Q2 2021.

So, yeah, they've probably driven the price up a little bit, but their purchases represent maybe 2% of all sales, so I assume the impact that has on the market in general would be pretty small.

[0] https://www.census.gov/construction/nrs/pdf/newressales.pdf

[1] https://www.barrons.com/articles/zillow-opendoor-stock-price...

Zillow only bought houses in certain markets (24 different metropolitan areas [1]). So their impact on prices within those regions would've been bigger than what you'd expect by comparing them to all home sales in the US. I admit that I have no idea how much bigger, though.

[1] http://zillow.mediaroom.com/2020-08-04-Zillow-Resumes-Buying...

The question should be, 'To what degree did Zillow influence the housing market?'

Interest rates are low so people have cash. New housing development was lacking even before Covid disrupted supply chains. Now we are facing a labor shortage that could play out in a number of ways, not limited to proletariat uprising or in contrast, a new age of entrepreneurialism.

I agree. Those low interest rates are also driving a ton of refis which people are using to make improvements and renovations. There's tremendous pressure on trade labor.

Also, I suspect immigration restrictions are putting a squeeze on the supply.

Zillow is just an aggregate typically MLS data. Sure it can probably have some influence via it's Zestimate product, but that's not what failed. Anecdotally 1/4 of the folks in my neighborhood currently have a for sale sign in their yard because of the price increases. I'm not sure where this upward pressure is coming from, but I fear another 2008.

> Anecdotally 1/4 of the folks in my neighborhood currently have a for sale sign in their yard because of the price increases.

If I might offer some thoughts as a real estate agent... so we are seeing some of this in my market as well. In particular, we're seeing houses that are sitting on market longer than they have the rest of the past two years. This is causing many people in my market to prognosticate that the market is slowing down / we're in a bubble / this is the top, etc.

Instead the problem is that there are sellers who see dollar signs and are listing their homes much higher than the market can actually bear. It's not that there aren't buyers or that the cost of capital has increased, they have just gotten greedy as seller. So - I don't know what market you are in, but my guess would be that many of those opportunistic sellers will be disappointed and that the market will self correct - not that it will crash, but simply self-correct.

That makes sense. The ones I do know that are selling - they're not tied to this area by family/jobs/other commitments and they probably see an easy cash grab because of that. One family was in their house for 9 months then sold it for $80k, or ~25%, more than what they paid for it (according to Zillow). Honestly not a bad payday but how long can something like that last?

>Honestly not a bad payday but how long can something like that last?

IMHO, it could be a while. Every market is different because real estate is so localized, but... there are relatively few asset classes. The general public invests in two of those: stocks and real estate. Sure, interest rates might rise but anything below 5% is a 50+ year low. Even pre Covid we were beginning to see a significant population reorganization, in part due to climate change, remote work, cost of living, and generational preferences. We simply do not have have enough housing - or enough of the right kind of housing - in areas people want to move. So, at least in my area I don't see a obvious stop to the demand.

That isn't to say that people are going to see an $80K increase in 9 months ongoing, but just that the appreciation will be there and the demand will continue.

The economy itself has been good the last 12 months in coastal cities. The upward pressure is coming from those who have money to spend upgrading the life.

I don’t fear another 2008 since the mortgage market hasn’t gone as crazy. I do think we’ll see a correction, but it’s more likely to be a slow down and increase in time to sell homes.

I think there is a feedback loop involved. Someone wanting to sell their home will likely look at the Zestimate to help them set an asking price. And then tack on a few thousand for negotiating room.

Zillow monitors the offer amounts by potential buyers, and if any are above the current Zestimate, that signals the algorithm to raise the home's value, even if the sale hasn't closed yet. This then becomes a "comp" for the neighborhood (since other realtors will check their site) and influences nearby sales.

I think this is an extremely likely probability in the markets it operated offers in. Even if they were buying 5-10% of those homes.

I also think realtors were slow to sue Zillow homes to clients, but have no hard evidence to support this. I do see a Redfin listed home in my neighborhood gets 1/4 the traffic of anything else, and I have to believe that influence from realtors. This is something that happens with for sale by owner homes too.

Do people actually use zillow for buying or just for perusing? Most of the listings on my zillow are these stale 60 day old listings where there is obviously something wrong with the listing, it doesn't look like the actual market that realtors are using considering homes here sell so fast. My understanding is real estate moves too quickly and you are going to be putting offers on homes your realtor shows you before they even hit market let alone aggregated on zillow.

I think it's true for any platform for housing markets. Once you have easy access to pricing of houses sold or being sold around you, as a seller you will try to be at the same ballpark and higher. So everyone ups the price by 5-10% so it becomes the new normal and repeat.

Like with most market making operations, your main risk is adverse selection. I strongly suspect that this was what killed the zillow operation. Suppose zillow is on average right with their house price prediction, but in 50% of cases they offer 10% too much and in 50% of cases they offer 10% too little. In their backtest this will show as a fair price prediction. However in reality, they will mostly end up buying the houses where they overpay 10%. Only their bad quotes are being hit so they keep consistently overpaying for properties.

I like this point, since it does resonate with what much of the anecdotal evidence was, i.e., Zillow offering too much for houses around the country. The high offers would be the ones accepted (and memorable), meaning Zillow would tend to over pay for houses (leading to the rut they are in).

Imagine being this angry about a job listing mentioning a Python library...

He's not angry. He's responding to people who are angry he flippantly made fun of the job posting.

> The requirement that people come to your company knowing how to use piss easy baby tools is an extremely dumb and lazy hiring practice.

Sounds angry to me!

Nah he's mocking the crap out of them.

He sounded at least a bit put out to me too

Worse. He sounds "fake angry" and spent what looks like an inordinate amount of time writing a crappy article to express his fake outrage. His understanding of time series seems OK at best.

I don't know, maybe, or only I liked it better when there was so much new houses being built that this wouldn't of happened. Some of us People of Age remember growing up in houses built in post-war baby boom buildings, so there was enough of a housing increase that people weren't paying so inflated prices for houses. Maybe this is what we get when interest rates are so low and so little money is needed for down payments, I know a bunch of people who are started getting into real estate in the last year because even though it's lots of money banks make it easy if you have credit.

> this is what we get when interest rates are so low

I think it's what we get when the voting base is homeowners who want to protect property values with restrictive building policy.

Edit: To expound a bit... one has to ask why, given that there is a huge and growing market of buyers and would-be buyers if they could afford it, the market is not producing supply like you describe for baby boomers. My anecdotal understanding of this is that there are just a ton of boomers who don't want to build anymore, who want to "protect the character" of their neighborhoods, and would prefer that new would-be-buyers look elsewhere.

You are absolutely on the right path, IMHO... in my market we have a lot of land, but the zoning of it is such that it is arduous to build. Likewise, what can be built profitably is not necessarily affordable for buyer. I see gaps at every price point - the first time buyer has to either buy a poorly built new construction that they can barely afford, or they have to buy older construction where they may not have the equity or capital to property maintain. At the top of market, IE: luxury, builders are hesitant to build without a ready buyer because of the massive outlay, especially knowing that those buyers are very selective. Middle of the market you end up with properties that smell of builders trying to push up price and perceived amenities while being very careful to not push up their costs. So you end up with nice houses on terrible lots, or you end up with cheap construction with a varnish of granite and stainless steel appliances to make it look better than it is.

It's normal to assert that open source generates millions or even billions of dollars that isn't captured.

In this case, it seems like an open source project destroyed billions of dollars.

That's a new idea for me.

The library didn't destroy any value.

The morons who thought that they would be able to gain a competitive advantage in a highly competitive market just by applying a publicly-available library did.

Well, it made money for someone, just not the ones who used the product.

A good prediction is one which is made BEFORE the outcome. All of these armchair data scientists had the opportunity to short Zillow BEFORE it was obvious that zoffer was losing money. No one did? The best outcome which could flow from this is a sort of adverse peer review in data science based on asset trades. To be honest, the zillow data team should have sought this themselves, the community could have debugged them.

I considered shorting when the Offers program was announced. Thankfully I didn't because I'd still be upside down on that bet.

Extremely well written article for someone who would like to dig deeper into forecasting, actually giving someone ne valuable links about the prophet tool that i used and recommended. I agree with the author that it is strange that the job posting only mentions the prophet tool, which is rather like a very basic and one particular library that i usually recommend to undergrads for their senior project.

I'm surprised that nothing here has been written about human in the loop. All time series models are flawed in some capacity, and are going to have difficulties in the current market given how it won't necessarily match historical seasonalities. Just having more human safeguards to flag wacky outputs seems to be a more reasonable explanation than not handling Prophet's shortcomings properly. Zillow would have likely faced similar issues with other time series packages as well

Reading some more, it does look like Zillow was in fact leveraging humans in the loop https://venturebeat.com/2019/12/11/why-explainable-ai-is-ind.... What seems like a likely explanation is that the proper balance between machine and human judgement was not calibrated to this current market.

"Part of the challenge of Zillow Offers’ human-in-the-loop system — any such system, really — is finding the balance between humans and machines. “In order to optimize this human-in-the-loop system, we’d like to figure out when the human is best, when an assistive situation is best, and […] when a machine is best,” said Fagnan."

Blind is suggesting another reason this went wrong - misaligned incentives and overriding the predicted prices

https://twitter.com/sh_reya/status/1457094567111528455?s=21

I use Prophet quite a bit because it allows me to add exogenous variables (temperature, wind, solar etc.) easily and assess their impact with reasonable outcome for electricity load analysis.

One thing that baffles me is when people try to use it to predict weather itself.

It's instructive to recall that the original use case for Prophet at FB was 'good enough' forecasts for a large number of time series. Both the modeling choices and interface are optimized for that point in design space.

I’m surprised I haven’t seen this rumor yet

https://twitter.com/sh_reya/status/1457094567111528455?s=21

There isn't a company or individual who has models that can accurately predict the future of housing prices. There are only charlatans who claim they can achieve the impossible. This guy can do no better, but he'll do his best to convince capitalists that he's got the right stuff.

My opinion is that we are not at the top of the housing market. People are realizing that it is not fun to live in dense areas.

Local markets may ebb and flow, but desirable houses will rise.

Phoenix is particularly dense and the house prices are cheap, which is why I believe Zillow sold them all-- they realized their actual profits weren't as good as what they projected.

>People are realizing that it is not fun to live in dense areas.

Sounds like you're assuming your personal preferences are the natural ones which "people" surely have by default.

While I respect your opinion, I would argue that if we were to look at the data, it would show some level of shift in people's preferences for where they want to live.

Reply on news.ycombinator.com