That article doesn't really cover the important bit. Yes, the distribution changes, but that's not just something that happens, it's adversarial.

https://mobile.twitter.com/0xdoug/status/1456032851477028870

I'm quite sure there were people in Zillow's data science team who understood this, and maybe even tried to say, hey, this model we developed can't really be used for this new purpose you want to use it. But it's hard being that naysayer.

I was at an interview with some people who were doing some things with machine learning on time series data... a topic I know little about, but enough to have heard of Eamonn Keogh and his many, many criticisms of how that's being done by ML practitioners with no time series background. What they were doing was pretty much everything on Prof. Keogh's "what not to do" list. But how do you tell them that? And how would it be if I'd already worked there for a while when you found out your company's business model has some pretty big flaws from a data science perspective?

As long as they pay well, there's certainly a temptation to say, I only make the models I'm told to...

I heard from someone on the inside that Zillow didn't lay off any engineers. It was all building inspectors and other field staff for their failed house flipping biz.