Winter is coming
The only slight consolation though I find is that this time when the fog clears it's not going to be a complete waste as we're going to have much improved data engineering processes, data gathering methodologies, some scale improvements and improved data parallelism and a further sizeable portion of the research field will be cut off and put into the "that's not real AI" category and used in production software. There will be doom mongers, but if we come out of this with a much more professionalized interaction between software and the physical world then it was all still worth it all.
Come back again in 15 years and we'll find a new generation taking yet another crack at this building on the missteps of the now.
It's also totally in something where it's obvious that this should work. We aren't even using "latest and greatest" ML algos, I'm pretty sure what we are using is a really cobbled together ML stuff from a few years ago, probably "latest and greatest" from half a decade ago when ML was just kicking up.
But holy shit, there are so many interconnecting and annoying bits in the non-ML part of the stack (where I am). Our codebase has gotten rather messy (for understandable reason) trying to negotiate leaky abstractions between different clients needs and international standards (and we're only in 3 countries)... And we have a very broken data pipeline (It works well enough to get the job done but I don't sleep well at night) for making sure there are good pulls for the ML engineers to deal with -- and this is code written by folks who should know better about concepts like data gravity, just when you're doing it hastily on startup timescales and startup labor it's (understandably) not going to come out pretty. And all of this is why I haven't even had time to poke into the AI bits, not even stand up an instance for localdev.
Supposedly our competitors aren't even using real AI, just mechanically turked stuff. Yeah. Of course. Just the real messy domain of dealing with these human systems is bad enough to sink a ton of money without even getting to the point where you have enough money to buy some expensive data scientists and ML engineers.
i am still not convinced about ML winter, as it has found its killer app with advertising (i mean previous AI generations didn't find an equivalent cash cow).
Also: why don't they just specify that this ML model has been trained with this type of medical equipment? Couldn't they make it part of the SLA to use the same type of equipment in the field as that of the training images?