> Astonishingly, this approach, done correctly, produces better forecasts than traditional simulations of the physics of our atmosphere.
It seems like this is another instance of The Bitter Lesson, no?
It seems like this is another instance of The Bitter Lesson, no?
I thought this was a good quote:
> We want AI agents that can discover like we can, not which contain what we have discovered.
Deep Blue wasn't a brute-force search. It did rely on heuristics and human knowledge of the domain to prune search paths. We've always known we could brute-force search the entire space but weren't satisfied with waiting until the heat death of the universe for the chance at an answer.
The advances in machine learning do use various heuristics and techniques to solve particular engineering challenges in order to solve more general problems. It hasn't all come down to Moore's Law.. which stopped bearing large fruit some time ago.
However that still comes at a cost. It requires a lot of GPUs, land, energy, and fresh water, and Freon for cooling. We'd prefer to use less of these resources if possible while still getting answers in a reasonable amount of time.
It's certainly true that "just throw a bunch of GPUs at it" is wasteful, but it does achieve results.
And even though solutions to many such problems were in the NP or NP-hard categories it didn’t mean that we couldn’t get useful results.
But it still gave us better results by applying what we know about search strategies and reinforcement to provide guidance and heuristics. Even Alpha didn’t use the most general algorithms and throw hardware at the problem. Still took quite a lot of specialized software and methods to fine-tune the overall system to produce the results we want.
Notably forecast skill is quantifiable, so we'd need to see a whole lot of forecast predictions using what is essentially the stochastic modelling (historical data) approach. Given the climate is steadily warming with all that implies in terms of water vapor feedback etc., it's reasonable to assume that historical data isn't that great a guide to future behavior, e.g. when you start having 'once every 500 year' floods every decade, that means the past is not a good guide to the future.
It's not exactly an LLM but it works in a similar fashion.