Also that these new AI systems are performing at the same level of the old non-AI approaches I feel is a real testament to the developers of those systems. This is not something we see in many other cases.
> One would think this would be one of the first areas that would adopt AI given the data available.
Actually, the challenge has been that there _isn't_ enough data available. Sure, we have lots of satellite observations and many other sources, but none of these paint a holistic picture of the atmosphere of the sort you'd need to actually forecast the weather with any precision. ERA-5 - a model-based "re-analysis" that assimilates many observations and tries to create a coherent picture - is only a few years old, and has been the keystone that unlocked all of this development over the past few years.
I mean keeping in mind the number of variables involved and, more recently, rapid climate change, el niño and forest fires that are much more difficult for an AI to keep in mind (I think).
It's a competitor to existing models and supercomputers, but it has to prove its reliability first.
Why would they use AI where weather models exist, and work? What exactly would AI bring into space?
- weather is not climate
- we are not in a situation where weather models don't work
This aint a retail product recommendation. If the AI makes a bad choice it will cost millions/billions of dollars and we could lose many MANY lives.
The problem is that they offer little to no advantage over the highly optimized ML-based forecast post-processing systems widely in use in the industry. You see an awful lot of hype from start-ups proclaiming their AI forecasts are "the most accurate ever"... when in reality they barely improve at all over the status quo that can be achieved with rather simple statistical modeling.