GenCast was trained on data from 1959-2023, so no surprise it can "predict" back 2019.
It's like the super trading algorithms who achieve perfect scores during backtest.
The question is, how does it perform on unknown events.
It's like the super trading algorithms who achieve perfect scores during backtest.
The question is, how does it perform on unknown events.
From the linked article:
> GenCast is a machine learning weather prediction model trained on weather data from 1979 to 2018
and a google blog https://deepmind.google/discover/blog/gencast-predicts-weath...
> To rigorously evaluate GenCast's performance, we trained it on historical weather data up to 2018, and tested it on data from 2019
I am guessing they did not want to set up the data pipeline to run inference in a live setting. But that is what I would need to see to be a true believer.
Still a cool result and article though.
The Google model is probably the best so far but ECMWF's own diffusion model was already on par with ENS and many point-forecast models (graph transformers, not diffusion) outperform state-of-the-art physical models.
What is missing is initialization directly from observations. All the best-performing models initialize from ERA5 or other reconstruction.
And GenCast was tested against and older model which performs worse.
> The ENS system has improved significantly since 2019, according to ECMWF machine learning coordinator Matt Chantry. That makes it difficult to say how well GenCast might perform against ENS today.
And the testing makes it "difficult to say." The obvious conclusion is "run a new set of tests" but they'd rather pay of the verge to publish half truths instead.