MetNet-3: A state-of-the-art neural weather model | Hacker News Reader

MetNet-3: A state-of-the-art neural weather model

blog.research.google

180 pointsapsec11272 comments

This is interesting in that I've noticed when a hurricane is about to hit Florida you have to filter through all this spam news to get actual actionable information about the storm. The terrible news agencies have no incentive to provide the information because they just need to provide click baity links to serve ads, and it feels like the information you eventually get isn't the most accurate, it's the most sensational. "THIS might be the biggest storm in thousands of years! PLEASE SHARE THIS LINK ON SOCIAL MEDIA!"

If this model can be used by independent media or by me, I could provide a blog which gives accurate information and actually helps people. That's a very interesting turn. I can't tell if this model is released publicly from this article or just available behind a Google service?

And, if it helps further the demise of these consolidated "local" news sites (which are always just content mills owned by some large national owner) then even better.

In Canada, I use weather.gc.ca . It's steered me through some terrible ice storms.

https://www.weather.gc.ca/city/pages/qc-147_metric_e.html

Does weather.gov suit your purposes?

https://www.weather.gov/mfl/

I’ve found the Canadian weather site to be quite inaccurate as compared to a local forecast from weather underground.

Especially when it comes to short term predictions of actual rain, it seems magnitudes better, and it updates its forecasts at a much higher frequency. The precipitation view is only available on the web view for some reason.

That said, the Weather Canada satellite view is indispensable. Even if the site hasn’t changed in literally 25 years.

What actions are necessary on your part?

Having been lucky enough to grow up in New England, my response to cold weather stuff is mostly… go inside, get a blanket and throw another log on the fire. But in Canada you all get a more serious type of cold I think.

Places like Florida or Kansas where the weather will actually come get you inside seem like pretty out there places to live.

> Places like Florida or Kansas

It's not too complicated with tornados. If you're urban, you listen for sirens, then take cover if you have to. If you have a basement, go there.

Otherwise, you just watch the sky when it gets spooky and kind of accept you might get Oz'd at any time. There's not many prep actions to take, other than maybe popping open the garage door and getting out lawn chairs if you have a good view.

Having lived in Europe all my life, it sounds pretty insane to live like this.

I'm thinking of moving to the Caribbean but storms / hurricanes are quite a mental jump for me.

I guess that's the price to pay for hot water in the sea

Hurricanes are a different story, they are highly predictable (even in the very rare cases that the models are horrifically wrong about intensity, like the recent Hurricane Otis, we still got the track pretty close), but gigantic.

Tornados on the other hand are almost impossible to predict. The best our weather service can do is say "this storm is the kind that produces tornados, watch out" (a tornado watch), and to set off the sirens when one is sighted (a tornado warning).

Hurricanes are high intensity over a very large area, lasting for a long time. Tornados are short lived, unbelievably powerful, and cut a narrow path through whatever they decide to mow over.

Hurricanes are also the sort of disaster that the US is pretty decent at handling. They are big enough to be a major new event, but small enough that one only hits a couple states at a time. Small enough to not outdo our ability to focus national resources, but enough to draw our attention.

Hurricanes hitting smaller, less rich countries seem like a much bigger humanitarian crisis.

Tornados are violent but very localized. Even the largest and longest lived affect a relatively small area compared to a hurricane. When it looks stormy you just listen to a local radio station, they’ll broadcast updates in real time. Also local TV news will have the weather guy on nonstop giving minute by minute updates.

Watch some of the YouTube’s of local news during major events like the Moore F5 tornado. Those guys are on top of it and don’t screw around. Getting surprised by a tornado happens but it’s very rare these days.

Severe ice storms in Quebec (and I'm certain elsewhere in Canada) can result in power outages. We had an ice storm in Montreal this year that resulted in most of the island losing power for a week, others for longer [0]. The Great Ice Storm of 98 resulted in the greatest deployment of Canadian military personnell since the Korean war [1].

In cases where we get a heads up on big ice storm, it's prudent for my family to get some gasoline for the generator and stock up on batteries and whatever is missing before the roads become rinks. Charging power banks is a good idea, and making sure you have a battery/crank radio since the mobile networks get saturated quickly.

Ice storms are infrequent, but with the climate changing as it is, I've no doubt they'll be more frequent in the shoulder seasons.

Also, I feel the need to say this. It's not uncommon for some of the first casualties of ice storms in Quebec to be due to someone running their gas generator in their garage or under their car shelter. Please never use generators or camping stoves indoors.

EDIT: Getting a heads up is also a great opportunity to reach out to loved ones that belong to vulnerable populations (namely, the elderly). It's best to shelter together if you're able to.

[0] https://montrealgazette.com/news/local-news/quebec-ice-storm...

[1] https://en.wikipedia.org/wiki/January_1998_North_American_ic...

The site(s) you are looking for are:

nhc.noaa.gov and https://spaghettimodels.com/

The vast majority of US weather data and forecasting comes from the National Weather Service, and you can access it directly at www.weather.gov

There are only 3 places you need:

1. https://www.nhc.noaa.gov/

2. Your local NWS site

3. https://www.tropicaltidbits.com/

Simple. Find your dot on about 10 PDFs, interpret a handful of weather variables, know the safety tipping points of each, don't get it wrong or you may be injured, and check back every 4 hours! Easy.

Usually this is what most people would be looking for:

https://www.tropicaltidbits.com/analysis/models/?model=ecmwf...

TBH, the NHC one is very good. Each storm has a "Forecast discussion" link with specific details on the things that specifically drove the forecast. The NWS publishes something similar for each area forecast, and it is often incredibly insightful.

It isn't necessarily as good as the best local weather coverage, but it might help to point you to which station is giving the best coverage.

And honestly, the local weather office forecast discussions are great, too. If they seem too dense with arcane language, that's actually something that ChatGPT does a great job of distilling. "Act like a professional meteorologist specializing in public speaking. Please read the following technical forecast discussion from NWS, and rephrase it to be more accessible to an audience that is educated, but not experts in meteorology."

Do you want sensational click-bait articles or do you want actual weather forecasts by people who understand the topic and know how to interpret the models? Take your pick. One is simple, the other is not.

The real surprise here is how well the product resembles the early Google mindset. A surprise release of something very useful, working far better than the competition, and also free to use.

Yeah, it is as if some group forgot the new "Be Evil" motto.

Interesting capabilities but why don't they report the capabilities of the model beyond a 24-hour cutoff? What do their 5-day forecasts look like?

[edit] Modern numerical prediction models are pretty good in the five-day range (~90%), I'm guessing the deep learning models diverge rapidly in comparison (though perhaps they're better in the sub-24 hour range). Both approaches benefit from more extensive data collection systems as inputs. See (full text):

"Advances in weather prediction" Alley et. al Science 2019

https://par.nsf.gov/servlets/purl/10109891

Does anyone's 5 day forecast look good?

5 day is closed to wild ass guess in my area. Mountains complicate everything

Anything over 24 hours is pretty much always very wrong in the mid Atlantic region.

How often do you need weather services in the middle of the Atlantic though? Seems like a pretty niche use case, except for the various islands there.

Edit: I thought something was sketchy and rightly so, after searching for "Mid Atlantic Region" I learned that it's actually a region in the north-east US, not "the middle of the Atlantic". Well, learned something new today :)

Mid Atlantic = south of New England north of Virginia (more or less), not middle of the Atlantic Ocean.

Not to be confused with a mid-Atlantic accent, which is possibly what you might get around the middle of the Atlantic if your interpolated a couple of specific local accents between the UK and the US

The Mid-Atlantic [coast] is part of the US east coast around NY, NJ, Pennsylvania, etc rather than literally the middle of the Atlantic Ocean. It's a fairly important region considering how much of the US and global economy reside there.

As someone who both lives in the mid-atlantic and regularly has to plan around the weather, I can assure you that you simply have a confirmation bias. Our forecasts are actually pretty good, even for ones 7 days out.

Where people tend to get thrown is micro-storms during the summer months. They are basically impossible to predict accurately, at best it's just known that a random assortment of towns in a given area will received heavy rain for a short time. Being able to read radar is the best way to deal with this, but it's very short term only (15min to 1 hr).

Reading radar is even harder because you need a 3D view of the atmosphere to understand why storms are coming seemingly out of thin air. This is especially a problem near rivers. We only get a 2D doppler slice.

You just have to look at the 2D slice and see if a storm pocket is headed your way. That's the best its gonna get for a layman.

yes? https://charts.ecmwf.int/products/plwww_m_hr_ccaf_adrian_ts?...

I get the impression this model is meant for very localized, short term prediction. Like whether or not its going to be raining in your neighborhood in the next hour.

It looks to me like it's "upscaling" ENS data to high resolution. Global general circulation models work with somewhat low-resolution data about terrain, and this model has found a function mapping the low-resolution weather predictions back to a prediction on high-resolution terrain.

(ed.: true also, but to a lesser extent, for "mesoscale" models (e.g. of just North America with boundary conditions to a global model))

If it did learn longer-range predictions (or the next model does?), I would hazard the model had achieved speedup by internalising the patterns of certain large-scale weather connections, e.g. the jet streams, Walker circulation, ENSO, Gulf stream... which I think will be fine for 99% of cases, the 1% being if these established patterns break somehow. ("freak weather")

At that point you would have to return to a general circulation model. When you take away the long-lived circulatory features that are familiar to us, and that are particular to Earth, predicting the weather is "just" fluid dynamics.

These are both just wild guesses, though

How cool! The paper was last revised on Arxiv over the summer; this blog post announces that MetNet3 is now powering weather predictions across google devices and services. (I'll bet it gets picked up and used for electricity demand forecasting if it hasn't already.)

From the paper: > While ground based radars provide dense precipitation measurements, observations that MetNet-3 uses for the other variables come from just 942 points that correspond to weather stations spread out across Continental United Stated (CONUS).

I don't know a thing about weather prediction, but the fact MetNet-3 can do it using data from less than 1000 points across the continental US is surprising.

The other line that stood out to me was: > On a high level, MetNet-3 neural network consists of three parts: topographical embeddings, U-Net backbone and a MaxVit transformer for capturing long-range interactions.

If I understand it correctly, MetNet-3 is sort of abstractly treating 'predicting the weather at each geographical patch' like a very big computer vision problem.

Where can I pay for an API that I can use personally? Seriously, I'm sold. I've seen other prediction models. This one looks fantastic.

I'd love to see how it fared with last month's Otis since it caught basically all the currently used models off guard.

My silly question for the day: Could you use interpretability techniques on this model to figure out the easiest places to perturb conditions to steer towards particular weather in a particular location (cloud seeding, or whatever?)

Is this the first part of the weather control machinery from Star Trek? In order to control, one must first predict?

It doesn't seem like MetNet outputs a complete 3D atmospheric state, just specific (and mostly surface-level) forecast predictands [1]. The analysis you're describing could be done with a traditional numerical weather prediction and data assimilation system (especially if that system implements 4DVar, since you'll already have the model adjoint available - well, technically the tangent linear, but still applicable here).

[1]: https://arxiv.org/pdf/2306.06079.pdf

I love this line of thought, I've also entertained it from time to time in other domains. (macroecon :D)

To some extent, yes, but you'd need more energy than is practical.

Weather is a chaotic system -- future behavior can be highly sensitive to local fluctuations.

That input might be too far outside of the training conditions for it to believe the sensors. It would probably assume those sensors you perturbed are faulty and ignore them.

Problem with these models coild be that they are trained with historical weather and patterns, but when new weather effects come up they will get worse over time.

These models are not as simplistic as you imagine.

they might not be simplistic, but we've already got experience with google making a model and then external conditions change, invalidating the model- see Google Flu Trends. After the team launched it and got their promos and moved on, the zombie jobs training the model failed to make useful predictions the next year. The model was not simplistic; it's just that it didn't generalize and needed constant attention from humans.

Maybe not as magical as you imagine

You're completely right. This model, as with any other predictive model, is subject to degredation in performance when the data-generating process changes. But given MetNet-2 came out in 2021, they'll probably release an updated version before the performance degrades due to changes in weather patterns.

Opposed to training them on future weather and patterns?

opposed to first principles modeling. the problem isn't that we don't have good weather models, the problem is they're chaotic, so you need exponentially more data for linear increases in forecast window. and I do literally mean exponential there.

I wish this had an API to be used via other weather apps. There are many native iOS weather apps whose interface I prefer to the Google Search app, and I'm sure the big ones (like Carrot Weather) would add this as a weather source if it were an option.

It might not come as much of a shocker, but a couple of researchers last year claimed a strong correlation between raw compute and predictive power in various fields/domains, including weather forecasting: https://arxiv.org/pdf/2206.14007.pdf#page=10 (page 10 specifically has graphs on Weather Forecasting vs. Compute).

In their study they claimed a strong correlation in these fields (vs. compute):

* Weather Forecasting

* Protein Folding

* Oil Exploration (at BP)

* Chess

* Go

... The latter 2 being games, which I personally do not find surprising. But I do find it inspiring that we can "just" calculate our way out of some important issues. That hopefully translates well to other fields.

Thank you for that link, Ive been subconsciously holding off on assuming there was a compute / predictive power correlation even though it seems natural. But it would probably be dangerously naïve to have assumed that connection. Anyway good for us! Go humans! (computers)

Glad you found it useful.

A small caveat, though: The correlation is linear with the logarithm of compute. So here's hoping Moore's law & friends live on a tad longer!

And a somewhat unrelated fun fact: The authors surprisingly found the lowest correlation between compute and the performance in the domain of Go (and not the real world). Although the data is very sparse, I suspect that it's due to algorithmic advances.

>I do find it inspiring that we can "just" calculate our way out of some important issues.

In the case of oil exploration, we can calculate our way into some!

Calculate, suffocate, annihilate. The fact that we now need computers to find oil says a lot about how much we've already used up, and the precarious position we find ourselves in as a result.

Computers have been used in the industry since at least the 1970s. These days, every major oil company has $XXX million in supercomputers cranking through seismic data in a number of extremely computationally intensive ways. Reservoir simulation is also computationally intensive, but can be done at the individual workstation level.

Here's a paper from May 1985 titled "Applications of supercomputers in the petroleum industry" - https://journals.sagepub.com/doi/abs/10.1177/003754978504400.... Found that without even looking for oldest example.

Correct me if I'm wrong but given that weather patterns are fundamentally chaotic, at some point throwing more compute at the wall probably won't produce anything better?

It's true - the Lyapunov exponent shows that even arbitrarily close points in the system's phase space become separated by exponentially larger distances in time. So even with a computer the size of the universe, you can't really go further than 14 days. I'd highly recommend this Omega Tau podcast episode if you're interested in hearing more about chaos and predictability:

http://omegataupodcast.net/119-chaos/

This is literally one of the defining points of a chaotic system.

In that case there's probably still room for improvement. Hell, having properly reliable predictions for at least one day ahead in all cases would be stellar.

That's an interesting paper (if a little lightweight) but it's begged the question: why is temperature measurement variance the rubric for evaluating weather models? (from footnote on page 9): "Consistent with the norms in this field, only the error in the prediction of maximum and minimum temperature is shown, but this result holds when we use other temperature indicators such as average temperature."

Any meteorologists on HN able to weigh in?

I mean...it's surprising this is a paper isn't it? Atoms, mass, charges, energy, forces etc are largely understood. The problem with weather, protein folding, oil exploration (and many other things simulations are run to predict) has always been that you can't do enough calculations in any reasonable amount of time (/money) so you have to figure short cuts which are approximately right. It's the same as graphics.

It's self evident that the answer to a lot of these things is just "more compute" and "better shortcuts". Like, GPUs and deep neural nets.

All this research from Google and still no discussion of using their access to billions of barometers from Android devices? sigh.

Who has more barometers in the field: Google or Garmin?

Most of Google's are always connected to the internet. Less so Garmin.

Cool work and I hope to be able to use it some day.

Most importantly though, does anyone know how they made the animation with the data sources? I feel like that came from something lightweight and convenient and I'd like to know what it was.

I was just recently wondering when we'd see some new weather models released that keep pace with the development of machine learning. Now if I could just access the raw forecast data from MetNet-3...

It's already happening. For example, ECMWF provides experimental forecasts by their own deep learning model, plus models from Deepmind, NVIDIA, and Huawai.

When I search with my Google app it says "Source: weather.com"

Here's the paper about MetNet-3:

https://arxiv.org/abs/2306.06079

How well did it forecast the rapid intensification of Hurricane Otis where other models failed?

Hold on, Google has its own weather model? Are they they only one that use it, or do agencies use them as well?

Reply on news.ycombinator.com