The unnecessary decline of U.S. numerical weather prediction
cliffmass.blogspot.com
cliffmass.blogspot.com
> In The Weather Machine, Andrew Blum takes readers on a fascinating journey through an everyday miracle. In a quest to understand how the forecast works, he visits old weather stations and watches new satellites blast off. He follows the dogged efforts of scientists to create a supercomputer model of the atmosphere and traces the surprising history of the algorithms that power their work. He discovers that we have quietly entered a golden age of meteorology—our tools allow us to predict weather more accurately than ever, and yet we haven’t learned to trust them, nor can we guarantee the fragile international alliances that allow our modern weather machine to exist.
* https://www.goodreads.com/book/show/42079139-the-weather-mac...
* https://www.andrewblum.net/the-weather-machine-2
Also his book Tubes, the Internet's physical infrastructure. He also appears to be working on a book on the electrical grid:
I read Tubes back in 2012 when it was first published, and I enjoyed it. I knew a lot less about networking back then, so it would probably be a good book to re-read now with a different perspective, and to think about how things have might have changed since then.
Anyone on here know what happened to Atmo and if the tech is ever going to come back so people can use it? It's a travesty that this tech exists but isn't being used- a lot of lives would be saved if NOAA were using this already existing technology.
Edit: Apparently as another poster pointed out I had just forgotten the correct name of it, and it is still around. I edited this to reflect that.
I edited my post above to use the correct name, but realize it doesn't make too much sense anymore.
I've also wondered if they have hand tuned the model to match local observations and geography in the Bay Area, and if it could generally apply elsewhere without a lot of manual work? In particular, it is mind blowingly accurate at getting the wind shadow around Angel Island, which is a complex thing to do, because it involves ocean wind that divides into at least 3 separate streams, and then recombines in complex ways. No other model I've seen can predict which of those 3 stream will dominate, but they usually can.
I've noticed the same with e.g. car navigation/map software- it generally works much better in the Bay Area where the developers and companies making them actually live, than elsewhere. I could imagine that in both cases the developers use it themselves in the place they live, and investigate/fix local errors themselves.
Your observation about navigation software is accurate in my experience too :)
GFS does not account for sea/shore breeze for instance, so if you're in an area where that may occur, then you will have to apply your own judgement about the conditions.
Now, if you were to feed weather station readings into an ML model, would the ML model be better at predicting the weather? Well I think it'd be better at predicting the things that the weather model does not model in the locality of the weather station, sure.
I've found GFS fine for that, but it is one of the worst models for inland or near shore sailing, even most of the other popular models that you can access from sailing weather apps like NAM and ECMWF are much more accurate in specific locations with unique geography. The resolution is just too low to account for any interacting geography with GFS. It gives the same forecast over huge areas with radically different conditions.
I don't use GFS just by looking at the wind layer though. Wind layer forecasts do not include terrain or local effects as you noted. But the necessary info is in the forecast and is accurate.
For instance, in the great lakes we tend to have large diurnal temperature swings and therefore strong sea/shore breezes. If the model is forecasting big temperature changes and an anticyclone with low wind-layer forecast, this is ripe for strong sea/shore breezes.
The biggest hazard we have in the great lakes is convective storms (squalls). They do not show up in forecasts because convective cells are very small. However, The GFS gribs do have pressure forecasts, and perception, and most importantly CAPE and CIN forecast layers. Combined with WPC synaptic charts you have the info needed to determine if 1) convective storms are likely to occur and 2) if they do occur, the probability that they will be severe.
I have noticed that where I am, the inland/offshore temperature differential is alone a pretty good predictor of overall wind speeds near the coast, not accounting for geography.
For something maybe more useful on the local scale, you can also look at a model like the HRRR (which I believe does take into account the terrain and other local effects from things like larger bodies of water). While this model only really covers the conterminous United States and southern Canada, I've generally found it good for showing the shorter-term, local weather details, including forecasting convective storms and winds on and around the Great Lakes.
> Anyone on here know what happened to Atmo and if the tech is ever going to come back so people can use it? It's a travesty that this tech exists but isn't being used- a lot of lives would be saved if NOAA were using this already existing technology.
They're a thriving start-up as far as I've heard. Weather is a tough industry. Given that hourly-refreshing, high-resolution forecasts are freely available already from NOAA, I doubt that proprietary forecasts like these really move the needle in terms of protecting public life / property.
I'd like more accuracy for the next 1-5 days as that's the time horizion I tend to use to plan to work outside on various projects, and am often frustrated by rain when the prior day's forecast didn't anticipate any. Or the opposite.
Amazingly, these models are starting to be able to actually predict that, but I agree that not a lot of people care about that level of detail.
> Specifically, NOAA's global model, the UFS, is now in third or fourth place behind the European Center, the UK Meteorology Office, and often the Canadians.
Would love to see evidence of that. It is well established that ECMWF is top in the game. I don't think it is reasonable to just state that Canada and UK are better without evidence.
With that said, I agree the US should improve.
On a tangential note, I worked on weather forecasting a bit before, and talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models. Not sure if they can't attract talent and so are somewhat stuck or if their higher ups are "old school" and can't really see the potential that this approach has to offer. It is just sad that the private sector is outclassing institutions and that these better models won't hit the public domain.
https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-...
https://github.com/ecmwf-lab/ai-models
And they are hiring with competitive salaries (for Europe).
I don't consider 24 hour forecasts "medium range". And I don't really view MetNet-3 a competitor to GFS or ECMWF at all (currently).
Regardless, my criticism is towards the evidence presented in the article and the framing of the article.
I don't think most people realize how much free value they get out of NOAA weather predictions.
Rick Santorum (a former conservative luminary) was pushing for a law that would prohibit making weather data available to the public, but would make it available to companies like Accuweather for free.
The brother of the Accuweather founder was Trump’s appointee to run the weather service.
It’s on of those things that illustrates that no public resource is safe from graft.
E.g., the scientists at the FDA who check the science on safety and effectiveness when processing approvals for new drugs would be political appointees who were vetted for loyalty to the President.
The blogger is Cliff Mass, a professor at University of Washington. He has written two papers on this specific topic (decline of US weather predictions). In general, as a scientist, he has published 100+ papers.
He only linked to his second paper [1]. I also found his first paper [2].
[1] "The Uncoordinated Giant II" (2023) https://journals.ametsoc.org/downloadpdf/view/journals/bams/...
[2] "The Uncoordinated Giant" (2006) https://www.e-education.psu.edu/files/meteo410/image/Lesson3...
But NWS didn’t correct their hurricane forecast to the whims of a mercurial President. Talking about the technicalities of the model library is fine but only after you pass the loyalty test.
I wonder if the listeria issues occurring now is also a symptom of this plus the FDA funding cuts over the past 20+ years.
https://static.project2025.org/2025_MandateForLeadership_FUL...
While AccuWeather denies involvement in Project 2025, there's definitely overlap between TFG's administration and the authors of the plan.
John Oliver of all people had a segment on this:
* https://www.youtube.com/watch?v=qMGn9T37eR8
* https://www.imdb.com/title/tt11110660/
* https://www.theguardian.com/culture/2019/oct/14/john-oliver-...
* https://time.com/5699545/john-oliver-weather-last-week-tonig...
The trend you're seeing in private sector expansion / growth is linked to the massive investments in Earth observation and derivative applications. It's not really a response to anything happening in government. If anything, the federal government forced some of this growth through commitments like the Commercial Weather Data Program.
[1] https://www.npr.org/sections/thesalt/2019/09/17/761682926/us...
Elections have consequences.
> We are not able to determine your qualifications as your resume does not show complete information for each job entry, such as beginning and ending dates of employment, duties performed, and/or total hours worked per week
First, my cv did include everything but the total hours worked per week. Second there was no instruction that I could find for writing my CV such that I needed to list the number of hours worked per week. Third there was no way to contest this. Even the person who invited me to apply who would be the hiring manager had no control over this.
I can tell you, this assuredly leads to the unnecessary decline of any system when there’s an impenetrable hr system sitting between applicants and jobs.
Because people who don't get hired can literally get their Congressional and Senate representatives involved.
This makes getting hired into the Civil Service hard. Internal candidates are usually at an advantage. Veterans always are.
The system is as you described. Internal candidates already know it and veterans, when transitioning out of uniformed service, get a full-blown class on it: how to prepare their "Federal resume" (which is unlike any resume that a reasonable person might otherwise prepare) and navigate the hiring process. So it's that on top of already having relevant experience and credentials or preference points.
I've been on the hiring side for Federal GS positions, and I'll tell you that it can be just as frustrating on that side too.
https://help.test.usajobs.gov/how-to/account/documents/resum...
Government has to comply with all these standards and by asking for hours, they can make sure themselves that someone meets them.
HR in companies has changed radically in recent years.
If you throw away your assumptions, and solely look at the behaviors that the function exhibits, you will find striking resemblance to the behaviors under communism, or fascism.
Money has no value any more. Power is the only currency.
To the public, they are strictly compliant, but the tools, such as the CV, the interview process, feedback, approval process, and so on, are irrational to anyone involved, except the true decision makers.
Their objective is not no longer meritocratic, to hire the best qualified or best performing, but to hire people that strengthen the political power of those that already wield it. It's more than nepotism, it's empire building.
Money can be exchanged for goods and services.
If your money can't buy you admission into university, or property that is sold off-market and privately, or access to off-market and privately controlled jobs, and so on, then your money doesn't help you at all.
Power is the real currency in an unfree market. As in communism.
Nowadays that has all changed. Forecast rain fails to materialize. Heat waves come out of nowhere and last much longer and are much more intense than they are forecast to be. Temperatures at my house regularly go 10 degrees above what is predicted, and regularly hit temperatures that would have been extremely anomalous when we first moved here.
This is all anecdotal. I haven't been keeping records, and I haven't done the math. It's possible that this is all confirmation bias or bad memory at work. But damn, it sure seems like a real phenomenon, and the most likely explanation is that the models were trained on data that don't reflect current conditions because of climate change.
The most striking thing about this is the speed with which things are changing. 36 years is nothing on what should be geologic time scales. I shudder to think what the future holds.
I regularly check 5 different forecasting models (through a subscription in the Windy app and elsewhere) and they are unable to agree on whether water will fall on my head later today. We're not talking long-range forecasting here, just tell me if it's going to rain in several hours or not. There is no forecasting model that gets this right.
However, I don't care if it's 22C or 20C, like you I care about am I gonna need a jacket or not.
I get that it's a difficult problem. And the inclusion of local weather radar to track actual rain has been very nice. But it only forecasts 90 minutes ahead, and rain can develop closer than that some times.
https://magazine.weglide.org/skysight-interview-matthew-scut...
Freakingly accurate weather forecasts for the niche sport of gliding. Builds on top of ECMWF and incorporates refined soil moisture. It’s really good in predicting mountain waves (and used by Perlan: https://perlanproject.org/perlans-sponsors-enable-us-to-fly-... ). Also convergences and thermal activity which is very dynamic…
Incredibly impressive.
I'm more upset about all the interest being paid on the US national debt that -could- have been spent on something great instead.
>NOAA, the National Oceanographic and Atmospheric Administration (the leading U.S. effort)
>The US Navy
>The US Air Force
>NASA
>U.S. Department of Energy
This doesn't make sense. Other nations don't have a NASA, and if they do have a space agency that space agency is almost certainly in the weather business.
JAXA: https://earth.jaxa.jp/en/data/2547/index.html
They also don't have expeditionary militaries like the USAF and USN, which need weather for the middle of the ocean or remote nation, far from NOAA's ability to deploy weather balloons.
And as far as I can tell, the only exposure to weather that the DOE has is for climate change reasons, and their interest in that should be obvious.
There's no waste. NOAA does the US, DOD does DOD, NASA helps out, and the DOE is looking at how climate change will impact the US's energy security.
So reason number 1 is horseshit, and I stopped reading.
NOAA
The US Navy
The US Air Force
NASA
U.S. Department of Energy
Thus, scientific and technical talent and computer resources are diffused over five groups, greatly undermining progress.“
I suppose if Elon has his way, the overlapping functions from these are on the chopping block.
The motivation here is that numerical weather prediction (NWP) models are used for a lot more than just weather forecasting - they're critical research infrastructure and tools. NWP allows researchers to simulate complex atmospheric flows and phenomena which might not be readily or directly observed - or to manipulate observed flows in ways to evaluate dynamical theory and the underlying physics. You can't really do any of this with the AI emulators, which output a tiny, tiny sliver of information relative to the vast array of data you can dump from an NWP system.
Regarding energy usage, you're right up to a point. Today's ML weather models are trained to emulate very large reanalysis datasets (e.g., ECMWF's ERA5) which are produced using the physics-based models. No one has yet to demonstrate an end-to-end AI/ML weather forecast which sidesteps these reanalysis datasets (in fact, the trend so far has been to include _more_ physics-based model datasets for fine-tuning, including archives of historical NWP forecasts or GCM simulations). So there's a massive sunk cost in energy usage to create those training datasets, and then there's the ongoing energy cost of training AI emulators from the ground up. And of course, there's the future energy cost of running more physics-based models to better support the development of AI emulators.
For pure inference/forecasting? Sure. Much faster to run AI models and much lower energy usage. But that's quite literally the tip of the iceberg when it comes to forecast model development.
If the question is can the Americans build a gigantic supercomputer and write fortran programs, the answer to that seems obvious. Yes we can do that when the relevant bureaucracy gets pointed in the right direction.
Maybe but honestly, it doesn't make a ton of sense to me that something that is so power hungry it needs new power plants is going to save energy.
I'm also skeptical that AI can really model a chaotic system better than a.. chaotic system simulation. AI is a statistical pattern matcher. I'm really struggling to understand how that would be better at prediction than a simulation of the underlying phenomena. The whole thing seems like a solution in search of a problem.
The fact that Sam Altman has a 200MW chatbot is not relevant here. For example the GraphCast system from DeepMind runs in under 1 minute on a single TPU device.
There have been real attempts at privatization of weather services in the US, despite the fact that our current system costs pennies a day per person (whether it's used for weather apps, or safe flying/sailing).
Take Cliff Mass - he's a professor of Atmospheric Sciences at UW whose research focuses on weather modeling, climate systems, and atmospheric dynamics. When someone with his expertise raises questions about climate science conclusions, should we approach this differently than when non-experts do so?
This raises an interesting question about the role of expertise and scientific discourse: Is there a meaningful distinction between how we treat challenges to consensus from qualified researchers in the field versus those from outside it? Or should acceptance of consensus apply equally regardless of one's credentials?
Individuals who disagree with the scientific consensus aren't always wrong, but most of the time they are. What separates these people is the amount of evidence they can bring to the table.
When has something being a bad idea or never having been tried stopped political apparatchik's from trying, especially given past appointee's opinions:
* https://www.youtube.com/watch?v=qMGn9T37eR8
* https://www.imdb.com/title/tt11110660/
* https://www.theguardian.com/culture/2019/oct/14/john-oliver-...
* https://time.com/5699545/john-oliver-weather-last-week-tonig...
See also Project 2025:
> The document describes NOAA as a primary component "of the climate change alarm industry" and said it "should be broken up and downsized."
[…]
> Project 2025 would not outright end the National Weather Service. It says the agency "should focus on its data-gathering services," and "should fully commercialize its forecasting operations."
> It said that "commercialization of weather technologies should be prioritized to ensure that taxpayer dollars are invested in the most cost-efficient technologies for high quality research and weather data." Investing in commercial partners will increase competition, Project 2025 said.
* https://www.politifact.com/factchecks/2024/sep/26/jared-mosk...
Also I don't think his data supports his argument. He shows, for instance, that GFS and ECMWF track each other in his own quality metric, that modern GFS is basically the same quality as ECMWF was in 2012, and that they are slowly converging. This is hardly a "decline."
The reason why ECMWF and GFS track eachother is that they're using the same data! The dominant limit on how good weather prediction can be is the amount and quality of the initial state data. ECMWF is proof that GFS could improve still, but let's be real: it's very good and unlike ECMWF it is free.
That, I think, is the real problem this guy has. The Accuweather CEO (who is a Trump supporter) and a lot of the private meteorology industry want NOAA to get out of the business of weather forecasting so that they can charge people for lifesaving information.
They pump ECMWF mostly because it's not free. Even the data showing it is superior is often somewhat cherry picked. For instance, it missed Hurricane Milton for quite a while while GFS was predicting its development and ultimate path pretty accurately.
Oh, the keming.
But I agree. Seeing that privatization of government agencies is part of the platform, this sounds like "I'm not endorsing, but..."
But I’m not saying just what role he might find tempting. Focusing on the particular role is a red herring.
There are lots of opportunities for disruptive people with big plans who don’t mind wearing the hat to get there. As we have seen recently.
How many trump staffers read his blog, do you think? Are weather models a hot topic among them?
Lewis Fry Richardson might have something to say about that.
So there's been no decline. In fact US weather prediction is getting better (by the article own chart). It's just The author is upset the US isn't moving ahead as quickly as other weather services. Maybe they should be upset by this but the way it's framed seems a bit dishonest.
The author makes an important point, on a subject that keep getting more important due to climate change, but appeal to blind nationalism doesn't work in his favor.
It's not so much a bias a willful blindness at this point. I have to tolerate it coming from politicians, but it is a different story when the author present himself as a serious scientist.
Yes, the U.S. could be the best at meteorology. they could be the best at anything, with their means. But not at everything. The U.S. are the best at military spending, and everything else is at best in second place.
https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_...