Modern Weather Forecasts Are Stunningly Accurate
theatlantic.com
theatlantic.com
Dont use your built in phone app for forecasts. Dont expect hour by hour precision (maybe for the next few hours sure, but not 2,3,4,5 days out). Trust nobody expect the trained professionals at NWS. These people look at ALL the models, they watch things like a hawk. I work with some of these nerds and it's crazy. They are like 'oh jimbob did you see the latest GFS kicking out the low up into canada? oh yeah doug that is crazy, but the euro says blah blah and NAM says blahblah... Their job is to literally make a forecast, for the same area, every day. Some people do it for years and years! Think of the experience, wisdom, little tricks and tips etc. Just go to weather.gov, look at it for 2 minutes. Read the foreacst discussion if you want to get your details.
I wonder if it does better in some geographies than others (e.g. high pressure system areas where things are more stable)
But they do a much better job of notifying about changes based on where I physically am, instead of for the city / region as a whole. Other weather apps I've used don't differentiate between "rain falling now 50 miles away" and "rain falling on my head", even though the radar map clearly gives them that info at a much finer level of detail.
Now my source is directly from NWS page, which is really nice since forecaster terms can be clicked to bring up a definition. https://forecast.weather.gov/product.php?site=LSX&issuedby=L...
https://www.metcheck.com/WEATHER/ghx_charts.asp
https://www.metcheck.com/WEATHER/discussion.asp?DiscussionID...
It has much more detail which helps as a cyclist, there is more than just average wind speed and temp.
It's especially nice for upcoming storms because they break down the region logically based on the relevant features (the river valleys and mountains), for instance: https://hudsonvalleyweather.com/wp-content/uploads/2019/01/1...
A summary forecast can only tell you so much, especially in changeable conditions and complex microclimates. Knowing the range of probable conditions is often much more useful than knowing the most probable condition.
You can make a version for your location here: https://bitbucket.org/subraizada3/weather-generator/src - it should just generate a single HTML file which you can host anywhere.
An older version also showed the 'normal' weather.gov forecast embedded to the left of the hourly forecast, you can get that if you copy/paste the deleted part of this commit back into the python file: https://bitbucket.org/subraizada3/weather-generator/commits/...
This is textbook availability bias. You overestimate the failure rate because the failures were memorable, especially the major failures. But the far more frequent accurate predictions are not recalled, because they don't leave as much of an emotional impression. For the same reason folks regularly overestimate crime rates, deaths from terrorism or rare diseases.
Meteorology has an entire subfield devoted to studying the quality of forecasts[0]. Before you bandy about numbers and anecdotes, stop and tell me what measure you are using. Is it percentage of degrees centigrade? Well you can't, degrees centigrade is an interval measure, not a ratio measure. If you use Kelvins, which has a mathematically meaningful zero, the percentage accuracy is suddenly very good. How do you count near misses of very intense weather? Direct hits by systems that were less intense than expected? How do you account for systems with very gradual gradients over very wide areas? How do you account for being early by an hour or late by an hour, but nailing the storm surge? How do you score for confidence? What's your rule for weighting false alarms? How important is mean error vs absolute error vs variance, and why?
Solve these and dozens of other problems of describing "accuracy" and weather forecasters might take you seriously. But until then it's probably worth accepting that they are the most effective profession of their kind and that we have a lot we can learn from them.
You really start to notice how wrong they are when you spend every day outside. I don't notice any more working inside again. But when you rely on weather forecasts everyday to figure out how miserable you're going to be in a day...you really notice..they might as well just be making things up half the time.
It depends a lot on what part of the world you are in - I think that the uk is particularly hard to forecast.
We can't predict exactly when it will rain, but we can get daily trends pretty good for 3-5 days out. This means you are five days ahead of the weather. You cant know "it will rain from 5:00am to 9:30am on february 4th, 2019". But you can know if we have a synoptic scale system that is moving through your area on that day, and know if there is a probability of rain.
Where do you work everyday outside? i would be happy to give you a 5 - 10 day forecast right now actually and can tell you it will be good. I am supposed to be writing some tests and documentation but i can drop some WX knowledge if you need.
As with many things, having a little understanding of the big picture can help you get more out of weather forecasts.
After you have the synoptic scale picture, then you zoom in on your target area and get specific if you want to. This local expertise is where you NWS office comes into play. You want the local forecast from these guys!
And, Americans only: if you forecast temperatures of 99°F and the observed temperature is 97°, good job. But if the observed temperature is 100°, then you made a huge mistake. People need to know when you hit the century mark!
here in the uk, I am often interested in whether or not it will rain in e.g. a 3 hour, 6 hour or 12 hour period. Either because I am going rock climbing, laying concrete, spraying weedkiller, painting outdoors, etc etc. Often this is very important (not just - 'oh, if only I took an umbrella today').
I have a strong impression that the ability of the uk's met office to forecast this is very poor, such that is only just worth consulting their forecasts. OK, sometimes we have settled weather and forecasting is easy (in these cases, looking towards the prevailing wind is also good). But in most cases the weather changes several times a day and the forecasts are poor. This can also be seen by e.g. checking how the forecast for a specific day changes as the day gets nearer - every day you look, it tends to have changed markedly. Also, the uk met office and norwegian met office (significantly better than the uk one, but still not so good) generally disagree with each other.
I really cannot square my 10+ years of 'studying' weather forecasts and the weather with what you are saying (in the case of the uk (pennine hills of northern England)). And 'stunningly accurate' is just a bad joke.
I presume that precipitation is somehow hard to model, compared to windspeed/pressure/temperature (which seem to be forecast more accurately, though I care about them much less)?
I know that here in the midwest, we never have any days where the forecast at 5am says 'all sunny and zero chance of rain', and then randomly a huge thunderstorm pops up. The atmopshere just doesnt work that way. People get upset about probabilities of rainfall and all that. Just think of the model perspective. The model sees square grid cells that are many KM apart. If the storm shifts from one grid cell to the next one, not that big of a deal in the grand scheme of things. However if you live in that grid cell, your forecast just changed from getting dumped on to clear skies (or vice versa).
You can also check accuracy for your local zip code (in US) at: https://www.forecastadvisor.com/
tldr: for the best US consumer weather forecasters the mean error on high/low temp for 1-5 day forecasts is around 3F and they hit within 3F within 70% of the time.
7-day has declining accuracy and 10-day is not much better than statistical analysis on historical trends.
The fact is, predictions will keep getting gradually better, more high res. But the fact our atmosphere & earth system is insanely complex. The system is chaotic so tiny small changes in each model run produce different results... We have code to represent some basic physics but still dont understand many of the important things. We still dont know exactly the rain process & snow formation process. We are trying to pinpoint the exact moment when dust & water vapor magically form into a cloud droplet so we can better describe the rain process. THEN we have to translate all of this insane math and physics into code.
Yeah, the extent to which models are verified against their own data is pretty interesting -- even the reanalysis data only represents a best guess of a place with physics similar to Earth. It includes a lot of advected observations, thousands of miles from the original observation site, but still useful because at least you know something about the air over the oceans.
Even if we really do get a good handle on cloud physics, ultimately there are limits on how much we can model. Parameterizations have to happen, because of limits to computer time -- a 24h forecast completed in 26h is worthless. I'm honestly surprised that they've been pushing to get models to third order and higher approximations, because the errors in your initial conditions are larger, but apparently every little bit helps.
But forecasts are pretty good considering that weather modeling is a initial value problem where we only know the initial conditions at a few observation points of varying quality, a boundary value problem where the boundaries can have complex interactions with the internal state, and simulation run time really matters.
For example, what is a 10% chance of rain at 10? That could mean 10% chance that rain will occur at some point from 10 to 11, 10% chance that it will be raining at a randomly selected point between 10 and 11, or 10% chance that it will continuously rain from 10 to 11. Also, it could be any of those three, except from 9:30 to 10:30. Also, it could be a 10% chance of raining at exactly 10.
So right there, I see 7 possible interpretations.
Add in a map, and we can apply those to an area. It may be a 10% chance of raining at some place within this region, but any one place gets a smaller chance. It may be a 10% chance everywhere in the region, but a 99% chance that some part of the region gets rain.
Sometimes percent means forecast confidence, sometimes it means that they can't predict what area will get hit with rain. Like synoptic conditions will say "this region will definitely get a few thunderstorms", but when you drop down to mesoscale then you can't tell which city will actually get the rain.
In longer-term forecasts, 10% usually means "I have no idea what's going to happen, so I guess precip is possible then." At mid range, like 3-5 days, it usually means "we haven't nailed down the timing of this thing yet".
If you actually want to know what the logic is, go read the forecast discussions.
We don't care if weather forecasters take us seriously. The weather forecasters should care whether we take them seriously.
For example, just the other day in NY, we had a forecast of light rain. We got a snow squall instead. That sure was fun.
People aren't complaining about a forecast of 80 degrees but it was 81 instead. It's those forecast that call for light rain and you get a nice fun snow squall.
You can start to imagine the jet stream whipping like a hose and understanding that the exact rate of its swing might vary, causing weather fronts to sweep across the plains a little earlier or a little later, or steering the atmospheric river a little north or south as it sprays the coast. This is the mechanism behind those sudden wild changes in Chicago, when you switch between northern to southern weather in a single day.
And in the short-term data, you can watch a radar animation and get pretty good at predicting when a rain wall or big thunder cell is going to cross your location, and what to expect on a quick errand.
The only thing you cannot reason your way through is when the maps indicate large areas of instability and you really just have to set your mental threat condition and wait to see whether something erupts over your head or not. Afternoon thunderstorms in the mountains or coasts often have this uncertainty to them.
A considerable amount of data in models for California is from observations that were made in China and Japan, that flow with the wind across the Pacific. Or conversely, you can look at it as a data hole that advected across the ocean, too.
I'd like to note that of the few blown forecasts we get in Southern California, almost none of them have to deal with the jet stream. It's actually the cut-off lows offshore that don't have steering winds, which makes it difficult to predict where it will go next. The same is true with hurricanes.
One has to appreciate the complexity of modeling such a large and dynamic system that the atmosphere is! There are cases where forecasting is more difficult. Certainly getting the temperature correct within a degree or two is one case.
Another issue is geography. I'm in middle of of USA and there is significant amount of data collected over the continental US as systems approach me. For those on the west coast I can see where forecasting is more challenging since there is far less data available to input into models (current atmosphere state 100s of kilometers out over the ocean). I'd suspect some European countries experience same situation, those further inland benefit from increased modelling data.
Reading the comments here I'm a bit surprised at how many quibble over slight details. The improvements over the past decades in forecasting models and the supercomputers that crunch the data has been significant. In most cases, like recent cold temperatures and snow fall amounts, models converge on a good solution as the event nears. This is consistent with idea that better data input yields better data out.
Weather and climate models are gradually converging and both are getting incredibly good.
They work even in astronomy. You take a climate model, set the parameters for Mars or some exoplanets and what you get is relatively good Mars climate model or good principled guess of of what the climate in tidal locked planet around ultra-cool red dwarf star is (TRAPPIST-1).
That's every single adult in the United States checking the weather four times a day. A bit more than I would have expected…
[NB I even check multiple sources of weather information: BBC, Windy and MWIS].
Goes back to shopping for that NASA spacesuit...clean air wherever you go.
I do follow ECMWF and ICON and a bit of OpenWeather forecast and the differences in forecasted temperatures are often 2-3 degrees C. And everything with a horizon over 72 hours is just numerology. (OpenWeather showed -19C in early days of Feb just 2 days ago, today it is -2C).
Now a model is just a model (I had some experience with industrial and financial models) so they are good at this and not so good at that. And they are recalculated and self-adjusted over time. I can understand.
But from the user perspective there is huge difference between +1C tomorrow and -3C.
Windy app is using 4 models and has nice graphical comparision of forecasts.
* - What I mean here is that meteorologists have good understanding of processes but the numbers that we are getting could be as well random.
I don't know if it's because current climate change is throwing off models but short term forecast accuracy is terrible and I'm recently acutely aware of that since I switched to public transport for my commute.
Really? I'd read those both as "near freezing, going to need a decent coat".
-3 means relax.
The numbers you are getting are not random. Sure the models are chaotic and non-linear, but still it's giving you hints at potential patterns, especially at long range time scales. People need to understand that wx forecasts, especially 3+ days out, should be read as guidelines for what is LIKELY to happen. Say that GFS predicts two inches of rain for your city over the next week as a wave comes through. What if the next model run shifts that blob of rainfall 25miles north? In the model terms, global terms, 25miles is nothing. But for you that is the difference between dry & soaked , and you think the model sucks.
[0] https://www.foreca.fi/Canada/Vancouver/15vrk
Disclaimer: I used to worked for MSN Weather, where we used Foreca's feed as the source for our weather data.
The weather people were the nicest. They could talk to almost anyone about their work!
I remember somebody in csiro talking to me about models getting accurate at the 1km square granular scale for a day. I think we're well beyond that for both cell size and duration now.
Everyone loves to talk about the weather and Bob Dylan was wrong about not needing a weatherman to know which way the wind blows
This small snippet of your comment in combination with knowledge about click/ad-based revenue for most outlets is really all you need to make yourself aware of nowadays.
Drama is what drove gossip for millennia and with the advent of media it began to drive sales. Doesn't matter if book, TV show or news article. It's all entertainment.
Actually, now that I think about it, perhaps if the history of vaccines and the success had been more widely disseminated, then we may not have had this anti vaccine problem. I agree that it would be nice to read more about successes.
"Stunningly Accurate" means that 7-day forecasts are now at the lower boundary of being considered "useful" so the bar is not being set very high here.
Still not a lot to quell the skepticism that some reasonable people have about the ability of scientists to accurately predict weather decades in the future.
Yes, because the goals of the forecast are different. A weather forecast seeks to predict a future position within the phase space of the system; a climate forecast seeks to predict the overall shape of that phase space.
"It will be 1C warmer in February on average" is a useless prediction if I'm deciding whether or not to wear a heavy coat tomorrow, since day-by-day variability swamps that average. But it is a very useful prediction if I'm designing infrastructure that needs to last 50 years.
It's a common misconception, climatology doesn't predict weather, but climate. And predicting average of weather is easier than predicting specific weather.
It's the same as in a casino. They cannot tell what will be the next throw of dice (weather). But they can calculate (predict) that you will, in the long-term average, lose (climate).
I use the android app and it gives me a simple comparison of 4-5 weather forecasts, an "average" forecast, and a "certainty" (a measure of consensus).
Obviously the average of the ensemble averages isn't as good as having the underlying ensembles themselves, but it's nice to see days when the forecasts agree and days when they have no idea!
Notes:
- Only the GFS27KM is reliable on free version and within next 48-72h.
- Temperatures are always conservative on extremes. When it says 37C, expect 40C. When 2C expect 0C.
I use the forecast put out by weather.gov that's supposedly tailored Of course, when they say "chance of precipitation is 80%, less than an
inch possible" and it doesn't rain, the forecast is semantically
"correct."
Like the El Nino impact on the SE US, where they forecast a 50% chance Maybe this is the sort of obfuscatory probabilistic forecast Mr. MeyerThat's like saying horoscopes and fortune-tellers are stunningly accurate: "a positive opportunity will present itself to you today - you only have to open the door" "you are holding a secret pain that is preventing you from moving on. Find peace and let go"
Where are these figures from? Your emotions, or something scientifically rigid? If the former, having this discussion is meaningless.
How would you determine, in a scientifically rigid manner, the limits on conditions which would validate that forecast as "right?" Or the inverse. What conditions would invalidate it as "not right?"
Unfortunately, this is an example of false precision. The highest resolution numerical forecasts run by NOAA have a grid of about 3km, already coarser than your one-square-mile "tailored" output. The effective resolution of numerical weather models is also 2-3 times coarser than their grid spacing (because of numerical diffusivity and similar effects).
What you're seeing isn't a "tailored" output, but instead an interpolated result from a coarser grid.
Forecasting of very high-resolution effects is the subject of active and ongoing research, but unfortunately popular meteorology does not do a good job of discussing current limitations.
Look at the confusion in this set of comments, for example, about what degree of forecast error is normal/acceptable.
That's great for you, but it means that your forecast (of [my elevation, my coordinates]) is not much more accurate than one for ([my elevation, my coordinates plus a few hundred yards]).
More technically: "surface" isn't a smooth variable when elevation changes quickly, but interpolation like that performed by weather.gov necessarily works on smooth fields. Applying post-facto elevation is great and worthwhile, but it doesn't improve the accuracy in a technical sense.
Sorry but that's stunningly inaccurate.
Also, the fact that their predictions were all rounded to the nearest 5cm should give you a clue at the expected accuracy and granularity of the final prediction. Missing the target by only one 'unit' of measure is pretty decent.