Advances in Weather Prediction | Hacker News Reader

Advances in Weather Prediction

science.sciencemag.org

87 pointsALee46 comments

One (rather easy?) thing I'd like to see in weather forecast apps is a confidence score.

I know that sometimes the weather is hard to predict. Right now I can't tell by looking at the weather forecast how confident they are. If the different weather models give significantly different forecasts, the confidence score should reflect it.

Just as an extra datapoint, when it comes to ocean racing, one of the biggest complaints about mesoscale weather data is that it does not have a confidence score. The weather forecast says that the north pacific high is going to shift 200 miles west, which changes your fastest course from San Diego to Hawaii from a west then southerly route, to a south then westerly route (the high pressure system sits directly in between)... if the confidence on this forecast is 95% then you know to commit to the southerly route, but if it is only 51% confidence and typically the high is 500 miles more east than this then you might make a different decision and monitor the situation as it develops.

Anyways, if anyone at the NWS is reading this, we do actually pay close attention to this stuff and a confidence score of some sort would make boat weather routing software a whole heck of a lot more accurate.

I have worked at my nation's meteorological office and they (and I'd assume others do too) provide this service. They do this by running an ensemble (at our office, a population size of 50) with different starting conditions of numerical weather prediction models, rather than a single run. This will give you a confidence in the predictions of the numerical models. Besides general forecasting, they are also using this for hazard triggers so it can basically answer questions like "In the next 72 hours is there a 50% or greater chance that wind speeds will be higher than 30 knots at the biggest airport?".

The Dutch weather forecast has an 'expert' forecast that includes confidence bands, see:

https://www.knmi.nl/nederland-nu/weer/waarschuwingen-en-verw...

The page is not very popular though

That's great, I would love it if there was something like that that covered a wider area.

Thanks, never new that service existed. They should advertise it more

I've an informal method for estimating uncertainty by looking at how the forecast for a particular day changes over time - i n the UK the BBC provides 10 day forecasts so I watch particular days and if they are consistent over time then I usually have more confidence than if the forecast for a particular day changes every day.

Mind you - I had a six hour walk in rain and snow last Saturday when the forecast was for a fairly nice day and had been for over a week :-)

The Climendo app takes an average of 3-5 forecasts and presents a confidence measure.

Not as good as proper confidence from the model ensemble, but better than what you normally get from a single forecaster.

There's a web interface too.. https://climendo.com

Take a look at the probability forecast that the Norwegian met office make available as an api - https://api.met.no

The National Weather Service office in Binghamton, NY publishes probabilistic snowfall forecasts that do a good job of breaking down the confidence scores (presumably other offices do this as well, this is just the one I'm familiar with).

There's no snowfall in the forecast at the moment, but you can see how it's broken down: https://www.weather.gov/bgm/winter

I don't like how they give you the worst possible weather for the day, for the whole area.

any biomes have a rain shower from one cloud over one neighborhood, but the whole area is now "chance of rain 100%", when the real user experience is 100% sunny/clear at all hours of the whole day

How good is weather prediction for other people? Is it actually accurate for you? Are even the current conditions accurate for you? I use Weather Underground, and a few weeks ago in the Bay, here are just a couple of the things I remember observing (out of the many wrong predictions): (1) One afternoon I was told there would be rain in 5-6 hours, then a couple hours I was told there would be none. (2) I started getting soaked while walking outside, and I checked the current conditions and was informed it is in fact not raining and there is no upcoming rain either. These were despite the facts that the weather stations were < 1 mile away for me and the conditions persisted for a fair bit (i.e. it wasn't just a random 2-minute shower).

My experience living in Dallas, Texas, which has some 1500 miles of accurate weather data as systems come across the north american continent from west to east.... usually the 10 day weather forecast was accurate to within 4 hours. This is really helpful for planning sailing trips, etc. Usually three days out the weather prediction is accurate to within 30 minutes.

As for the California SF Bay area, weather prediction is mostly useless, day to day weather prediction is wildly inaccurate. I think just on Friday there was a prediction that we were going to have a bunch of fog through Tuesday, and it was clear the next day. Wild storms will be predicted, and then never appear, and other times weather that was going to hit Oregon smashes in to the bay for three days. Many times we've gone out sailing expecting calm forecasted weather (Mother's Day 2016) and got 25mph winds and other times we went out expecting forecasted 15mph winds and it was closer to 6-8mph.

My guess is that the lack of HD weather radar in the pacific and the complex topology of the coastal areas and the bay, and also the chaotic interaction between ocean and inland weather systems make for a much more challenging forecasting system than doing forecasting of the plains states. The unique geography of the golden gate being a venturi tube between the low pressure 55F/13C pacific ocean marine layer and the high pressure 95F/35c central valley some forty miles inland creates what the local slang describes as "the wind machine".

If you are in the United States, you may be interested in the scientific forecaster discussions produced by your local NWS office. They provide more insight into forecast confidence and general regional dynamics. Theyre produced several times a day and are often quite fun to read.

Its amazing how far we have come in weather prediction. A few decades ago, 5-day forecasts did not exist and now we get 10-day forecasts that are reliable enough that we take it for granted.

In my experience, nothing beats getting the forecast straight from the horses mouth (NWS that is). Some companies are notorious for producing 30+ day forecasts, which cant have any meaningful levels of skill.

I never understood why NOAA/NWS didnt just create their own mobile app. I use Wx[0], which parses NWS data directly and can be found on f-droid.

[0] https://gitlab.com/joshua.tee/wx

Corruption, or so.

There was almost legislation to stop them from distributing data to the public directly, because then companies can make more money doing it.

Also check out IEMBot for live (immediate) access to raw NWS products [1]. It should be noted that the scientific discussions are exactly that - while quite valuable, it can take some effort to become familiar with the abbreviations and raw weather science.

1. https://weather.im/iembot/

Interesting... do you see the NWS forecasts being more accurate then Weather.com/Weather Underground? It always felt to me like the former was (basically/effectively) a low-pass-filtered version of the latter, not actually any more accurate, but I have't really compared them that carefully.

The NWS produces expert commentary (aka "forecaster's discussion") that can explain the deviations from forecasted weather. For example, there is a stalled weak cold front over the US right now. Many areas are getting variants of "Though not currently included in our forecast, very minor rain chances could return on Wednesday with models indicating some eastward movement of the ridge and an increase in PWAT values" based on the underlying uncertainty behind where exactly this frontal system will stall out.

I do a number of outdoor activities and I think forecasting has become very reliable, with the caveat that precise precipitation amounts and timing can vary over a short distance. If you are concerned about rain in the next couple of hours, take a look at the radar maps, and see if the situation is shaping up as predicted.

Yeah the amount is secondary. I don't really have a good feel for 0.01 inches vs. 0.1 inches anyway (both seem small and then I constantly get surprised). Trouble is when they literally say no chance of rain and then I get soaked, or when they say it'll rain and there's not a single drop.

I pay close attention to precipitation forecasts due to the nature of my job. In the midwest, National Weather Service forecasts up to seven days out are reliable and useful. If the NWS says there will be a rainstorm in a week, there usually is one and vice versa. The quantitative aspect of the forecasts is less accurate. It is common to see a small storm become a large one or for a scary storm to fizzle.

This is the page that I check every morning: https://www.wpc.ncep.noaa.gov/qpf/qpf2.shtml

Oh interesting! Thanks for sharing that.

Parts of the Bay Area are pretty famous for having microclimates. I wouldn’t really use traditional weather reports for determining if it’s currently raining right here, but I find them accurate enough for determining if I need to carry my rain jacket or umbrella for the day.

This wasn't a microclimate issue. I'm pretty darn sure the conditions at the station simply could not have been so radically different. And re: forecasts, if I had just looked at the weather to see if I needed an umbrella that evening I would've concluded "yes" if I had looked at 2pm and "no" if I looked at 3pm. Sometimes they do get it right, but other times they're just literally useless.

The point of having the umbrella with you for the day is so that when you walk out the door and it’s raining you can open the umbrella.

Yes? Not sure how this is a reply to the comment...

Similar experience here in South Florida, the two weather apps I am currently using not always agree on the next day weather and often are wrong on the same day weather forecast. It seems to me that there was better accuracy a couple of years ago and somehow they have degraded. I am puzzled :/

Besides microclimites, which are a whole other discussion, predictions vary per service. It looks like AccuWeather has a 2% better rating for San Francisco, it's not much but it's something. https://www.forecastadvisor.com/California/SanFrancisco/9412...

but compared to a smaller city also with microclimates on a large body of water the prediction for San Fran is 15% better - https://www.forecastadvisor.com/Minnesota/Duluth/55811/

That's the first time I've seen that site, and for my city it's kind of frustrating if it's true. The Weather Channel rates the best (by far) compared to my preferred app, but I find The Weather Channel app to be an awful user experience for how I want to see weather presented to me.

The Weather.com data is sold to Google, Facebook and many others. Perhaps you find a better user experience on those platforms?

I highly recommend Darksky[1], or one of the copycat services like the AccuWeather Minutecast. They use computer vision based approaches processing radar fused with conventional forecasting, and do quite a good job. I believe that some other "hourly" forecasts are still published at the beginning of the day, rather than continually updated.

[1] https://darksky.net/

Upper midwest here. Hourly predictions are accurate more often than not, and a few days out is chancey.

Big storms are often exaggerated, though I suspect that's because the risk of flooding, ice and other hazardous travel conditions are serious enough to be overly cautious.

We had predictions for major snowfalls a few times, and every time was completely overblown, and then one hit exactly as predicted. Definitely glad I wasn't on the road that evening.

Ahh yeah, you guys probably get a lot more extremes. :)

If you look at the models themselves you can look several days out and get an idea of what could happen.

https://mag.ncep.noaa.gov/model-guidance-model-area.php

Select GFS (Global Forecasting System, the usual model) and NAMER (North America), then Precip P06 (six-hour precipitation accumulation), and then Loop All.

Over time, you learn the behavior of the models; for example the GFS's longer-range predictions tend to overestimate peak rainfall from super heavy storms (it'll predict three inches and we'll get one).

It's not as convenient as Wunderground but it's data straight from the source.

Other occasionally useful models are the NAM (shorter range and limited to North America) and the HRRR (high time resolution but very short range).

here in italy is wildly inaccurate. medium and long term stability/instability trend holds, but day by day or hour by hour prediction are almost completely worthless.

Not widely publicized is that 5G cellular is going to eliminate one of the primary data inputs that have made weather prediction successful.

The US FCC decided, on their own, that this was not an important problem, compared (most likely) to the amount of money to be made building out 5G.

Let's stick with facts instead of fear mongering.

I dislike Ajit Pai as much as the next person and wish he had heeded NASA's request to delay the 5G role out. That said, it's not a forgone conclusion that 5G will significantly interfere with let alone eliminate the ability of radio spectrometers on weather satellites to measure water vapor using the 23.8 GHz band.

The new FCC UMFUS regulations that govern 5G require require signals to use a set of frequencies all of which are greater than 24 GHz.

That said, of course NASA is concerned about malfunctioning 5G transmitters leaking into the 23.8 GHz spectrum. Hopefully out-of-band emissions don't become a problem. It's the FCC's job to ensure that it doesn't, and the lack of carefulness so far in the process doesn't inspire confidence.

The fear is based on the already significant interference caused by consumer devices in other bands. 5G network infrastructure is likely better controlled, but doing so requires good regulation.

There is some possibility that the 5G systems could actually be used to measure some atmospheric properties, similar to the way Navstar GPS L1/L2 signals are analyzed, but since that isn't a requirement of the 5G spec/regulations, it won't be baked in.

>Not widely publicized is that 5G cellular is going to eliminate one of the primary data inputs that have made weather prediction successful.

When you say 5G, do you mean actual 5G in general or are you talking about mmWave (FR2, sometimes called 5G high band vs low band)? It's been frustrating having these things get mixed up, because the 5G standard has a lot of improvements aimed at more efficient spectrum utilization, further reduced latency, and other changes across a unified massive range of frequencies compared to previous standards. It should be quite useful therefore for existing spectrum as well, just as WiFi 6 brings improvements to 2.4 GHz utilization. T-Mobile for example has said its initial plans for 5G include using its 600 MHz spectrum, with mid-band and mmWave going to certain urban areas.

For whatever reason though most media and even tech people often incorrectly use 5G interchangeably with mmWave, and since 5G itself has plenty of changes it's not always clear what aspects someone is worried about. I assume in this case it's specific frequency blocks of mmWave that would be the interference concern, but I honestly don't know enough about the weather sources to be sure of that vs some other change to modulation causing more out of band interference or something like that.

He's talking about 5G leaking into 23.8 GHz.

Can you elaborate on this? Do you have any references?

https://news.ycombinator.com/item?id=19674218

I have to shout out for windy.com (I am not its developer sadly). It is probably the best weather site (and app) out there. You can even choose between weather models.

I often wondered the same thing myself, why don't weather providers provide their own accuracy rates? I started to make a "simple" system for estimating NOAA accuracy, and immediately ran into trouble determining what accuracy meant.

If they predict a high to be 86deg, and it's really 85, what does that mean as far as accuracy goes? If we use the Kelvin scale, even a 10 degree error makes it seem pretty accurate, though a person's experience in those extremes will be very different.

But I think the biggest problem is that the simple weather forecasts that we use on a daily basis, is a poor representation of what weather forecasters actually do. They're modeling how weather systems form, move, and interact. If a model predicts storm forming and moving a particular direction, but the 10 day forecast is off by 100 miles causing it to rain a day later, what does that mean for accuracy? Another model could just use the average weather as their forecast, and might score pretty high as far as long term accuracy, but would be pretty useless from a user's perspective.

So, if someone forecasts a high of 86 with a 99% confidence level. What would that mean. That it'll be 86 somewhere near there, that it'll be close to 86 at that location that day, or that it'll be 86 at that location within some timer period? You really can't boil all of those variables down into a single number.

And then you'll run into issues tracking the confidence of the confidence levels. Ad infinum.

NOAA hosts a Python SDK and REST API for historical data. And I believe they present at the SciPy conf every year (upcoming in July)

I found this talk by Uber's Danny Yuan super insightful. Forecasting is probably the subset of ML I am most excited about ;)

Two Effective Algorithms for Time Series Forecasting

https://www.youtube.com/watch?v=VYpAodcdFfA

It's kinda cool that we can predict in enough time to save lives and equipment. If our brains or metabolism had been slower, then this might not be the case, even after we've developed the mathematical theories that we have now.

Reply on news.ycombinator.com