Using Machine Learning to “Nowcast” Precipitation in High Resolution | Hacker News Reader

Using Machine Learning to “Nowcast” Precipitation in High Resolution

ai.googleblog.com

72 pointsluthfur12 comments

I chime in on this because I'm working on deep learning for precipitation nowcasting using radar for my Ph.D. I was very excited when google released the press statement at NeurIPS about their work in this area. Unfortunately, after reading the paper, I have to say that their approach is fairly basic. Basically they threshold the precipitation in 4 thresholds (no rain, light, medium, heavy) and then use a U-Net like architecture, treating it as a classification problem. I think that the works of Shi et al are much more interesting in this regard:

https://papers.nips.cc/paper/5955-convolutional-lstm-network...

http://papers.nips.cc/paper/7145-deep-learning-for-precipita...

What I think is that Google wanted to use a lighter model that can be applied to the whole continental US. I expect them to integrate this in google assistant, like: "hey google, tell me when it's going to rain"

I don't get why the put nowcast in quotes. Nowcasting is pretty common term in meteorology.

It's also cool that they could get this far without physics. Of course, "HRRR model begins to outperform our current results when the prediction horizon reaches roughly 5 to 6 hours". Simple associations that could be discovered by neural networks work in the short term, but the atmospheric physics is needed to understand the long term evolution of storms.

I think this blog article did not frame the novelty of this study very well. It is not the nowcast that is the problem, it is getting nowcast at a spatial resolution higher than weather forecast models. But in that case, the limitation is more about computation rather than physics. What their U-Net does seems to be a fancy way of doing statistical downscaling.

Needless to say, this should not be interpreted as machine learning can replace physics in weather forecast or operational meteorology (see Lorenz 1963 paper). It shines as a great data assimilation technique though.

Also, physics-based models (Numerical weather predictions NWP) assimilate much more data than just the last few radars scan. The assimilation phase takes a couple of hours, but when the physics kicks in, the NWP models win hands down.

I've been out of the field for about a decade now. So how often are they running data assimilation into models now? It was basically just 0Z and 12Z back then, because it's computationally intensive. (And of course, you only have radiosondes every 12 hours, and satellites only get you so far.) As are model runs.

Like, of course NWP models win. It's not just the physics, it's all of the other assimilated obs that are advecting over your area of interest.

But it's the problem is always always computation time, to an extent that most people on HN won't get. Maybe the finance guys. But you have to process a mountain of new data, then run the model very quickly for it to be any use at all to the public.

We should use reanalysis for nowcasting, that would be super accurate. /s

In the UK, netweather.tv shows weather radar with a 5 minute temporal resolution, 50 meter spacal resolution, and a 10 minute latency (less if you pay I think).

I regularly use it 'by eye' to predict when a big band of rain is coming. I can very effectively figure out if there will be more or less rain 5 minutes from now. "Shall I walk to the car now, or should I wait 5 mins?".

In comparison, the results of this work seem disappointing.

A physics-free approach to precipitation nowcasting using a CNN.

Sounds a lot like darksky.net they have an interesting story starting from kickstarter to getting acquired by longnow.

dup: https://news.ycombinator.com/item?id=22051889

On HN, it's not a dupe if a story hasn't had significant attention yet. See the FAQ: https://news.ycombinator.com/newsfaq.html. Allowing reposts in such cases is a way of mitigating the randomness of /newest and giving good stories multiple cracks at the bat. If you submit a lot of good stories (which you have!) it evens out in the long run.

By the way, from the IDs you can tell that this submission was earlier than yours. It made the front page later because we put it in the second-chance pool (described at https://news.ycombinator.com/item?id=11662380).

ah got it thanks !

Now cast, let me look outside

Reply on news.ycombinator.com