A visualization of global weather conditions, forecast by supercomputers | Hacker News Reader

A visualization of global weather conditions, forecast by supercomputers

earth.nullschool.net

153 pointsmoonlighter41 comments

Time to whip out the hairy ball theorem: https://en.wikipedia.org/wiki/Hairy_ball_theorem

Yes I heard what I just said.

Does the hairy ball theorem imply that there is always some place on the earth's surface where wind speed is zero?

Yes, assuming that the wind vectors are orthogonal to the Earth (and that the Earth is spherical).

Also, by the Borsak-Ulam Theorem ( http://en.wikipedia.org/wiki/Borsuk%E2%80%93Ulam_theorem ), there are two points on the Earth directly opposite one another that have the same wind speed.

I think slight disturbances from being perfectly spherical are fine (since the Euler characteristic wouldn't be affected). The wind vectors not always being orthogonal to the Earth seems more damning, but maybe it is enough to just consider their projection onto the Earth's surface.

Is it a problem that air can change in density/that it can be compressed?

The javascript shows they are thinking about this as particle traces of wind velocities. In visualization, the technique is called Line Integral Convolution (LIC). Each line follows the direction of the wind vector, and the overall brightness is the sum of the magnitude of the winds under the line. This particular variant just uses the wind speed at one point under the line. LIC won't accidentally miss field singularities like vector-arrow representations, and there are variants that provide smooth animations.

It's great that we've advanced from needing VTK to being able to do a visualization like this as a live movie, translating GRIB files to JSON.

Also interesting: http://earth.nullschool.net/about.html

Thanks. As mesmerizing as the visual is, the explanation is useful.

I'm always curious about the underlying modules used. Hadn't heard of when.js before: https://github.com/cujojs/when

Beautiful!

If you turn the globe upside down, the controls for rotating it are backwards.

Some way to pan/zoom without pausing the visualization would be nice, but probably very difficult without moving to webgl-based rendering.

Beautiful design. This would be awesome to have hanging on a wall.

I considered having this displayed on a spare monitor I have but ultimately decided against it as the animation uses too much CPU for the rendering.

Maybe I'll port it over to C++ if I find some time.

Can it display different data by playing with the url?

e.g. I'm looking at

    http://earth.nullschool.net/#current/wind/isobaric/1000hPa/orthographic=-11.40,53.26,489

The only thing I can get to change is the lon, lat, zoom at the end. Are there more things I can change?

At 10hPa, it's windy as dicks around the arctic circle. The antarctic, however, is relatively calm. Is that a seasonal thing? Will it be the opposite in June?

Polar-night jet stream: https://en.wikipedia.org/wiki/Jet_stream#Polar_night_jet

looks broken here if i rotate it until it redraws which is a shame...

i do find this interesting coming from a native rendering background. the idea of converting data from one, fairly inefficient format (FORTRAN friendly from the look) to a spectacularly less efficient format still is a bit mind boggling without appreciation for the web stack (JSON is not for run-time in my world - its for 'tools' or 'compile' time).

the rendering is also quite underwhelming on a desktop PC - especially that it cuts out whilst rotating. there is some obvious stuff here that can be cached on inspection of the comments on github...

for instance, since there is a lot of data processing already, how about 'unprojecting' the data to remove the extra interpolation overhead from having to apply a transform and its inverse? Just because they have chosen to project their data onto a sphere doesn't mean you have to follow suit... its probably a useful format for meteorologists or cartographers but its not suited for rendering at all.

of course doing that will have similar results to a low pass filter unless you use a much higher density grid than the source (since you want a regular grid in your result and have irregular data points) - but visually that is very acceptable as a compromise.

the obvious guess suggestion is webgl and a 3d canvas, and dropping any fancy svg or other elements if they need to be visually sycned up with precision and rendering all of it yourself in a single consistent way. in the browser world its often a bad idea to rely on the implementation of anything if you need guarantees of quality - there is a lot of variation and a lot of bugs that have persisted for years on end...

i'd also suggest partitioning the data once its in 3d - a kd-tree or regular octree is quite easy to implement and understand and perfectly suited for this imo

Is this current weather conditions?

I checked it against the BBC Weather pressure map for the North Atlantic and Europe and the low and high pressures seem to match so it does look fairly "live".

It says on the about page it is updated every three hours.

http://earth.nullschool.net/about.html

EDIT: If you click on 'Earth' it brings up a menu with time controls.

This is really cool, but it is clearly heavily influenced in design by Viegas and Wattenberg's Wind Map: http://hint.fm/wind/

The creator should really acknowledge their work...

Relax; they credit it on the about page: http://earth.nullschool.net/about.html

I have heavily acknowledged their work in tweets, on fb, on github, and in the about page. I have also thanked them directly. Also happy to note that most articles about the site also acknowledge the influence from hint.fm.

Out of programmer curiosity, is there a reason you have to stop rendering the globe while its moving?

The NCEP data provides only 1º resolution, so bilinear interpolation is used to fill in the gaps. How much interpolation is needed depends on the zoom level and the projection. On top of that, the distortion caused by the projection must be applied to the interpolated wind field. All of these heavy calculations are done up front so the animation can be as fast as possible. So each time the orientation of the globe changes, we have to redo the calculations.

This makes more sense in light of the view supporting zoom, which didn't occur to me the first time I visited the page. Perhaps add a note in the corner?

"On top of that, the distortion caused by the projection must be applied to the interpolated wind field."

I think I'm missing something. Are you overlaying a drawing canvas over the globe and handling your own custom projection then? Is it not possible to dynamically draw to a webgl texture and let the gpu take care of projection?

I do understand why you'd have to restart on zoom. Overall its an excellent project you have here.

Exactly! The globe is SVG and the animation is a Canvas layered on top. However, I'm still using D3's projection logic to calculate the distortion. Yeah, could save on redoing the distortion calc if just globe rotation changes, but would still need to redo the grid interpolation. So, to keep it easy I just rerun the whole deal.

WebGL would be fun to learn, but AFAIK not supported by mobile browsers yet.

WebGL is at the least supported by the default browser on the Galaxy S II, and presumably newer models. It doesn't support floating point textures, however.

Beautiful!

How difficult it is to set this up for a custom local site that produces surface wind forecasts in GRIB2? Would it work for a small grid like this http://www.norcalsoaring.org/BLIP/BYRON/index.html ?

Your visualisation reminds me of what could be possible for a meteorologist with a setup provided by WSI:

http://www.wsi.com/products-media.htm

With their tools for meteorologists you can layer up all the data layers from the GRIB, radar, observations, pollen etc. as well as the forecast data to 'see' and explore the weather in quite astounding ways. Think of what you have here but in lots more resolution with untold extra layers of data - it is a fun way to understand the world that we should all be seeing and doing by now instead of just getting a screenful of dumb icons.

It seems that the likes of WSI are quite happy to serve the market for dumb icons rather than make their deluxe weather tools available to all on an app. Imagine if everyone could be an amateur forecaster and submit useful observation data from their phone to get fed back into the 'model'.

The meteorologists are keeping the best tools from us thinking we would not be interested, your site encourages me to think otherwise.

Wicked cool visualization!

Amazing

Honest question.

Why does weather behave different over oceans when compared to land?

No obstacles for wind, water vapor, temperature gradients of water..?

Probably because wind speeds are faster over the ocean due to less friction. Mountains, trees, structures, etc. all affect that over land.

Also a larger heat exchange.

This. Even a little terrain is a lot more dynamically interesting than the ocean.

There are also strong heat / surface moisture gradients on land (think edges of forests, cities, coasts), which play a big role in convection.

By comparison, the ocean is an infinite plane so the dynamics are much more dependent on large scale forcings like the Earth's rotation and there is less local variability.

I'd wager a guess it's the water vapor.

Great question.

I think this is a data issue, not a weather issue.

The data they are using over oceans comes from microwave scatterometers, which use radar to measure disturbances of the ocean surface caused by near-surface winds. Basically, the power of the radar return is affected by the interaction of the radar waves with the irregularities in the water surface (https://earth.esa.int/applications/data_util/SARDOCS/spacebo...). To get a wind speed and direction from the radar power, you have to hit the surface from several different angles and perform an inversion.

There are several ocean winds space missions that do this continuously from ~90-minute Earth orbits. For reference, one is QuikSCAT -- http://science.nasa.gov/missions/quikscat/

There is no analogous sensor that works over land. Hence, the data over land is not nearly as uniform, dense, and regularly observed.

All that said, this particular visualization seems to have a calibration issue, because the land winds seem to be off in magnitude. I'm not enough of an expert to know if this is a problem with their data merging, sensor selection, stratification (e.g., winds at land surface may be lower than at sea surface, but up 50m they may be comparable), or something else.

You can get other surprising things from remote sensing over oceans, including salinity, temperature, and incredibly precise elevation.

Can you estimate the density of that data in space/time? Doesn't this vary with the overflights of the satellites?

The land data is scarce in e.g. the Sahara, but the density over western Europe is pretty high. The land based data is continuous as well, and not affected by cloud cover.

I think it's a 50km spacing grid over the ocean. There are multiple satellites, so they get multiple scans per day. Also, they might ingest the data into a model, and report the model outputs, so this could enhance temporal resolution.

Another issue with in situ sensors is calibration and uniformity. Unless they were installed by a central authority, they are likely to all be different. This makes analysis harder.

Reply on news.ycombinator.com