An attempt to create the highest resolution real-time map of global temperature | Hacker News Reader

An attempt to create the highest resolution real-time map of global temperature

blog.forecast.io

123 pointsrjsamson39 comments

Celsius please. But besides that: Very nice, very clear and way better than supid sun and cloud icons with temperatures next to them over some vague areas.

Absolutely.

Only about 320 million [1] out of 7 billion people still use fahrenheit.

[1] http://en.wikipedia.org/wiki/Fahrenheit#Usage

It's less than the percentage of the world still using IE6!

Even if normalized for people that actually have Internet access? Honest question.

The IE6 share is going to be higher when normalized for people with Internet access, so the comparison is going to hold by a wider margin.

Doh. You're right. Not sure what I was think at that moment :)

I am proud of forecast.io for sticking with a 'normal' color palette here, rather than the non-standard purples they have chosen for radar; as a meteorologist the purple radar hurts my brain, but this map is completely cool.

Also there are definitely some quality control issues with the RTMA data, so you can bump into the garbage-in-garbage-out issue with it, but overall, this is a very nice start to something that could ultimately be quite useful.

Thank MapBox. If it were up to me, I'd do something funky like a blackbody-esque color palette and then get yelled at because it'd be hard to read. :-)

Also: Validation is tricky, since we can't just compare the output to ground station observations, as we incorporate ground station data into the model. Eventually I want to generate alternative versions that randomly exclude specific stations so we can use them for comparison.

I think RTMA already includes ground station measurements, so analyzing performance using a leave-N-out strategy wouldn't be a good verification:

http://nomads.ncep.noaa.gov/txt_descriptions/RTMA_doc.shtml http://eamcweb4.usfs.msu.edu/mm5-case/RAWS/RTMA%20papers/pon...

Instead, I think you'd need to find temperature measurements that are completely independent and use them for verification. Along this line, I'm not sure how refitting the data to ground stations would produce a better match anywhere except at those ground stations (overfitting). Or are you using ground stations that are truly independent?

When we compare it to RTMA, we leave out RTMA from the list of data sources. Likewise, eventually I'd like to do the same with a subset of the ground stations we use.

(The problem with finding completely independent measurements is that we'd want to use them as an input!)

This is pretty impressive, i wonder what the update interval and source is.

http://www.mapbox.com/labs/forecast/temperature/ I really like the interface, even the zoom buttons could be removed or hidden as you can scrool-zoom.

Make the temperature in Celsius please!

EDIT: There seem to be a problem with the algorithms, the largest zoom displays colder temperatures( look at the temperature map ).

Largest zoom,

http://i.imgur.com/GVt3iSy.jpg

Normal zoom, picture was resized by me

http://i.imgur.com/t1GgrOM.jpg

I was also impressed but concur that there seems to be significant loss when zoomed out. I'm not sure how they sample for the larger scales, but it seems very inaccurate.

The color palette adjusts based on the zoom level in order to improve contrast. Roughly a bazillion people pointed out that it's "broken", so I guess that was a dumb call on my part!

The legend doesn't update properly, so it really is broken.

Shouldn't it adjust based on the extremes (Or standard deviation) within the zoom area? In that case there would always be some purple, and should fix at least the specific problem the parent showed.

This isn't a map. This is a simulation. You made a simulation, and are outputting the results.

This teaches you nothing whatsoever except that your model has pretty colors.

Next thing someone is going to take these results, use them as input data for a new model, then send the results of that new model back as data for the first.

This isn't a map.

What would a better map be? Is your point that they are doing interpolation on something that is already interpolated? Or are you implying that there is no way to create a map of temperature using only point measurements? I would like to know how well this data matches the raw station measurements (and verification measurements) but I think it's a decent visualization of likely real time temperature across a region.

> Is your point that they are doing interpolation on something that is already interpolated?

Yes.

> but I think it's a decent visualization of likely real time temperature across a region.

Maybe I'm being pedantic, but to me this is a 2D visualization of a simulation. But it is not a map.

As a cartographer, this absolutely is a map. A map showing a simulation. "Map" just means there is a spatial component to the visualization.

Hmm, there are some superlatives ("highest resolution") in here I'm not sure about. Lots of groups create maps like this for modeling purposes. One for sea surface temperature is:

http://ourocean.jpl.nasa.gov/SST/

This is a blended product (i.e., multi-instrument, and gaps filled) with 1km resolution. There is also a 1km MODIS land surface temperature data product:

http://modis.gsfc.nasa.gov/data/dataprod/dataproducts.php?MO...

Neither of these is real-time (more like daily).

"Real-time" is the key. As far as we know, there isn't another real-time global data product that is this high resolution.

And while we actually use MODIS data as an input to our temperature correction model, it is, as you mentioned, land surface temperature, whereas our map represents near-surface air temperature (i.e., what you'd get in a normal weather report).

Good point about "near-surface air temperature".

Just for the fun of it, for those of you in the bay area, here's a surface temperature forecast for today, 16:00 PST. It is from the forecast site that I've been maintaining for some time now - http://www.norcalsoaring.org/BLIP/BYRON/index.html

http://raspbucket.s3-website-us-west-2.amazonaws.com/BYRON/F...

(Based on 3km/750m WRF model run two times a day based on the NCEP data. )

I wonder if they are aware of the super high resolution weather satellites that NOAA is putting up into orbit and will come on line, I believe, end of this year. I was talking to a high level NOAA official on the technical side and he was saying that it will essentially provide a remarkable, i.e., revolutionary increase in prediction accuracy and be able to provide on the ground climate level predictions.

I believe most of what they do is aggregation, including of NOAA's data, so this may just improve forecast.io.

Nice work!

There seems to be a typo in the Stats (my emphasis):

>Pixels: 16-bit unsigned ints, representing "deci-kelvin" (i.e., divide by 100 to get the temperature in Kelvin).

It should either be centikelvin or divide by 10.

On another note: It would be cool if you could hover over a certain area to see its temperature.

Oops, it should read centikelvin. Fixed! Thanks.

Please use Celsius or at least show both.

I don't get where the data is coming from. A lot of it must be interpolated.

Amazing stuff! It's things like this that makes me proud to be a developer.

This leaves me wondering, how does one go about designing such a system? Are you planning on doing a technical write up soon?

I didn't realize how hot eastern china was!

Most of China's 3million+ cities are on the east coast, the west is relative unpopulated. I was there in November and while the results of this map shouldn't have surprised me, considering the extreme city expansions I saw, I was still very surprised to see that half the country is at the same level as Saudi Arabia!

In lowest zoom level, if you scroll the map, it snaps back to the original center after scrolling fades out.

It's a cool project, but...

>>It regenerates every hour, providing a constantly updating snapshot of air temperature around the globe.

Once per hour is not "real-time." It's "once per hour."

Having done a lot of real-time programming myself, it's really in the eye of the beholder. And there are snobs who think that nothing less than several kilohertz is real-time.

But historically, "real-time" means "not batch mode" - where batch jobs are executed whenever there are available resources, in an unpredictable manner.

Real-time is responding to real world input on some periodic schedule that is appropriate. In this case, one hour sounds fine for a world temperature map. Are you expecting to make minute-to-minute decisions based on the global temperature distribution?

Local temperature I'd expect to be updated faster, but really it's a matter of your use-case.

There are plenty of existing "hourly" climate data products (http://www.erh.noaa.gov/pbz/hourlyclimate.htm and http://rda.ucar.edu/datasets/ds093.1/ are two examples.) Calling this one "real-time" is needlessly confusing, because "real-time" sounds like it should somehow be better than "hourly," but it's not.

Real time means that a calculation ends before the next update. Real time doesn't say anything about rate but means there is a constant rate.

very cool

Reply on news.ycombinator.com