Hacker News Reader

Comment by DanielBMarkham | Hacker News Reader

Okay, I'll bite. Is the raw data available -- with no extrapolation or filling. Just a big series of: station, long/latt, date, max-temp, min-temp? Data that no human hand has touched since the original reading?

That would be a wonderful CSV file to play around with. It couldn't be that big of a file either.

View on HN

Looked at the paper linked in TFA [1]. Said something about "HadCRUT" temperature data, so I Googled it. Third link down was [2], which has data downloads. Also checked wikipedia, which linked to [3], which has that and many more datasets.

[1] http://hadobs.metoffice.com/crutem3/HadCRUT3_accepted.pdf

[2] http://www.cru.uea.ac.uk/cru/data/temperature/

[3] http://hadobs.metoffice.com/

Alright I've clicked on all those links, read the text, and downloaded the files.

I've got files for variations, I've got files for averages, I've got files I don't know what they are.

Now back to my question: anybody know where the unaltered data, by day and location, is? Direct link would be awesome.

This isn't too much to ask, right? I mean, if we've been keeping temperature records, surely there has to be the raw data somewhere in an easy-to-consume format? (I'm not trying to be cynical or sarcastic. For all I know there might be good reasons for such data not existing or I might have a case of the doofus here)

There was a thread on HN some time ago and I saved this link, which has some instructions on where to find the data and a description of the format.

ftp://ftp.ncdc.noaa.gov/pub/data/gsod/readme.txt

I wanted to start a little project to process it but never got around to it.

Hope this helps.

I've seen claims somewhere (maybe at "Watts Up with That") that no unmodified climate station data exists, that it's all been modified or corrected to adjust for something.

Is this data raw?

Well, it does exist, you just have to go back to the original sources, e.g. the meteorological organizations of the various countries, militaries, etc.

As far as I know, the position of the CRU is that they have no original unmodified raw data anymore. Don't know about NASA and NOAA.

This article claims that 3 out of the 4 datasets available, the ones from the CRU, NASA’s Goddard Institute for Space Studies (GISS) and National Climate Data Center Global Historical Climate Network (NCDC GHCN), are not independent and that the latter two were felt by NASA to be inferior to the CRU's. The article doesn't have anything on the raw data, however.

http://pajamasmedia.com/blog/climategate-stunner-nasa-heads-...

The NOAA data is exactly what I was looking for.

Of course, it's only the U.S., and it's only for the past 80 years or so. But the dataset looks clean.

I'm no climatologist. I don't know a lot about how temperature is measured. But your question is based on several assumptions that I think need analysing.

1) What temperatures are we talking about? Surface air? Water? 2) Who's collected it? There's probably dozens of organizations that collect it. I doubt there's one single repository for it all. 3) How was it collected? Are the data comparable? Especially historically. You can't simply ask the weatherman what the temperature was in Antarctica in 1874. 4) Even if you had universally comparable data for the past 150 years, you still have trouble. Temps in urban areas for example skew results because of heat trapping. Those factors need to be accounted for.

Science is complicated for the very simple reason that the natural world is complicated. There's no such thing as "pure" data.

I trust science industry (for the most part) over time. The over time part is key. There are generational checks and balance in science. Young turks looking to make a name are attacking the holes in theories all the time. If something is faulty, we'll find out...from other scientists.

I don't need to trust individual scientists, just the process of science. Which I do.

Can we have part #17 and part #73 of the conversation some other time, please?

I just wanted to know where the measurement data was. That's it.

Reply on news.ycombinator.com