Climategate | Hacker News Reader

Climategate

blogs.telegraph.co.uk

27 pointsRichClaxton27 comments

The real scandal is more subtle, and much more serious.

There is evidence from some of the leaked e-mails and source code that the main climate model is, in the words of an unnamed developer called Ian, useless:

I am seriously worried that our flagship gridded data product is produced by Delaunay triangulation - apparently linear as well. As far as I can see, this renders the station counts totally meaningless. It also means that we cannot say exactly how the gridded data is arrived at from a statistical perspective - since we're using an off-the-shelf product that isn't documented sufficiently to say that. Why this wasn't coded up in Fortran I don't know - time pressures perhaps? Was too much effort expended on homogenisation, that there wasn't enough time to write a gridding procedure? Of course, it's too late for me to fix it too. Meh.[1]

This is the climate model that is relied upon for public policy decisions in governments across the world. This is the model that will be used as the main arguing point for global warming in next months meeting of world leaders in Copenhagen to address the issue. This is basically the model that is responsible for political decisions that will relocate billions of dollars across the world.

And at least one of the programmers thinks it's rubbish and doesn't work.

[1] http://www.cbsnews.com/blogs/2009/11/24/taking_liberties/ent...

It's interesting that this information isn't offered on articles describing emissions caps that are being agreed to in advance of these upcoming talks on climate change. I'd imagine that from now on other climate research departments will keep a tighter lid on their internal communications at any rate.
I never really thought I'd miss Bush but at least he was smart enough to not take the global warming stuff seriously to begin with. Even during the worst depression since the 30s Obama still wants more regulations. This is just amazing.

We're not in a depression, we're in a recession.. and it was the lack of regulation that caused it not abundance.

it was the lack of regulation that caused it
That's a debatable point. The Fed's interest rate reductions in early 2000s have caused recent credit/real estate bubble. If you listen to the Austrian School economists you would hear that without regulation (central banks) the market would set the price of money (interest rates) more efficiently. We would have more frequent booms and busts but of much much lesser magnitude.
So it was politically motivated regulation that caused this recession/depression.

It wasn't a housing bubble that caused the GFC, if that was all it was it would have been contained to real estate. What brought down so many large institutions was the toxic debt and dodgy dealings they had accumulated, this was allowed to happen due to deregulation. Particularly, the introduction of the gramm-leach-billey act deregulating banking, insurance and securities into one finances industry... and very specifically the exemption of swap agreements from SEC regulation (ie toxic debt). At least that's my opinion and the opinion of some very respected economists. http://en.wikipedia.org/wiki/Gramm-Leach-Bliley_Act#Criticis...
Further evidence is the current calling, from all quarters, for tighter regulation of the finances industry to avoid another GFC not less.

Artificially low rates would force anyone to seek yields at any costs, so portfolios were stuffed with all sort of junk, so your 'contained to real estate' point is not correct.
I would also argue that banks accumulated toxic assets because theey believed that they would be bailed out (moral hazard argument), which is the direct consequence of regulation.
'Too big to fail' idea has been around for quite long, one can start from LTCM more than 10 years ago, and it of course existed before that.
Fraudulent behavour is common to free market economies, but normally market participants develop mechanisms to filter out scammers (for instance J P Morgan used to say that nobody could become client of his bank without an introduction). It is when the government takes over controlling responsibility and then fails to deliver (Madoff) we have massive problems.

Extremely debatable. Better regulation would have prevented the housing crash but the regulation Americans got (with bipartsian support!) had the effect of making borrowing to buy a house ever easier, thus inflating prices, leading to a crash when it became obvious there were no greater fools left.

Which specific pieces of regulation made borrowing easier? An interesting wiki on the subject is the GFC timeline- http://en.wikipedia.org/wiki/Subprime_crisis_impact_timeline
There's mention there of a political goal ... "of increasing minority home owners by at least 5.5 million by 2010 through billions of dollars in tax credits" ... which is not the same as regulation.
The timeline does attribute removal of regulation for the financial sector .. and also the exclusion from regulation for credit default swaps, which is pretty much the reason Bear Sterns, AIG, Lehman Bros et all went under. They stuffed themselves to the gills with toxic debt, which is fine while everything is going swimmingly, but when the housing bubble bursts it becomes more than just a burst bubble... which is all it would have been if the finance sector had been properly regulated.

The Community Reinvestment Act encouraged banks to make risky loans and if banks didn't make such loans the government would start snooping around in their business. The understanding has always been that the government would bail out massive losses in these loans. Some of the deregulation made things easier for this to happen but the recession would have been regardless of these factors. Job growth over the last ten years has been almost entirely focused in housing and related industries.

An act which may encourage banks to lend is not regulation. The belief that gov't would bail out the big players does not indicate over regulation, or even appropriate regulation. Quite simply it was the regulation exemptions that were provided which made this thing far worse than it could have been.. and far more widespread. The idea that large financial institutions would conduct themselves responsibly without regulation, and not give way to greed, is complete rubbish and has been proven so, even the most conservative on capitol hill admit that these days.

This juvenile practise of downvoting simply because one disagrees does not change the fact that legislation != regulation, in fact one can legislate to remove regulation. And btw, the Community Reinvestment Act was introduced in 1977, I doubt it played a major influence in the GFC of 2008. In 2000 regulation was removed for credit default swaps, Lehman, AIG, Bear-Sterns,FM & FM and others all went under due to overload of toxic credit default swaps. Proper regulation could have avoided that... now we have the undesired result of Government owning some of these failed creatures.. which is even worse than government regulations.

Yeah, it wasn't lack of regulation, exactly. It was corruption.

If you count underemployed, discouraged workers and the U-2 figure then we're at about 18% unemployment right now. If we're still in this situation after ten years despite all the artificial stimulus I think it's just a joke to call it anything less than a depression. I'd argue we've been in a depression since the tech bubble burst and that we've just been papering over the mess for the last ten years. Considering that job growth has been basically zero for that period I'd say we've been kidding ourselves with statistics for awhile now.

Er, no, he doesn't say that the main climate model is useless, or that "it's rubbish and doesn't work". He doesn't say anything even slightly like that. The particular thing he's saying is "meaningless" is a figure that has basically nothing to do with actual climate simulation.
Clearly he's not impressed with the quality of the code. Fair enough. There's a lot of very poor code around doing important things. It looks to me as if after all Ian's cleanups, he ends up with code that produces essentially the same as the pre-cleaned-up code, which suggests that the code-quality issues he found don't invalidate the actual calculations.
Another thing that's notable: here we have a lengthy, ill-tempered diary of someone's attempt to wrestle with this big hairy codebase and the big hairy pile of ill-formed data (supplied from the outside world; this isn't all CRU's fault) that it processes, and in all his notes of what he's doing and his interactions with colleagues we don't see anything like "Ran the code again. It's weird -- it doesn't predict any substantial warming at all" or "Joe told me to change X and Y to make the figures look scarier" or "Wait, this bit here is obviously fudged". This despite a clear willingness to moan about his colleagues' work.
In other words, this seems like it would be exactly the sort of place where you'd see evidence of conspiracy and dishonesty if there were any, and there is no such evidence there.

No, you are wrong. Read thee whole quote here http://www.cbsnews.com/blogs/2009/11/24/taking_liberties/ent... and observe that the programmer is convinced the whole database of reporting stations is bogus, and the model has been tweaked multiple times: "Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog."
also "One thing that's unsettling is that many of the assigned WMo codes for Canadian stations do not return any hits with a web search. Usually the country's met office, or at least the Weather Underground, show up – but for these stations, nothing at all. Makes me wonder if these are long-discontinued, or were even invented somewhere other than Canada!"
So much for 'scientific model supported by hard data'.

I read not only the whole quote but the whole (very long) file from which the quote was cherrypicked.
Of course the model has been tweaked several times. What the hell else do you think scientists do when they have a simulation of a large complicated system and there's a constant influx of data for seeing how well it works?
The fact that they have to work with messy data is hardly their fault.

The fact that a sum of squares yields negative numbers is their fault. The fact that they can't reproduce their own numbers is also their fault.

My point was that it seems like the model might be flawed, or at least is poorly written and/or documented. As you say, this Ian guy is clearly not impressed with the code. This qoute describes it rather well:
Ulp! I am seriously close to giving up, again. The history of this is so complex that I can't get far enough into it before by head hurts and I have to stop. Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog. I could be throwing away all kinds of corrections - to lat/lons, to WMOs (yes!), and more. So what the hell can I do about all these duplicate stations?...
The problem I see is not something as dire as a conspiracy, but the fact that this is the foundation of the western worlds policy on global warming and climate change. I think we all know how hard a model like that must be to write and maintain, and it seems like there's a good chance it doesn't give reliable results.
I would say that it is a huge problem that multibillion dollar decisions are based on a model that may or may not be accurate, and that maybe we should start the discussion on climate change by putting more money into actually finding out where we stand by creating good peer-reviewed models with open data. As it stands now it seems like we might be flying blind.
Sorry if I didn't make myself clear.

Whatever makes you think there is only one global climate model?

Er, no, he doesn't say that the main climate model is useless, or that "it's rubbish and doesn't work". He doesn't say anything even slightly like that.
It doesn't make any difference what judgements he pronounces. What matters most is his factual observations. At one point he notes that a sum of squares routine sometimes produces a negative result(!). He also notes that they are not able to reproduce their earlier results using the model. Reproducibility is one of the foundations of science. Not only can others not reproduce their results, they can't even reproduce their own results.
This raises a further question that I haven't seen mentioned: was this code and data under version control? And was that lack of version control considered a bug or a feature? We are talking about billions of dollars being spent based on models that apparently had no source control . If they did, they should be able to reproduce.

This is a dumb and sensationalist article that adds nothing to what we already know. The most interesting news to come out of climate gate in the past few days are a behind-the-scenes timeline of scientists illegally suppressing responses to FOI requests, the fact that many IPCC analyses might be irreproducible due to poor code quality, and private admissions of climate scientist doubt in certain analyses while maintaining a public face of absolute certainty:
>1)http://camirror.wordpress.com/2009/11/25/willis-eschenbachs-...
>2)http://www.tickerforum.org/cgi-ticker/akcs-www?post=118625&#...
>3)http://camirror.wordpress.com/2009/11/23/john-mitchells-revi...

Inane and (of course) overblown. "The conspiracy behind the Anthropogenic Global Warming myth (aka AGW; aka ManBearPig) has been suddenly, brutally and quite deliciously exposed", he says. He offers the same out-of-context snippets as everyone else. Blah blah "eco-fascist" blah blah "Libtards" blah blah. A volley of links making such sensible claims as that the Copenhagen meeting is "a step closer to one-world government".
Nothing new to see here.

It's not for nothing that the Daily Telegraph is nicknamed the Torygraph, here in the UK: it's notable for a rather specific right-wing political spin, and climate denialism is part of the package.
There are certainly things to be worried about in those emails, but proof of a conspiracy they ain't.

I prefer the Economist's overview:
http://www.economist.com/sciencetechnology/displayStory.cfm?...

Alright HN, this is getting embarrassing.
Should I soon expect to see articles arguing for the "final nail in the coffin" of evolution because of gaps in the fossil record?

This isn't just gaps. This is an outright conspiracy to present a certain case and make sure no opposing view is heard.
If we had such a conspiracy in the field of evolutionary biology, then yes, I'd be rethinking how seriously we should take the theory of evolution. Fortunately, biologists are a lot more open with their data than climate scientists.

Of course I disagree that any reasonable person would equate AGW and evolution, as you have done. In the spirit of Thanksgiving, that's as harsh as I'll be today.
I think your main point is that you feel there are too many threads on this subject.
On this issue, I almost agree with you. However, this is a watershed event, so perhaps it really does deserve more posts than the release of a new iphone app.
Happy Thanksgiving.

Reply on news.ycombinator.com