Hacker News Reader

Comment by machina_ex_deus | Hacker News Reader

Here's the real reason: they are all copying code from each other. Tons of tons of code. Literally all climate global circulation models share a lot of code.

If the author had actually seen how crap the climate models look, maybe he'll realize the argument is actually backwards and climate models are bad.

I've tried code reviewing some of these models just to get an idea of how reliable they are. My mind was blown by the extremely poor quality of the code combined with literally no redundancy between different models in some parts. If some code written in 1990 just made a slight mistake, nearly all climate models would have the same bias.

And I did find some distributing surface level inaccuracies, such as constants being taken from very old inaccurate measurements in the 1980s physics books (at least according to the comment), some constants being very low precision compared to the accuracy required (some constants with only 2 decimals of precision), and some things that aren't constant at all and should depend on temperature/pressure/CO2 levels/height.

And these are important in a system that actually spends huge efforts to model factors with much weaker impact.

View on HN

One thing that is worth keeping in mind is that the end result can end up still being right.

Academic Fortran is horrible but it's also (hopefully) maintained by people who know what they're looking for — egregious errors are like errors, things like "this is constant when it shouldn't be" sounds more like a modelling assumption.

People got relatively good climate (well, temperature) estimates from the 80s (if you input the right CO2 data for the future), but I'd image the modern ones more subtle predictions could be quite wobbly (i.e. people complain about them having way too many parameters but I'd imagine many of these are for specific "tasks" rather than massively dominating the overall model)

There are two individual claims, that should be inspected individually:

1. Data from the field showing warming up / other real world observations. I'm not disputing this. This has been corroborated by many different methods without any clear single point of failure.

2. Given 1, what is the probability that the reason for the observed effects are CO2.

This is the crucial policy question, and the problem is that the evidence for 2 is actually purely from these models.

It's easy to take the conjunction fallacy here and immediately jump to 1&2 being more probable than 1 alone, but it's wrong thinking.

The problem is only magnified, in my opinion, by the fact that only few parameters have even had the same kind of detailed analysis that CO2 had. And then the code has clear bias towards the narrative, and the forcing of CO2 in the model is extremely detailed in the parts where the narrative says it's important (radiation forcing) while neglecting many other effects, and taking various things that actually depend on CO2 to be constant.

And finally the problem is that the policy decisions are the complete opposite if 2 is wrong: if we're heading towards more chaotic weather and CO2 isn't the culprit, you would want to rely on fossil fuels instead of the weather (a.k.a solar and wind) for energy.

The correlation with CO2 output is very very good.

The Berkely earth group went in with basically exactly the same question, did everything from scratch and broadly replicated the IPCC curves

> The correlation with CO2 output is very very good.

Sorry to recite a common wisdom that "correlation does not mean causation".

Ergo all causal hypotheses are unfalsifiable? After all if a model fits the data well, that's just correlation.

Do you have a competing theory? No one has been able to propose one that fits as well as greenhouse gases, and god knows people have tried.

However, there is a strong first principles model on greenhouse gases effects in the atmosphere: CO2, Methane, and others. All produced as byproducts of agriculture and industrial revolution technologies with ballpark accounting going back to 1800s. In addition there are mass balances accounting for carbon uptake by biomass from satellite imagery.

This isn’t a shot in the dark using excel linest(atmospheric _co2, global_average_temperature) trend plots and calling it a day.

There is sound first principles model to solar thermal energy retention. It may not predict the global average temperature in the next 15 years within 0.2C but you have got to be kidding me bringing up that intro to statistics caveat “correlation!=causation”.

Like do you know how inaccurate most physics modeling is and still tends to be useful?

> One thing that is worth keeping in mind is that the end result can end up still being right.

“ On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.” - Charles Babbage

I think the issues you’re mentioning are potentially impacting the world policies (and not a lot of things fall in that category). So if you have some time, it would probably be a great idea to write a detailed post sharing your concerns somewhere.

A writeup would be nice, but frankly, the issues he's mentioning are "to be expected" for anyone who has spent enough time in academia. They were the same issues I'd see in physics and engineering. I wouldn't expect the climate folks to be any different. Why would they? There's no incentive for them to be.

Right.

This appears to be an example of modern software engineering principles, or even the notion of basic automation, not having reached climate researchers.

I'm not surprised.

Electrical engineers, engineers in general, insurance mathematicians, physicists.

They write code. System critical code. Code that puts fundamental research, lives, and trillions of dollars at stake. They're very good at keeping track of the complexity. Not always so good at relaying it. They're sometimes legally required to test. Testing sometimes means doing the same manual thing excessively. That happens. Sometimes not.

To this day, I can't believe how many extremely smart people write code for a living, but because of their training, they don't think of themselves as "software developers". So they don't read up on what software developers do to not kill themselves. (Of course, #notall...)

Eh - sorry. I worked with both engineering and physics professors and PhD students when I was in grad school. Everything the top comment says about climate science applies equally well to those disciplines (in academia).

It's not because climate science folks are less rigorous, or less capable. It's because academia indirectly punishes those who do care about SW quality and correctness.

> I think the issues you’re mentioning are potentially impacting the world policies

That said, the climate models tell us very little useful information we didn't already know. I don't think they have much policy impact. Climate change as a phenomenon is not proved by computer modeling, but by real world measurements and established theory about atmospheric processes.

Irregardless of the code quality, the uncertainties in the model are enormous. (No code can simulate the earth without uncertainties.) We can't use such a tool to finetune our response. Gambling for 2.5°C and getting 5°C is too great a risk. We need to cut emissions as quickly as we are politically able to achieve.

I agree large scale modeling generally should be taken with a (big) grain of salt. Looking at you epidemiologist and economists...

> That said, the climate models tell us very little useful information we didn't already know. I don't think they have much policy impact.

You're wrong in thinking so, these models have an enormous impact since they are what are being used as 'scientific evidence' for the current 'climate change' craze. Without these models there would be only those measurements you mention which in no way indicate the type of trends which the models are being used to warn about. All the measurements show is that it sometimes gets really warm, sometimes really cold, sometimes really windy and sometimes really dry. Measurements do not show a steadily and unerringly upward climbing curve of 'sometimes really warm' and - important in this context - they do not show the current period to be the extreme which it is made out to be. Taking the continental USA as an example - an area for which much data has been collected in a standardised fashion for a long time, making such comparisons possible - it is noticeable that most temperature records - high but also low are found in the early 1900's, mostly centred around 1936 [1]. If you look at newspapers from around that time the tone was comparable to what it is now (in more ways than one given the threat of war on the horizon but that is unrelated to this issue). Looking back further it can be seen that the average temperature has been steadily dropping (as "average over many years", not "average of the last 5 years") since about 8000 years ago when the climatic optimum is to be found [2] somewhere. It is also noticeable that this temperature decline does not follow a straight line, sometimes the average temperature was supposedly - these measurements can only show averages over longer time frames, not year by year - higher than the average during the optimum, sometimes they were lower. The trend is clear and is pointing downwards towards the coming glaciation, whenever that may be. Without the models to use as 'evidence' there would be no 'climate deniers' (an astoundingly dishonest term since most people who are called such do not deny anything about climate, they just do not trust these model calculations with good reason) since current phenomena would just be "weather", not "proof of climate change" - just like they were in the early 1900's.

These models are The Science™ for climate alarmists, no more and no less. They are not the oracles of wisdom as they are made out to be since they suffer from the Garbage In, Garbage Out problem like so many other simulations. As such they are not good tools to use to direct policies but more clubs to wield against recalcitrant opponents.

Does this mean the climate does not change? No, it does not, it changes as it always has and may change in the direction the model calculations show, at least for a while - look at the variations in [2] for an indication of such changes - and it makes sense to prepare for the consequences of a warmer - or colder - climate when possible. Does this mean there is no need to cut 'emissions'? No, it does not mean that either, there are other good reasons to reduce those since these emissions do not just consist of CO₂ but encompass other substances which are actually polluting. It does mean that the laser focus on 'cutting emissions' is most likely the wrong approach since there are far more effective ways to prepare for the effects of a changing climate. No amount of emission reduction will stop the next Katrina disaster but raising levies will, no amount of extra carbon taxation will solve the problems caused by drought but investing in desalination plants [3] will, no milestone in electrification of road vehicles will help with dealing with increased precipitation but upgrading storm drain systems will, etc.

There are other reasons to want to reduce the dependence on fossil fuels as can be seen in Europe which had become unhealthily dependent on natural gas from Russia or the many wars fought by the USA to secure the supply of oil from the middle east - these substances often end up being used as tools to achieve some geopolitical end. Reducing these dependencies where economically feasible is a good investment in future stability and security.

Many of the mentioned reasons - and more - speak for a reduction in 'emissions', they just do not focus on CO₂ as being the reason for reducing them. What all these reasons lack is a sense of urgency and that is most likely why these models are used in their stead. It is much easier to get people scared, then tell them what they have to do to feel safe again than it is to try to convince them of the wisdom of reducing pollution and reducing dependencies for a more stable future.

[1] https://en.wikipedia.org/wiki/U.S._state_and_territory_tempe...

[2] https://en.wikipedia.org/wiki/Holocene_climatic_optimum#/med...

[3] ...yes, yes, they produce brine which needs to be handled in some way - this can be solved in many ways so

> they are what are being used as 'scientific evidence'

No, you're wrong about this. The scientific evidence for climate change is multi-faceted and abundant.

All science involves some modelling. No model captures all forces on a system. Despite this, we are able to determine causal relationships based on simple models and empirical observation.

99% of experts consider the evidence for climate change caused by human carbon emissions convincing. Do you think that would be the case if the only evidence was a large, opaque, uncertain, buggy program?

[1] https://en.m.wikipedia.org/wiki/Scientific_consensus_on_clim...

Far more than 99% of ‘experts’ in 1400s Europe considered the Sun to revolve around the Earth.

Appeals to popularity have obvious pitfalls.

There's a difference between the scientific community of 1400s Europe and today.

Different in some aspects, certainly not in all possible aspects.

But perhaps in the aspects most relevant to this discussion?

Such as…?

No, actually the comparison is rather apt given the way some subjects are treated - as quasi-religious dogma where criticism is being treated as heresy. Critics of the leading narrative around the changing climate are called "climate deniers" who are "anti-Science" and either uneducated knuckle-draggers or nefarious plotters in the pockets of the enemy - usually the fossil fuel industry. Critics of the narrative around SARS2 are called "anti-vaxxers" and "Science-deniers" who do not "listen to The Science™" and probably are *-ists and *-phobes as well.

The parallel is clear, this is how critics of the narrative around the movement of the Earth, Sun, stars and planets were treated in the 1600's. In that case it was the power of the established Church which saw its monopoly on "truth" threatened by some upstart with a telescope. If his revelations about the orbits of the planets was accepted it would undermine their interpretation of scripture which would undermine their authority so they lashed out to silence the heretic. There is a striking resemblance to the way critics of current narratives are treated, as heretics who need to be silenced before their wayward teachings can poison the populace. This is now how science tends to be done and it is not how science (with a small 's') should be done. Everything should be open to criticism, theories should be mercilessly attacked. The more these attacks falter, the strong the theory will be. Should the attacks succeed in proving the theory wrong everybody will benefit from it.

In the discussion around climate models this means the models need to be subjected to rigorous and merciless testing to find their flaws. They should be tested against historical data, the code should be scrutinised for everything from simple errors to the use of fudge factors and 'magical constants' or 'damping mechanisms' or any other such unproven/unprovable manipulations of the data. If the models withstand these tests and actually manage to follow historical data on which they have not been tested to a reasonable degree of accuracy they will be the stronger for it and their predictive ability will be far higher. This would be a good thing for everyone involved.

> 99% of experts

Ah, the fabled /9[789]%(.9)?/ argument raises its ugly head again. This claim has several sources, let's have a look at them.

In 2009, the University of Illinois sent a survey [1] online to about 10200 scientists (of many disciplines) with the following two questions:

1: Do you agree that global temperatures have generally risen since the pre-1800s?

2: Do you think that human activity is a significant contributing factor?

3146 responses were received of ~10,200, and of that 31%, 90% said yes to the first question while 82% said yes to the second question. When narrowed down to actual experts in the field (mostly meteorologists) 64% answered 'yes' to the second question. Of the 3146 responses 77 were selected which came from self-professed 'climate experts'. Of these 77 individuals 75 answered 'yes' to the second question. 75/77 = 0.97 -> 97% of experts agree

In reality a self-professed "expert" group of 2.4% of the answerers were selected who in turn formed 0.8% of those who received the survey. The well-known saying about statistics - there's lies, damned lies and statistics [2] - comes to mind here.

Then there was the '2013 Cook study' [3] entitled 'Quantifying the consensus on anthropogenic global warming in the scientific literature' which is often cited as another source for the '97%'. Cook and his team searched the Web of Science for the phrases "global warming" and "global climate change", then categorised the results to what they considered to be their level of endorsement of anthropogenic global warming. While Cook's conclusion - “Cook et al. (2013) found that over 97 percent [of papers he surveyed] endorsed the view that the Earth is warming up and human emissions of greenhouse gases are the main cause.” - was taken for gospel by those who were looking for more endorsement of the earlier '97%' study. What is it based on? Cook created a category called “explicit endorsement without quantification” (papers in which the author did not say whether 1 percent or 50 percent or 100 percent of the warming was caused by man). He had also created a category called “implicit endorsement” for papers that imply (but don’t say) that there is some man-made global warming without quantification of the effect. These were the categories used to calculate the '97%'. Many scientists whose papers were classified in these categories protested to Cook [4] for being selective in which papers were cited and misrepresentation of their positions on the subject of anthropogenic climate change.

Cook's data is available in the study [3], the positions on anthropogenic global warming look like this:

   Explicitly endorse AGW 50+% = 0.54%
   Explicitly endorse – does not quantify = 7.72%
   Explicitly endorse AGW without minimizing it = 24.36%
   No Position = 66.73%
   Implicit minimizes/rejects AGW = 0.45%
   Explicit minimizes/rejects AGW = 0.13%
   Explicitly minimizes/rejects AGW as <50% = .08%

These 7 groups are summarised in 4 categories by Cook:

   Endorse AGW 32.6%
   No AGW Position 66.4%
   Reject AGW .7%
   Uncertain on AGW .3%

The study arrives at the '97% endorsement' by leaving out the 66% who do not have a position which leaves the 32.6% who endorse some form of AGW against the 0.7% who reject it. Apart from the mentioned problems with the categorisation this is another example of lies, damned lies and statistics [2].

In short, don't use the /9[789]%(.9)?/ argument in an attempt to appeal to authority, especially not in the context of a discussion around the viability of global climate models as evidence for future climate change. First because of the mentioned reasons but also because the '97%' argument has no bearing on the validity of climate models since it just - rightly or wrongly - claims that '97% of experts agree that anthropogenic climate change is a fact'.

[1] https://www.sciencedaily.com/releases/2009/01/090119210532.h...

[2] https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statist...

[3] https://iopscience.iop.org/article/10.1088/1748-9326/8/2/024...

[4] http://www.populartechnology.net/2013/05/97-study-falsely-cl...

You claim your standpoint is scientific. But the only peer reviewed article you cite is the one that contradict your claims!

There's no doubt a scientific consensus that human carbon emissions are causing global warming. There are multiple extremely heavyweight sources for this, starting with the IPCC reports.

I'm not a climatologist and can't claim to know the extensive literature. But I do understand the proposed mechanism, I trust people are generally not falsifying data, and I don't see any reason for why thousands of scientists and the IPCC would conspire to make up a lie like that.

On the other hand, the fossile fuel industry has a very clear incentive, and it's well documented they have financed researchers supporting their interests.

I did study meteorology (but not climatology). I do understand the proposed mechanism but I do not trust the models, mostly because I know the limitations of weather and climate models and recognise the magnitude of 'fudge factors' in play when these models are proposed. In many ways they resemble the old joke about the theoretical physicist who was charged with figuring out how to increase the milk production of cows. Although many farmers, biologists, and psychologists had tried and failed to solve the problem before him, the physicist had no trouble coming up with a solution on the spot. “First,” he began, “we assume a spherical cow ... ”.

You mention the clear incentives the fossil fuel industries have in toning down the environmental impact - I explicitly use that term instead of 'climate impact' which has replaced it in popular media since I consider the former to be a known and proven problem - their products have. This is certainly the case and stands without doubt. You do not mention the clear incentives the 'green' industry has in pushing climate panic. Don't you think this is a factor or do you simply agree with their messaging? Does it bother you that several fossil fuel giants have jumped on the 'green' bandwagon so they get to play both sides in this game of charade?

> There's no doubt a scientific consensus that human carbon emissions are causing global warming.

Sure, when you increase the emission of greenhouse gases there is some impact. How large this impact is is less clear, nor is it clear that the increased CO₂ concentration in the atmosphere is the main cause of the recent warm spell or whether it is caused by it. Historical climate records show CO₂ concentration trailing temperature changes, not leading them, the theory being that change in concentration is caused by the change in seawater temperature with colder water absorbing CO₂ (and other gases) which is released when the water warms up again. Another factor in play is the concentration of water vapour in the atmosphere - it shares 3 of the 4 main absorption areas in the spectrum with CO₂ - as is cloud cover (which has a significant effect on the planet's albedo (reflection coefficient)). Since temperature, CO₂ concentration, water vapour concentration and cloud cover are related and interdependent a change in one causes a change in the others. These relationships have not been not taken into account in most models until recently [1] while they are sure to play a role. There are many other factors which are either not taken into account or set at assumed values.

[1] this is one of the areas in which climate models have been improved in the last years: 'Cloud feedbacks have been confirmed as a primary source of these differences, with low clouds making the largest contribution. New observational and modelling evidence strongly supports a combined water vapour-lapse rate feedback of a strength comparable to that found in General Circulation Models (approximately 1 W m–2 °C–1, corresponding to around a 50% amplification of global mean warming). The magnitude of cryospheric feedbacks remains uncertain, contributing to the range of model climate responses at mid- to high latitudes.'(https://www.ipcc.ch/site/assets/uploads/2018/02/ar4-wg1-chap...)

You know more about the subject than I do, but I don't think you're entirely contradicting one of the author's key points: It's incredibly hard to rewrite a system from scratch - and in effect, to just write a system from scratch.

But it would make sense to have a "professional scientific fortran programmer" go in and do some housecleaning to make things right, without whining, "Why can't I just rewrite it in Java???" Heck, it sounds like fun to me, but that's because I'm basically a professional janitor, and hey, hurricanes and polar vortexes and all that.

So why doesn't that happen? Is testing improvements too expensive? Do scientific programmers just have no respect for the craft whatsoever? Is there nobody with the necessary interdisciplinary skill set?

My experience with academics is that most don't know / don't understand the benefits of coding things in a better way, so they don't even consider hiring someone to help.

The other issue is that the incentives aren't necessarily aligned. They don't get payed to share their code or to make their code reusable. The code is used to publish one or a couple of paper and then thrown away. Maybe someone else will reuse it in the future but the original author doesn't necessarily gain much from that

The way I see it is analogically to writing secure and memory safe code before people knew these things matter.

You needed someone with an incentive to go over code, find the bugs, break into computers and for the entire industry to mature from using strcat to having bug bounties and fuzzing and protect zero. It didn't happen over night and there had to be incredible incentives on the attacker / bug finder to exist, to probe and test the parts that aren't tested, and to serve as the devil's advocate.

The analogy is that it is hard to find bugs which happen silently. Computation inaccuracies are a huge class of bugs that also happen silently. There are no incentives to look at old code.

There's just nothing like it for climate models, and the default behavior of code development is that whatever isn't tested or validated by something will always stagnate with bugs.

The code copy just makes everything worse because implementing the same thing in two different ways by different people is actually good at catching bugs.

So even if I wouldn't have trusted a single climate model, I might trust several climate models developed independently.

The way it works now is that they always prefer to rely on code of others, because academia is driven by citations, and creating new models from scratch means you're missing out mutual citations.

It's a systematic problem, the same way security had been a systemic problem: current methodology just doesn't produce code that fits the requirements. Nobody had neither the incentive nor even the idea that these things should work differently.

Admitting that this is a systemic problem is just a politicized subject.

There are plenty of people saying these models are problematic but getting those working on the models to admit that enough such that the entire system can change its methodology, that won't happen because of human reasons.

> The way it works now is that they always prefer to rely on code of others, because academia is driven by citations, and creating new models from scratch means you're missing out mutual citations.

Ah! This is the interesting point that stands out, as I hadn't anticipated such an incentive. It's a nasty one.

I guess that you need at least some scientific knowledge about the project's subject in order to work with the domain and maintain the code, but not on an expert level, because that would be a waste of resources put a great scientist on maintaining and polishing old code (even if it is important).

And why would a good programmer agree to suffer to the whims of a Professor that is clueless about programming even if the domain itself is important and interesting?

> So why doesn't that happen? Is testing improvements too expensive? Do scientific programmers just have no respect for the craft whatsoever? Is there nobody with the necessary interdisciplinary skill set?

Academia relies on paying grad students peanuts and dangling the carrot of being able to get citations under their name to make up for it. There's no way to publish a paper by cleaning up code, much less get cited for doing so.

Bear in mind there's basically zero "general" funding available. Everything has to be paid out of a specific grant and they want to see it being used for that specific purpose, which likely ends with publishing a paper.

But the short version is yes, academia has no respect for the craft. Papers are what matters, so code will always be the minimum necessary to publish a paper, which is generally: copy someone else's fortran/numpy code, tweak it enough until it runs (once, on your laptop) and gives the result you were looking for, publish.

> "professional scientific fortran programmer"

That is too late You CAN'T start with a developer that, hopefully, do good code. You MUST make good code to be the default, from the start. Is the only chance of lasting improvements.

ie: Is necessary a "Rust/Ada for Fortran", and this ALSO must make, somehow, make some significant impact in current Fortran developers, make them salivate for the chance of do the change.

But iterate on Fortran(C/C++..) and hope that will be better? NEVER will work.

> Here's the real reason: they are all copying code from each other.

Right, code reuse is dangerous and something to be avoided. From now on, let's write everything from scratch.

If we reuse code, we risk missing a potential time investment!

I don't think this reflects the OP's concern: It's not that code-reuse is bad in and of itself, it's that the code being reused is being reused without maintenance or review.

Good point. For example, I'm sure no trained professional would ever copy code from StackOverflow.

This gave me both a good groan and chuckle.

You're pointing out a place where I think many software engineers are either untrained, unprofessional, or both. There's already a large body of discourse around the problems that come from these practices—licensing violations, security breaches, and stupendously dumb bugs.

Professional rigor is a thing I wish our industry had more of, but in practice you don't see much rigor outside of safety critical applications because there is rarely incentive for it.

I think that's pretty clearly not the point. As someone who has worked with legacy scientific code written in Fortran, the copying is not the issue, it's blindly copying code that was written 50 years ago, probably by a graduate student, and not reviewed or revised in any way since. As the parent comment points out, if everyone reuses that code, the same errors propagate. It's not the type of reuse that comes with a nice GitHub repo and an established mechanism for reporting and fixing bugs.

In my case, one of the Fortran files essential to my group's research has 11 (!) versions with only filenames like file.f.goodworking (that's literally one of the extensions) to differentiate them and no version control or history.

Reply on news.ycombinator.com