Hacker News Reader

Comment by ekianjo | Hacker News Reader

Where is the story hère though? So they looked retroactively and found some models in the past that fitted with current observations even though they had no clue that China would become such a powerful force in terms of CO2 emissions?

If you run random models and wait long enough you will always be able to find at least one model that predicted the current observations. That does NOT mean the models were right, it means that you are basically ignoring all the other models that made bad predictions. So in order to make a headline out of this you would need to make a extensive review of all the models we had back then and found out HOW MANY were actually not bad.

Would not make that much of a headline because you would find most models were bad. Anyone involved with forecasting will know what I mean.

View on HN

Whether or not China emitted lots of CO2 doesn't matter here. The old studies didn't predict how much CO2 the world would emit. They predicted how much the planet would warm, for various amounts of CO2 emissions. It turned out they were correct about that.

Among the models this study looked at was Hansen's. That's the NASA scientist who testified about global warming to Congress in 1988. It'd be quite a coincidence if there were lots of models with a wide variety of predictions, and the one model that was presented to Congress just happened by chance to be one of the few that got it right.

But if you can actually show that there were in fact a bunch of models by scientists of similar stature, which made really different predictions and were left out of this study, then that would be interesting.

Thanks, I feel like I finally found a gem of a comment in the middle of all this heavy back and forth.

Thank you for an intelligent comment.

There are people downthread who are arguing that because numerical instability bugs exist and climate models are complicated, climate models can’t be trusted. Literally any flimsy pretext is used to dismiss work that people don’t like.

This conversation is dominated by people who have just enough knowledge to defend their pre-conceptions, but no more.

Your comment doesn't match what the paper actually says[1].

Your claim that "The old studies didn't predict how much CO2 the world would emit" is directly contradicted on the first page. The paper begins by pointing out that "model projections rely on...accurate assumptions around future emissions of CO2". In fact, one of the criteria the authors used for choosing a model to include was that it came with a time-series prediction of "future projected atmospheric CO2 concentrations".

With that restriction, the authors found 11 papers containing a total of 14 models. Unless the authors of this paper left out other predictions from the original papers, the only papers included in the study that "predicted how much the planet would warm, for various amounts of CO2 emissions" were Hansen's, and the other 9 papers included in this study only made one prediction for both emissions and temperature.

This paper also points out that Hansen's 1988 "most plausible" model, which you described as "the one model that was presented to Congress", was not accurate under the usual definition of accurate: "H88's “most plausible” scenario B overestimated warming experienced subsequent to publication by around 54%."

Then how does this paper lead to the headline that "Even 50-year-old climate models correctly predicted global warming"?

What the authors of this paper have done is to compare errors in the models' predictions of CO2 emissions to errors in the models' predictions of temperature. In other words, the models were wrong about both, but in the same direction. They overestimated both future emissions and future temperature.

Even so, the claim that the models were accurate is a linguistic trick. The table on line 326 shows what the authors are calling skill score: "A skill score of one represents perfect agreement between a model projection and observations."

Not a single model has a "skill score" of one, or for which one is within the uncertainty. In fact, all of the models are wrong in the same direction, even after this paper has attempted to correct for their incorrect assumptions about future CO2 emissions.

The linguistic trick is that this error is not, according to the authors, "statistically significant". But that's a tremendous abuse of the phrase. The authors are effectively saying: "The null hypothesis is that the models are right, and we've failed to disprove it." They've assumed the conclusion.

I suspect that this paper is being promoted for political, not scientific, correctness.

1: A PDF of this paper can be found at https://pubs.giss.nasa.gov/docs/tbp/inp_Hausfather_ha08910q....

All that says (in your second paragraph) is that if you're going to predict the actual temperature, you need an accurate CO2 prediction, which of course is true.

But here is Hansen's original 1988 paper: https://pubs.giss.nasa.gov/abs/ha02700w.html

From the abstract: "We make a 100-year control run and perform experiments for three scenarios of atmospheric composition. These experiments begin in 1958 and include measured or estimated changes in atmospheric CO2, CH4, H2O, chlorofluorocarbons (CFCs) and stratospheric aerosols for the period from 1958 to the present. Scenario A assumes continued exponential trace gas growth, scenario B assumes a reduced linear linear growth of trace gases, and scenario C assumes a rapid curtailment of trace gas emissions such that the net climate forcing ceases to increase after the year 2000."

Note the word "assumes." There's no attempt to really model what humanity would choose to emit. None of these assumptions exactly match what happened with CO2 and other greenhouse gases, so of course none of them exactly match the temperature either. But adjusted for actual greenhouse emissions, they come remarkably close.

Were they completely accurate in that way? Certainly not. People knew a lot less back then. It took a while to nail down things like the effect of water vapor, which is a greenhouse gas but also makes clouds that reflect sunlight away. But they were close enough to make good policy decisions, and if we'd paid attention, we'd be much better off today.

Incidentally, the case for global warming doesn't depend on models. There have been warming events before, and there's enough evidence in geological history to tell us what's going to happen. Hansen's book has details.

> But adjusted for the actual CO2 emissions, they come remarkably close.

Take a look at the table I mentioned (line 326), in particular "ΔT / ΔF skill". That's the change in temperature over the change in forcing (ie, CO2 etc). Hausfather et al call it "implied TCR". In (overly) simple terms, it's the predicted increase in temperature per unit of CO2 emitted.

Look at the numbers: 0.51, 0.41, 0.63, 0.42, 0.83, etc. The best is 0.87. The average was 0.69.

The authors attempted to adjust for actual CO2 emissions and found that, after the correction, the models were predicting 40%-90% of the observations, with an average score around 70%.

Is that "remarkably close"?

Well, we might disagree about the meaning of that term. But there's another observation that we should agree on. Note what the authors say after that table:

> The average of the median skill scores across all the model projections evaluated is 0.69 for the temperature vs time metric.

> Using the implied TCR metric, the average projection skill of the models was also 0.69.

In other words, the models are, on average, as accurate with or without the correction. Adjusting for actual greenhouse emissions doesn't make the models more accurate (nor less).

I would say it was close enough to drive good policy decisions. It's unfortunate that it didn't.

If you are correct, what are the policy implications? In particular, is it your view that we should stay on fossil fuels, rather than trying to move to renewables? And are there any possible future scientific findings that could lead you to change your view?

Yeah, this could be survivorship bias: https://motls.blogspot.com/2017/03/selection-of-climate-mode...

As someone who's written basic fluid dynamics codes, I'm blown away by the complexity of climate models. The errors I ran into in debugging my relatively straightforward codes were incredibly subtle. Like pressures, velocities still "looking right" but still being wrong, due to numerical viscosity and other artifacts of the inexact solutions to Navier-Stokes. This is for flow in a box in a steady state. For an entire planetary atmosphere at a time scale of decades, I can't even imagine how they can track down every error without duct taping "corrections" (overfitting) on everything.

Doesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom".

But also yes, it's complex, that's why independent groups dedicate huge amounts of time researching and building them and then models are compared against each other.

> Doesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom".

Not in the slightest. For example, the output of a numerically unstable [1] algorithm, run on 64-bit floats, can diverge arbitrarily from the output of the same algorithm run on real numbers with infinite precision (which computers can't do). Often this will lead to obviously wrong outputs, but not always. There are numerically-stable versions of most of the workhorse linear algebra algorithms, but (1) sometimes it's possible to use a faster, numerically unstable algorithm, and this is a matter of judgment that can be mistaken and (2) it's easy to introduce numerical instability via subtle bugs, or even via the exact arithmetic steps you use to calculate a result (in particular, trying to add or subtract two numbers of very different orders of magnitude is a big no-no).

[1] https://en.wikipedia.org/wiki/Numerical_stability

This is not a rebuttal to the parent. They’re talking about stability of prediction of aggregate properties, and you’re fixating on numerical precision errors and algorithm bugs in particular implementations. The one has little to do with the other.

Numerical instability leads to obvious, wildly inconsistent errors. It isn’t subtle. Literally no climate simulations are going off the rails because of these kinds of first-year grad-student code problems.

Correctness of numerical codes is a huge and under-appreciated concern in climate models. There have been some attempts to apply software correctness techniques to climate models. For example, https://people.engr.ncsu.edu/jwb/papers/altuntas-correctness...

Much more work in this direction is needed.

... but if you think, "hey, so the models could be wrong, how reassuring", think again. It's just as likely that reality is worse than predicted.

> It's just as likely that reality is worse than predicted.

This cannot possibly be correct; it's the same reasoning as the argument "Either the sun will rise tomorrow or it won't. These two possibilities exhaust the entire space. Therefore, the odds of the sun rising tomorrow are 50%, one in two." The premises are completely correct -- and unrelated to the conclusion.

It isn't the same reasoning at all. Our prior for what could possibly happen should be informed by what has happened before, and we know to the best of what geology can tell us, that many mass extinctions on earth were caused by CO2 mediated warming, including the one that wiped out 95% of all species at the end of the Permian. https://www.nytimes.com/2018/12/07/science/climate-change-ma...

How can you say that unless we know how to sort good models from bad ones before observations kick in? As far as I know its always very easy to fit historical data with thousands of models but very few will actually be useful for prediction and we have no idea which ones.

This blog post is highly misleading and poorly-informed. The author of it does not seem to know that climate scientists do test their models against reality, leading him/her to waste a lot of words explaining how cargo cult science is bad, and how overfitting is being done, etc. These points are valid in the abstract, but don't represent how real climate science works.

Amazingly, the author at one point says:

"In proper science, the different theories are bloodily competing with each other."

as a contrast to climate science, where supposedly these models are just peacefully cooperating to raise more funds. Anyone who has been in the middle of a conference where people with competing models are comparing results (to each other and to observations) knows how off-base this is.

total heat content of the planet is way way simpler than fluid dynamics

... this is sarcasm? I honestly can’t tell.

Can you elaborate your point of view a little more?

How can you distinguish survivorship bias different from... developing better models? If there are any models that made quantitative predictions of global temperature change (whether warming, cooling, or neither), please let me know. I am extremely interested :)

P.S. Read the paper here: https://pubs.giss.nasa.gov/abs/ha08910q.html

> even though they had no clue that China would become such a powerful force in terms of CO2 emissions

I think you're making the same mistake that you're attributing to the story. It really doesn't matter what China does per se. The global energy consumption, roughly, follows a log-linear trend. If the energy comes from fossil fuels, the CO2 emissions would also follow the same trend [1]. It doesn't matter whether it's China or someone else; the CO2 emission would go up if we don't innovate and rely on fossil fuels.

As an analogy, think about the Moore's law. Does it matter whether the chip is made by Intel or AMD?

[1] https://commons.wikimedia.org/wiki/File:Global_co2_emissions...

Disagree. China's move to a more market based economy has resulted in them tripling their fossil CO2 emissions compared to the 1990s. India doubled. EU and the US are slowing down in emissions but on the whole China is driving the huge increase of CO2 emissions worldwide, matching their rapid economic development. If you did not take that into account you would miss on the acceleration of CO2 content in the atmosphere over time.

The manufacturing offloaded to chinese coal would then have been run on USA gas & shale oil: same difference.

There is no physical phenomenon that causes the log-linear trend, its just a model that happens to fit for a small select period of history. Moore's law isn't an actual law I hope you understand that..

Rising prosperity comes with rising energy consumption. If China didn't export so much it wouldn't be able to import the fuel that it used. Its dramatic rise was definitely not a given.

> There is no physical phenomenon that causes the log-linear trend, its just a model that happens to fit for a small select period of history.

Of course it's a model. Please look up the Kardashev scale and correlations that people have developed based on it [1]. The energy output is not necessarily coupled with the CO2 emission unless we burn fossil fuels to generate power.

[1] https://en.wikipedia.org/wiki/Kardashev_scale#Current_status...

Your comment contains hypotheticals ("run random models and wait long enough you will always be able to find at least one model that predicted the current observations") that are NOT what the present study did.

As noted in TFA, they looked at 17 forecasts from 14 models and found that the temperature predictions of 10 of them were consistent with what has happened in the meantime. And if you adjust for forcing (CO2, methane) then the predictions improve further.

> As noted in TFA, they looked at 17 forecasts from 14 models

Pardon a little pedantry:

>> The researchers compared annual average surface temperatures across the globe to the surface temperatures predicted in 17 forecasts. Those predictions were drawn from 14 separate computer models released between 1970 and 2001. In some cases, the studies and their computer codes were so old that the team had to extract data published in papers, using special software to gauge the exact numbers represented by points on a printed graph.

They compared annual average surface temperatures across the globe to 17 forecasts, from 14 separate models.

But....how many models did they look at, before choosing those particular 14 models?

TFA doesn't say.

Articles written in this style provide rich fodder for conspiracy theorists, particularly because there is no shortage of examples in the past where authoritative, trustworthy organizations have gotten caught in lies. Rare is the climate change article I've read that can't easily have similar holes poked in it.

What's the real truth here? Based on the literal content of this article, no one knows. It is speculation vs speculation.

I've said it before, and I will say it again: if your persuasion tactics aren't working, it might be worth considering whether you should change your tactics.

TFA article does say. They looked at 17 forecasts drawn from 14 models. You are the one who implicitly believes they are lying about that and only reporting 17 out of some huge unknown number. The reality is that the vast majority (>99%) of climate change research does not produce the type of forecast, or really any forecast at all, that one could do this analysis on. Therefore, the 17 forecast number is actually very easy to believe.

With sincerity, I think you may misunderstand what GP is pointing out. I understood “forecasts” to be a simulation run using a model. In that case, they took 14 models and ran those each at least once to come up with 17 predictions.

What GP wants is transparency around how many possible models were there over the last decades from which they chose the 14. This gives the reader a better understanding of whether 14 is a lot or a little. It is a fair point IMO regardless of their overall stance on the issue.

They did a literature search and chose what was reproducible. This has been covered in TFA and highlighted by a comment nearby: https://news.ycombinator.com/item?id=21709446

How many models are currently published for reference?

Simulation is one way to produce a numerical forecast, but the prediction of any forecast model would never be the result of a single simulation run. Model predictions are always (or should always) be reported as an expected value, or the average over many simulation runs. The expected value of a random variable (e.g. mode output) is probably the most fundamental concept in all of statistics/probability, and many mind-wrenching (read: wrong+incoherent) internet hot takes are rooted in logical fallacies about the science of prediction. The reason that there are more predictions (17) than models (14) is that several of the models (eg Hansen 1981) consider several distinct “scenarios” based on differing assumptions about future levels of GHG emissions.

Secondly, the forecast models evaluated in the study are from the likes of James Hansen, William Nordhaus, and the IPCC (in other words, authoritative sources). So at the very least this adds weight to the hypothesis that the models published by the IPCC are accurate and methodologically sound. Sure, there is some probability that the models and reality corresponded by random chance, but that probability is made ever more diminishingly small as evidence accumulates.

So, I don’t think any of the comments in this thread that accuse the authors of retrospective cherry-picking of climate models are anything but embarrassing examples of pseudo-intellectual contrarianism and/or confusion over basic statistics.

The models considered by the study, their predictions, and the methodology used are all located here and were easily found after 30s of digging: https://github.com/hausfath/OldModels

Agreed. The point of criticism that the authors didn’t include all relevant models is among the lowest of the low-hanging fruit I can imagine one picking. And while the process of peer review isn’t without its flaws, I can also imagine that fruit has long since been picked by the reviewers, who have far deeper knowledge of the published models than I or just about anyone else in this thread, save for the actual climate scientists among us.

Thirded. The posting of climate-related stories on HN brings out a lot of un-informed skepticism. The resulting back-and-forth tends to crowd out everything else.

In many of these cases, it's clear that the commenters have done no background reading, and have no climate science background. Their comments thus consist of simple armchair speculation (your "lowest of the low-hanging fruit").

In these comments, I have found people with strong opinions on the lack of seriousness of climate science, who have no idea we have gravimetric measurements of ice sheet mass. This is a dataset going back almost 20 years that has revolutionized geoscience. HN commenters have expressed strong opinions about melting ice, with no knowledge of this data and what it means.

I have found people who think that climate science consists of having a stable of models each with a host of knobs, and the research method is to turn the knobs or choose new models until the results look dramatic enough to publish.

The gap between the firmness of these claims and the knowledge to back it up just boggles the mind.

The most basic step people could take to fix this is to read the executive summary of the IPCC AR5 report, or (even better/easier) the executive summary of the NCA (https://science2017.globalchange.gov). This report was specifically written by people who really know this stuff, for people like us.

Serious question: based on what I've written, do you believe I am one of the people you describe?

Your comment is an example of the "middlebrow dismissal" [1] and overdone and uninformed skepticism that is prevalent on HN climate threads.

There's no indication that the authors are cherry-picking their models but the comments above are assuming this (without reading the original paper to get a sense of the care of the analysis). It's really easy to find supposed gotchas in this way - basically any analytic step can be questioned.

If you live in the Bay Area, AGU is next week at Moscone Center. It's the biggest one-stop-shop in the world for geoscience research. Maybe it's worth paying a one-day admission fee to see some of this in person?

[1] https://news.ycombinator.com/item?id=5072224

> Your comment is an example of the "middlebrow dismissal"

--> It's a word coined by pg that's sort of a term-of-art for the depressing tendency of the top HN comment to be something which pooh-poohs the company/topic/idea in the submission, doesn't add anything to the discussion, but (crucially) is worded convincingly enough to not just end up on the bottom of the thread.

--> e.g. "Lol that suckz because Google can steal ur passwerdz!" would not end up at the top of the thread. We wouldn't be terribly worried if someone actually said "Wait, I just audited NativeClient and it turns out you can achieve arbitrary code execution. Maybe you should avoid this software." The danger zone is comments which sound like the second but, on reflection, only tell you about as much as the first.

I love the first reply:

"Ironically the concept of a "middlebrow dismissal" seems to me to enable what it condemns--e.g. HN'ers can now just post "hey that's a middlebrow dismissal" instead of a detailed statement of why a particular response falls short. Basically I think pg made it worse by giving it a catchy name."

- something which pooh-poohs the company/topic/idea in the submission [Invalid]

- doesn't add anything to the discussion [Invalid]

- We wouldn't be terribly worried if someone actually said "Wait, I just audited NativeClient and it turns out you can achieve arbitrary code execution. Maybe you should avoid this software." [If you'd have read more closely, and less tribally, perhaps you might have noticed something along these lines]

> There's no indication that the authors are cherry-picking their models

My comment wasn't that they are cherry picking their models, it was that it is (as worded) not entirely clear that they are not. I also explained why this is may be harmful.

> but the comments above are assuming this

Mine weren't assuming anything.

> basically any analytic step can be questioned

Perhaps. But survivorship bias isn't a conspiracy theory.

> If you live in the Bay Area, AGU is next week at Moscone Center. It's the biggest one-stop-shop in the world for geoscience research. Maybe it's worth paying a one-day admission fee to see some of this in person?

I'm satisfied enough with the science to support doing something about climate change. My interest in this conversation is the apparent unwillingness of similarly minded people to consider weaknesses in their persuasion campaign. How do you foresee getting past the current impasse we find ourselves in?

I think you want to argue with people on the internet. Not playing.

In other words, no disagreement allowed.

Why don't you demolish me with your science and facts? Does that question even register?

You were demolished.

In a popularity contest manner perhaps. Nothing I wrote about was addressed.

Would you like to try? The authors of the paper seem unwilling, perhaps you'd like to give it a shot.

> TFA article does say.

It says something.

> They looked at 17 forecasts drawn from 14 models.

It does not say this. What you have written is your uncritical interpretation, what I wrote is a literal quotation, which allows for (satisfies) the conspiratorial possibility that I raised.

> You are the one who implicitly believes they are lying about that

You are speculating. Speculation is fine, but it's best to realize (and state explicitly) when you're doing it.

Know how I know you're speculating? Because while you only believe you can read my mind, I can actually do it, and I do not "believe they are lying". I was only pointing out that as written, they could be lying.

And I even told you the reason why I wrote that comment: "Articles written in this style provide rich fodder for conspiracy theorists..."

If you truly want to change minds, you might want to learn how the minds you want to change work. Go spend some serious time in conspiracy forums, I suspect you might learn something useful.

And rather than largely ignoring the actual words of someone who's trying to help out your cause, treating them as an enemy rather than someone who may have some insight you lack, it might be helpful to keep a more open mind. Perhaps if those with knowledge severely lacking in the mainstream discourse weren't rate limited, some here may find some new ideas to consider.

Even if they said how many models they looked at before choosing 14 models (and explained why they chose those 14), conspiracy theorists and people skepticalof climate science would still argue that it is not legitimate because of survivorship bias and several other reasons.

"You looked at 1000 models, but what about the 1000 models that scientists made and didn't publish. You didn't look at those!"

People can always suggest more tests you need to do or poke holes in any analysis. That doesn't mean you just stop ding the work.

Overall it is impressive that different models made over 30 years can predict fairly accurate results. In my opinion that shows that the underlying physics of the models is correct and makes me trust the models even more.

> Even if they said how many models they looked at before choosing 14 models (and explained why they chose those 14), conspiracy theorists and people skeptical of climate science would still argue that it is not legitimate because of survivorship bias and several other reasons.

Undoubtedly some would, but are you willing to consider the possibility that the number/percentage of people that would do so might be altered by the language used in articles on climate science?

> People can always suggest more tests you need to do or poke holes in any analysis.

Some analyses are easier to poke holes in than others. If this article was tightened up in the manner I describe, excluding the possibility of survivorship bias, then an entirely new approach to denial would be required. As written, survivorship bias is a very real possibility.

Wikipedia says: "Survivorship bias or survival bias is the logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility. This can lead to false conclusions in several different ways. It is a form of selection bias."

Survivorship bias is a real phenomenon, not a conspiracy theory. It is neither illogical or unscientific to consider the possibility that is might be in play behind the scenes here, and the article as written makes no attempt to rule it out.

> Overall it is impressive that different models made over 30 years can predict fairly accurate results. In my opinion that shows that the underlying physics of the models is correct and makes me trust the models even more.

I suspect this is because your mind does not consider it a possibility that the models discussed do not result from survivorship bias. A conspiratorial mind however, would most likely strongly consider that a possibility, and therefore compute a completely different level of trust.

Imagine for a moment how the human mind works when reading something. Typically, if the mind is primed with preexisting beliefs, it is going to be actively searching (interpreting) for confirmation of those beliefs, and this behavior will likely be stronger if the topic happens to be one associated with one's identity. So, your mind processes an article such as this optimistically (uncritically), whereas a "denier" or "conspiracy theorist" is going to read it pessimistically (critically).

https://en.wikipedia.org/wiki/Confirmation_bias#Biased_inter...

https://www.verywellmind.com/what-is-a-confirmation-bias-279...

Here is another interesting example. See this reddit thread on the article: https://www.reddit.com/r/science/comments/e63ic5/of_17_clima...

The top comment is: (/u/aClimateScientist) "Hi all, I'm a coauthor of this paper (Henri Drake) and happy to answer questions."

Someone asks the question: "How did you account for survivorship bias of models? IOW how did you select the 17? I mean if I asked 1,000 psychics to predict GDP growth, I could probably find 10-20 who were pretty close and say that that proves they're clairvoyant."

As of when I'm writing this, neither /u/aClimateScientist nor /u/avogadros_number (the article submitter, who seems to have considerable background in climate science) have addressed this question.

(Or, same thread: a moderator [Removed] comment.)

My question to you is: do you think it is possible that the manner in which your mind evaluates this particular situation (an unanswered question, in a thread with 29 total comments) may differ from how the mind of a "denier" or "conspiracy theorist" might evaluate the situation, and how that evaluation may in turn be used (or not) to tune your respective mental models and subconscious heuristics?

A followup question is: do you believe this perspective is irrelevant (has no bearing on) the climate change dilemma?

------------------------------

I sometimes wonder if the climate change dilemma was reproduced in a sim-style video game, would the human mind be less resistant to recognizing and reacting to the behavior of their adversary? If the adversary is a machine rather than a human, might the tribalism component of subconscious decision making decrease? This is one of those things that seems so obvious, I wonder if there might be some studies on it.

As shocked as people here are at the continued existence of "deniers" (or more accurately, their mental models of deniers), I am similarly shocked at the psychological unwillingness of intelligent people to acknowledge certain unfortunate realities. It's kind of comically/tragically ironic if you think about it.

But we don't know how those 17 were chosen in the first place. There would be survivorship bias if models which match observations get more attention.

With that being said, I doubt that survivorship bias is strong enough to explain their combined result of 10/17 models being significant.

Since one of authors posted the paper, we do know:

> “We conducted a literature search to identify papers published prior to the early-1990s that include climate model outputs containing both a time-series of projected future GMST (with a minimum of two points in time) and future forcings (including both a publication date and future projected atmospheric CO2 concentrations, at a minimum). Eleven papers with fourteen distinct projections were identified that fit these criteria.

> Eleven papers with fourteen distinct projections were identified that fit these criteria.

This brings to mind the old joke: "I used to smoke weed. I still do, but I used to, too."

Specifically, the phrase "were identified that fit these criteria" sets off my skepticism spider senses. Kind of like the popular "97% of scientists..." meme, that doesn't quite hold water when you dig under the covers.

To be clear, I'm not saying deceit is taking place, I am simply saying that I am skeptical.

I wasn't able to find any details on the specifics of the literature search, such that the search could be reproduced. Any idea if that was included in the study and I somehow overlooked it?

>Where is the story here though?

The motive here is the "trust climate scientists now because they've been right about everything all along" narrative. There's no other reason for it.

The motive could be “think twice before dismissing climate scientists because they have a pretty good track record.” Your interpretation seems to display the exact bias you are attributing to others.

The bias I'm suggesting is an appeal to authority, and you'll have to be more clear on how I'm displaying that, since I am neither suggesting I am an expert, nor am I showing off my long history of "being right" about other things.

It's not an appeal to authority when they back it up with data and numerical analysis.

You can back up study X with all the data and numerical analysis in the world, and it still doesn't increase study Y being right, except through appeal to authority.

So, you dismiss this work based on your speculation on what the authors could in principle have done wrong?

There would actually be substance to your comment if you could point us to the models the authors didn't consider.

Indeed, the authors (I am one of them) would appreciate this. I'll personally add any extra papers you find to the plots. If it turns out we really left out some important papers, we (or at least I) will publish a revision to the paper.

I would very much like to hear your opinion on the perspective I am (unsuccessfully) trying to draw attention to via my comments in this thread. I believe this, and things like it, are far more important than people are willing to consider.

> they had no clue that China would become such a powerful force in terms of CO2 emissions?

Why would the models care? In fact world GDP growth has been right on trend since the 60's. It's true that it's concentrated in areas we didn't expect, but that's always true. China did much better than expected, the Americas were right on curve, Europe a little worse and eastern europe in particular fell off a cliff. So what?

Those models surely predicted aggregate carbon output pretty well. Are you saying they didn't?

They compared both temperature over time and temperature vs forcing. The latter test factors out the "quantity of emissions" which as you say are much harder to predict (geopolitical), so just measures the quality of the climate modelling.

while I generally agree with your comment, I don't think it is necessary you'd actually have to know about China, you just need to know the general growth trends and if emissions seem to correlate with it, whether that meant cocentrated manufacturing in China or distributed around the world, or even concentrated somewhere else seems like a "detail"

It's not even a detail. It's totally irrelevant.

> Anyone involved with forecasting will know what I mean.

I'm not involved with forecasting, but am generally very skeptical of people 'debunking' climate science in non-scientific forums such as these. Can you back up your claim that these people haven't done their job properly? Have you read the article and their methodology? It seems to be behind a paywall unfortunately so I can't assess your claim right now.

Since you didn't point out any specific flaw in the study, but mentioned merely general biases which any scientist is aware of, suggesting they apply to the article but not really pointing out how, I really can't take this criticism of a peer reviewed article seriously.

I assume I’m not the first making this point, but I think they’re evaluating all the models, not just the ones that they know did well.

Yeah, this is like the stories about the guy who correctly predicted the winner of the Presidential election decades in advance (Bloom County said Trump would win back in the 1980s was it), or did well on some stock investments that did well based on events they didn't actually predict.

According to: http://www.realclimate.org/index.php/archives/2019/12/how-go...

>We gathered all the climate models published between 1970 and the mid-2000s that gave projections of both future warming and future concentrations of CO2 and other climate forcings – from Manabe (1970) and Mitchell (1970) through to CMIP3 in IPCC 2007.

If you're a cynic or suspicious: "projections of future warming" means they intentionally excluded studies that projected cooling.

Otherwise: it could have been written as "projections of future temperature", but they came across no studies that projected cooling.

This really makes me wish for open access journals, as most of the questions in this thread could be answered if it was possible for anyone to look at the paper.

We didn't find any quantitative and peer-reviewed projections of cooling, or we would have included them. If you know of any, please let me know and I'll publish an update to the paper.

I don't know if it helps, but these are the papers the media cooling stories came from. They are peer reviewed and quantitative, but perhaps not exactly what you need (that is to say, not sufficiently predictive).

Rasool and Schneider 1971 [1] https://pdfs.semanticscholar.org/4db2/1045b17ebcdd6a8c6adeb1...

Earl Barrett 1971 https://www.sciencedirect.com/science/article/abs/pii/003809...

Hamilton and Seliga 1972 https://www.igsoc.org/journal/14/72/igs_journal_vol14_issue0...

Chylek and Coakley 1974 https://science.sciencemag.org/content/183/4120/75

Bryson and Dittberner 1976 https://journals.ametsoc.org/doi/pdf/10.1175/1520-0469%28197...

Sean Twomey 1977 https://journals.ametsoc.org/doi/pdf/10.1175/1520-0469%28197...

[1] edit: Schneider later retracted findings

P.S. Our coauthor at NASA was able to host the paper on their official site for anyone to read. https://pubs.giss.nasa.gov/abs/ha08910q.html

I share your wish for open access journals.

Reply on news.ycombinator.com