The Official Forecast of the U.S. Government Never Saw This Winter Coming | Hacker News Reader

The Official Forecast of the U.S. Government Never Saw This Winter Coming

businessweek.com

24 pointsShivetya54 comments

FWIW, my next-door neighbors, who were dairy farmers for many years, felt quite certain that it would be a very harsh winter this year. Their prediction was based on a few natural indicators: the fact that old apple trees along our road were bearing fruit for the first time in a while; and the coloring of a certain type of furry caterpillar (sadly, like a typical city transplant, I don't recall the details at the moment). There was also something about where birds were placing their nests.

My lovely neighbors are full of observations and predictions like this, sometimes preceded by references to the Farmer's Almanac... what one could call "country wisdom" with perhaps a smile on one's face... but it really seems like they are usually right about such matters. They were so convinced of the harsh 2013-2014 winter that they decided to leave for two months of it, which they haven't done in like 50 years or something--which turned out to be a great call. Of course it could be coincidence or random chance, and I probably don't hear about or remember the times when the predictions are wrong. But I do love that there are other constructs for measuring and predicting weather other than what we typically hear about from the weatherperson on the news.

Your neighbors should publish a paper, so we can fold these observations into our predictions, if they correlate with future weather while being uncorrelated with the other sources of information.

Or if it's really that accurate, sell the info to people trading orange juice and become stinkin' rich.

More likely, as you say, it's just confirmation bias. It'd be surprising if the relevant patterns were simple enough for caterpillar and tree biology to pick up on but too complex for our climatologists and meteorologists to spot in decades of PhD students looking for theses. It's not impossible, though.

> More likely, as you say, it's just confirmation bias. It'd be surprising if the relevant patterns were simple enough for caterpillar and tree biology to pick up on but too complex for our climatologists and meteorologists to spot in decades of PhD students looking for theses. It's not impossible, though.

Right. Most caterpillars and trees I know have only had their PhDs for 5 years or so.

Well, I think it's unlikely, but not as unlikely as your quip (while amusing!) would imply. The computations would be performed on fuzzy caterpillar and apple tree hardware, but the correlations would have been discovered and tuned by natural selection over many iterations.

if there's anything this crowd can get behind, it's fuzzy logic and apple hardware.

Very nice! I had considered going the fuzzy logic route, but missed the other.

Well, something that anemometers, thermometers, hygrometers and pyrometers can't tell you is what the air smells like.

If the animals that reacted differently, for example, to air with a different blend of gasses, and that different behaviour caused that animal to survive the surprise snow storm, you'd expect that this behaviour would become more widespread in later populations of that animal.

The genetic "memory" of the cold event would be carried by the surviving population, since only animals with behaviours that contribute to surviving the cold event end up procreating.

Now what if caterpillars change colour in response to atmospheric gasses, temperature, or diet? Would a caterpillar that changes colour with temperature be a suitable indicator of a coming cold snap?

http://mamajoules.blogspot.com.au/2008/10/caterpillars-chang...

So I recommend everyone start a macrophotography habit, keep photographing the insect life in your neighbourhood with geotagging turned on, upload to Flickr, and then the climate scientists of the world will have something to work on.

That's roughly how it would happen if it happens. I think it's unlikely that they predicted that this winter would be especially cold because I think the causes of this winter being especially cold are unlikely to be local. A caterpillar in Georgia could respond to the temperature, pressure, atmospheric mix, humidity, or whatever in Georgia; it cannot respond to, say, pressure differentials between Georgia and New York.

Your lovely neighbours cannot be correct except by chance; if there was any validity to their predictions, it would imply not just that they knew the future of the weather, but that the future of the weather was knowable, which it certainly is not. Your tendency to see their predictions as accurate is likely a conflation of your confirmation bias and your choice-supportive bias.

Humans naturally search for meaning in chaos. That is all that their observations of caterpillars and trees and birds amounts to. They are trying to attach significance to a chaotic universe in order to reassure themselves that their place in the world makes sense. It does not. This is mathematically verifiable.

If the birds and the caterpillars and the apple trees could know the future of chaotic systems, it would imply that we live in a deterministic universe where birds and caterpillars and apple trees could not exist. QED.

"If the birds and the caterpillars and the apple trees could know the future of chaotic systems, it would imply that we live in a deterministic universe where birds and caterpillars and apple trees could not exist. QED."

Wut?

Complex dynamics 101: life is a complex dynamical system. If the math of chaos is wrong, the processes underlying life would not operate as they do. Life would not exist as we observe it.

Also, your iPad would not work. As I said, QED. This is not controversial.

For something so "not controversial" that it doesn't require citations, your comment is pretty faded.

The Mandelbrot set is both chaotic and deterministic. QED

> Also, your iPad would not work.

What?

Or, perhaps that weather has cycles and is not an uncorrelated random walk, and that 50 years of living with your life strongly influenced by the weather, along with several lifetimes of your parents' wisdom gives some insight into deep patterns of nature that are not readily apparent. Perhaps these persistent stories about old people predicting things based on caterpillars and trick knees are hints that the world might be more complicated than you think. And if you aren't so quick to assume that you understand the world because you think you understand a theory, you might investigate these hints. Maybe you will find out that there is something actually going on, and you'll have an exciting new world to explore.

"Chaotic system" does not mean what you think it means.

"it would imply that we live in a deterministic universe..."

You say this as though it's impossible. You may not believe we live in a deterministic universe, but, that's just, like, your opinion, man. The rational stance (I think) is to: (1) admit you don't know, (2) acknowledge the possibility of a deterministic universe, (3) for whatever chance the universe is deterministic, delude yourself into believing that it's not, because the illusion of free will is probably good for your mental health.

I think you're doing #3 really well, but skipped over #1 and #2.

Just because a system is chaotic doesn't mean nothing about it can be predicted.

Reading the comments is a really depressing exercise.

Do people really know the difference between weather and climate?

I'm increasingly less impressed by the claim that we can't predict weather across weeks, but we can somehow predict climate across decades. (I've never been overwhelmed by it, but it seems even less sensible over time.) And part of the reason why is that we can't seem to predict climate across years, and I'm really unconvinced we can't predict climate across years but we can predict it across decades.

Wrong predictions are wrong predictions; it is irrational to keep excusing wrong predictions over and over again and insisting they'll just get more right in the unknowable future.

If the models can't look into the future even a few months and see a high probability of the vortex detaching (and I'm fine with a probabilistic prediction, in fact I'm deeply suspicious of anything else), what use are they for predicting anything else? One wonders if the models are even capable of expressing that prediction (in the machine learning sense of the word "bias").

We've gotten much better at predicting weather a few days out[0], because not only has our understanding of the Earth's climate improved by leaps and bounds, but we've also had the chance to iterate thousands of times over the last couple decades to see how well our models perform and adjust them based on that evidence.

On the other hand, we've only had the chance to iterate a few times over the last couple decades in predicting longer-term trends several years out, and even though our understanding has increased here as well, we don't have the benefit of very many results to confirm if that understanding is anywhere near enough to predict well over these sorts of time spans. Apparently arctic sea ice has decreased faster than predicted, but average global temperatures are falling out of the "95%" ranges of most of the models of a decade or two ago[1]

[0]http://www.nytimes.com/2012/09/09/magazine/the-weatherman-is... [1]http://www.economist.com/news/science-and-technology/2157446...

Predictions aren't magic. They're based on looking at past data and using that to draw conclusions about future data.

Weather, being extremely variable even within a day, is a lot harder to predict than, say, "It will be colder in January than in August (in the northern hemisphere)." That one's a slam dunk.

Watching the warming trend, it seems more predictable than weekly weather. Even now, with this winder, the north pole is warmer than normal which is part of what's causing the weather systems we're experiencing in North America (as you referenced with the vortex).

Why on earth would the models have something like this vortex detaching built into them? We've never seen it before, or seen it so rarely that nobody built it into the model. That doesn't say we can't predict climate, it just says we didn't predict this particular consequence of climate change.

The idea is that there's a mostly random high frequency medium amplitude component (weather), a somewhat predictable medium frequency high amplitude component (seasons), and a low frequency low amplitude component (climate).

So if you observe for long enough, the higher frequency parts mostly average out to zero and you can get an idea of the low frequency component. Even without being able to predict the other components.

That predicting behavior at larger scales is easier than predicting it at small scales is a principle you'll encounter repeatedly in the physical science and engineering disciplines. In the physical sciences, models are often top-down, as opposed to in computing where they are almost always bottom-up. In fluid dynamics, for example, the behavior air in a turbulent flow is chaotic at the small scale. Yet, we can still design airplane wings because we have pretty good approximate models for how such flows behave at larger scales. Or to use a physics example: it's easy to predict where a baseball will land, but very complicated to predict where an electron will be in a molecule.

The analogy I like to think of is a ping-pong ball in the ocean. At any given second, its location will be hard to predict, as at the second-granularity, its location will be dominated by local phenomenon. But, over the course of months and years, we can often accurately predict where it will generally end up in the world due to ocean currents.

I, too, am a bit skeptical, but this is boiling climate science down into too simple an exercise. We do, for instance, know that things are warming up, and we have many different models to predict what effect that will have on, say, weather trends over a longish period of time. It's kind of like the stock market -- I might think that the fundamentals of company ABC are good, and believe that they will be up over the next five years. Some people do make reasonably good predictions of this sort. Predicting the open and close of ABC's stock tomorrow, however, or the high and low over the next three weeks, is another story, and there's reason to believe that we do no better than random chance at this kind of prediction, in aggregate.

See NASA scientist, "the father of climate change," James Hansen's 2009 book entitled Storms of My Grandchildren. He's highly critical of models, though notes that they may be useful in certain specific scenarios. What he focuses more on is the geological record because models can only be as accurate as the information that goes into them.

I want to keep this brief, but I also want to quickly note that the cap and trade for greenhouse gases is business as usual for energy producers. Don't believe a word of it; the short story is that it worked poorly in preventing acid rain and won't work for combating climate change. We need a fee and dividend system that doesn't come back to us in tax breaks. I should note that Hansen comes across as very credible and apolitical; I don't trust anyone else when it comes to climate science.

His most recent work is available here:

http://www.columbia.edu/~jeh1/publications.shtml

It is harder to predict very unusual events.

Also, I am not sure what difference it makes? We want to predict climate to inform environmental policy. But a lack of certainty does negate the damage that pollution does.

There is an enormous difference in localized weather predictions, and whole globe climate predictions. That is, it is possible to predict, say, average temperature across the entire world, while being unable to reasonably predict average temperature in any one place.

It seems to me that people are misunderstanding looking backwards and looking forwards...

Well maybe if CERTAIN PEOPLE IN THE MEDIA (mostly) and in POLITICS (sometimes) didn't trot out lines to the effect of "Oh, it's summer! and it's hot today! This must be PROOF of global warming!" every summer when it got hot, then we wouldn't get this kind of blowback when it's unexpectedly cold out.

There is a problem in scientific reporting.

Scientists want to be honest and truthful. People interviewing scientists don't have much scientific understanding, and don't have time to listen to a full answer. And they'll often then turn to someone else providing an opposing viewpoint for "balance". This new person doesn't care about scientific integrity and they're happy to spew their purely opinion based opinion. (And this is ignoring the people paid to provide FUD who ised to work for the tobacco companies and who know use similar tactics to deny clomate change).

Considering I still hear "Global warming is fake; this is the coldest winter I can remember" on a near-constant basis, I'd have to say the common layperson knows very little.

I feel the same way when I hear "global warming is real, this is the hottest summer I can remember" from the other side of the argument.

I'd trust a layperson as much as a scientist. Which is to say, not at all.

Global Warming seems to be a political distraction, driven by some businesses looking to get special favors (carbon trading).

Let's just ignore the problems we face now, like pollution and resource depletion, dubious food quality, inhumane factory farming...

People will always seek to make money from a perceived opportunity. Whether the opportunity is the climate changing or people mistakenly afraid that the climate is changing does not in any way change whether or not the climate is actually changing.

In other words, an increase in umbrella salesmen does not any any way prevent the sky from falling.

Frankly, I don't know if global warming is true. Having examined some of the claims and by using my own intuition, I think it probably is. But its validity shouldn't matter to you if you truly care about the problems you listed.

For example, let's look at two of the other problems you listed that we are supposedly currently ignoring: pollution and resource depletion. Both of those problems are usually tied to the burning of fossil fuels, which just so happens to be the suggested cause of global warming. The suggested fix of global warming is typically burning less fossil fuels, which would in turn reduce pollution and lower our resource consumption. If you truly feel that those problems are what we should be focusing on, you should encourage people to believe global warming is a problem that needs fixed as it would further your goals.

Personally, I don't care if somebody makes a mint off solar power. I'd rather have money go that direction than the oil conglomerate. Even if both groups were equally "evil", I'd still vote for solar because I believe it's better for the ecosystem.

>you should encourage people to believe global warming is a problem that needs fixed as it would further your goals.<

Not sure if you missed a /sarc on that? Anyway, I think the root cause is clearly 9 billion humans on the planet, thus I got myself 'fixed' after having 2 kids (2.1 is the replacement rate), but I'm not going to lie to people about some warming to make my point... in fact, that would probably distract yet again from seeing the underlying problem.

It's quite depressing that people can't see three basic things:

1) exponential population growth is unsustainable (e.g. more than 2.1 kids)

2) resources per capita goes up when capita goes down (a good thing)

3) make a better world (technology, etc.) before increasing the population size... don't make it your kids' problem.

What has to do a climate scientist with "some businesses looking to get special favors (carbon trading)"

Are you saying that they are part of a conspiracy?

After spending a good deal of time reading about government funded nutritional 'science' over the last 50 years, in which high-carb hysteria took over (making many of us much less healthy, but making certain large businesses much richer, edit: and making food price inflation appear low), it's quite easy to make a case that research institutions dependent on government grants will absolutely not want to upset the status quo, and junk science is the result.

Let's also look at how many economists said "nobody could have seen the debt crisis coming". Clearly, skepticism that would harm the elites is not welcome in respectable academic institutions.

Let's say a democratic government does 99% of what rich or connected people want it to do. (Perhaps that's marginally better than a dictatorship) But the government wants to prove 'scientifically' it's doing the right thing. So it funds nutritional scientists and economists (and I will assume climate scientists) that see the 'right things' and give the policies legitimacy.

You say conspiracy, but the supreme court would disagree - this is just 'free speech' - corporations are people too! I'm sure you've heard "It is difficult to get a man to understand something when his salary depends upon his not understanding it."

No need to be deliberately obtuse--there are many people in industry and government who stand to benefit from carbon trading. There are also many scientists who research climate change. The two groups do not have to be intimately connected for them to benefit from each other.

Were all the loan originators at Countrywide part of a conspiracy to sell bad loans, or were they just "following the money"? Why would grant-seeking scientists be immune to the same incentives?

The OP has said that he doesn't trust scientists.

So, is he saying that scientists are part of a conspiracy.

And before calling others obtuses, read what the OP has said.

> Do people really know the difference between weather and climate?

In the UK, people worry about the climate and moan about the weather.

So are polar vortex dips now going to become a regular thing?

Because I could see rock salt becoming a valuable commodity (ironically returning us to roman times?)

What is all that salt and brine doing to wildlife and water tables?

I liked Nate Silver's chapter on climate modeling predictions in his book The Signal and the Noise. He wasn't impressed. Maybe somebody needs to start a fantasy weather league.

The hilarious ending of course would be the fantasy weather league correctly predicting the weather.

That scroll is obnoxious. You shouldn't suddenly jar the text position out from under the reader's eyes.

That is because the weather is a chaotic system. It is governed by the mathematics of complex dynamics.

Predictions diminish in accuracy with increasing time because the number of inputs rapidly approaches infinity, and each one of them can turn out to be determinative of the overall behaviour of the system.

It should be no surprise to any scientifically literate person that a weather forecast issued in October has no validity by January. Any resemblance to reality is purely accounted for by chance.

This is not amenable to the Mr. Fix-It mentality that inevitably shows up in these discussions. There is good reason to believe that weather prediction will never improve, no matter what the scientific and technological developments, because the future of a chaotic system is mathematically intractable and genuinely unknowable.

The same argument applies to climate science. I'm sorry to bring you the very bad news that we live in a chaotic universe, not a deterministic one. Get over it. Wear a helmet.

Edit: The mysterious part is that the scientists at the so-called Climate Prediction Center would issue a 3-month forecast. What part of complex dynamics did they not understand? Sometimes an expert is just a guy with slides.

Prediction centers might not be able to tell the exact date and place of a storm that far out, but they can at least tell if a season is going to be bad well ahead of time.

Edit: It says this right in the article: climate forecasters focus on things that change more slowly, such as temperatures of the land and oceans... they try to say whether a given three-month period will be wetter, drier, hotter, or colder than average.

I'm sorry, what part of complex dynamics did you not understand?

It doesn't matter how slowly things change. It doesn't matter how big the inputs are. Complex dynamics teaches us that scale does not matter; very small inputs can and do end up being more important than very large inputs.

I'll say it again: scale does not matter.

No, they cannot tell if a season is going bad well ahead of time. It is a fallacy to think that they can. They can tell that a season is going bad exactly at the moment the season starts to go bad.

Why is it that every time I raise the issue of the unknowable future of chaotic systems that people down-vote me? Is it so hard for you to accept? Check the math and the science: it is sound. I am sorry you do not like it, but the mathematics of chaos is not some fringe idea. It underlies everything. I'm astonished that so few people have noticed this.

Chaos theory does not mean that literally anything can happen. Weather changes are still bounded by energy inputs and outputs. It can't rain if the humidity in the air is too low. The jetstream isn't just going to reverse course anytime soon, although it is shifting slowly. It's physically impossible to have snow if the weather is too warm - complex dynamics can't change that. I'm not saying a forecast will never be wrong, but not every case is an edge case. With good data and a useful model, you can be right most of the time.

You are taking an extreme stance on the topic. There's room for modeling and forecasting even in highly complex dynamical systems, the models are getting better with advances in applied statistics research. They are not perfect, but they are not completely useless either.

For example, here's an article I just found on modeling stochastic nonlinear dynamics in ecological/oceanographic applications: http://arxiv.org/pdf/1211.1717.pdf

As for caterpillars predicting climate patterns, I agree with you that it's unlikely, unless there are some simple environmental indicators the caterpillars learnt/evolved to take into account but we humans haven't paid attention to yet. This doesn't mean the changes are unknowable.

> they can at least tell if a season is going to be bad well ahead of time.

Proof needed.

e.g., the UK Met office predicted this would be a dry winter...

Every season the Heidke skill score is high, they did well. Here's an example http://www.cpc.ncep.noaa.gov/products/precip/CWlink/ghazards...

A ridiculous measure, because it depends upon the definition of "correct", which is an arbitrary reduction of continuous data to a single axis, and because the scoring is conducted by those who have a vested interest in the outcome, and that's just for starters.

How about applying some statistics, like the probability that a correct forecast (whatever that is) could be accounted for by chance, given the population of forecasts and conditions in which it takes place? P-values have their limitations, but that's no reason to discard them entirely.

Reply on news.ycombinator.com