Hacker News Reader

Comment by cgore | Hacker News Reader

Correlation is not causation but it's rarely accidental.

Correlation is usually accidental.

e.g. http://www.tylervigen.com/spurious-correlations

Saying that correlation is usually accidental is a stretch. Basing that claim on that (admittedly amusing) website is even more of a stretch: the time series on that website are all highly autoregressive with minuscule sample sizes, which makes spurious correlations extremely likely [1].

Speaking (very) roughly, the fact that they are autoregressive constrains how 'kinky' the shapes can be. Visually speaking, the time series will have a small number of inflection points and will appear interpolated between these points.

This greatly reduces the search space: you just need to line up a couple of kinks in a space of ten samples, or match two smooth and vaguely line-shaped objects, rather than find a convincing relationship for 100s of observations not tied together in time.

[1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.611...

I've got some homeopathic remedies to sell you. People took them, and they got better!

I think if you measure random unrelated data sets against each other you're quite likely to find spurious correlations. If you design an experiment between potentially related variables with a potential hypothesis on how they are related and then observe data that matches the hypothesis, you probably have a reasonable chance the hypothesis is actually the explanation.

That is the point of experiments.

I see your blog post and raise you a youtube video.

minutephysics Correlation CAN Imply Causation! | Statistics Misconceptions

https://www.youtube.com/watch?v=HUti6vGctQM

Make a list of all the things that have changed over the last century.

My height has increased over the last century.

How many things on that first list are caused by my increasing height? How many caused my increasing height?

Things that have increased over the last century:

- Calories available

- Vaccination rates

- Access to neo-natal care

- Knowledge about fetus and infant development

Correlation is insufficient to prove causation but in many cases it's a great hint.

So as calories available increases, it causes my height to increase?

I'm gonna need a new wardrobe...

You've confused an individual trend with a trend of the aggregate maximum value.

A couple of those may have a minor influence on the limit of my height, but increasing those factors has no effect on my height.

Or how about my age?

And don't forget other highly correlated values, like the number of movies or books published, Chinese population, cumulative deaths in war, and number of artificial satellites. There's a correlation of my age (and height) with each of those.

Just two temporal trends that move in the same direction.

No causation, though.

Reply on news.ycombinator.com