Correlation does not even imply correlation
andrewgelman.com
andrewgelman.com
Obviously, the statistics I present in the book and on the website are tongue-in-cheek, but I like to take issue with that line, because typically, people don't fully understand what it means. I use a variation of the line in the OP's post -- that "nothing other than correlation implies causation."
Really, correlation is just one part of a package of other things which all together imply causation.
http://www.amazon.com/Causality-Reasoning-Inference-Judea-Pe...
note: I might actually really confusing my word definitions here, btw.
What I meant to say before is that correlation cannot imply causation in and of itself because it is merely an observation—you must also build a framework of causal assumptions within which your observed correlation could have only arisen if there was a causal relationship of interest. That final statement is the "implication" we're all talking about.
So, "correlation along with an acceptable and proper set of counterfactual assumptions is capable of imply causation sometimes". Just rolls off the tongue, really.
[I'm not sure whether this is actually statistically true, but you get the idea, and I'm sure there's a similar example which actually is true.]
http://en.wikipedia.org/wiki/Correlation_and_dependence
Ie unemployment is not correlated with age since both young and old people tend to be unemployed...it is a non-linear relationship.
But you won't find it. Think about any causation. Now ask yourself why we now that it exists. The answer will always be correlation.
Another example: mendelian inheritance patterns in a pedigree study.
If you know of a good measure of non linear correlation please let me know. And publish a paper in science or nature like the MIC/MINE people did (a measure that has issues in practice).
In using those methods you may never calculate correlation as a number, but when those methods find something you still rely on the fact that there is a correlation.
The point of the article is that correlation (of observed data) does not logically imply correlation (of the underlying phenomena, usually in a more general setting than the data allows).
I'd go with A implies B means (not A) or B, ie binary implication. That'd seem to capture the spirit of "correlation does not imply causation" since we can have correlation (A) and not causing (not B), ie not((not A) or B), ie no implication. So I guess we can use it for this too as at least a possible definition.
If we say A is correlation in the sample set, B is correlation in the population, I think we can replicate the above argument and say we can have correlation in the sample set (A) and not correlation in the population (not B), ie not implication. An obvious example would be if we have 2 variables and plotting them gave a simple quadratic curve (say, y=x^2 across x=[-10,10]) - we can find a sample set with strong positive correlation and another with strong negative correlation - but neither really describes the population as a whole.
So being mathematical and using the boolean logic implication definition I think it's a valid statement - but defining implication in context is pretty important.
I don't agree in the general case. A counter-example is that a purely random data set will have correlations, which by definition imply absolutely nothing about anything other than the random data set.
This is addressed in the last sentence of the OP:
> That is, correlation in the data you happen to have (even if it happens to be “statistically significant”) does not necessarily imply correlation in the population of interest.
To create scientific knowledge you need to make a prediction (aka a hypothesis) and test it. To make an hypothesis you need some notion of a causal mechanism; to run the experiment you observe an interference in that mechanism to see what happens.
If all you have is a statistical correlation, and you haven't identified or altered a mechanism...you really don't have much. That is what this author is getting at.
The common meaning of correlation is close relationship embedding some causation in it (A implied B or B implied A or the same thing is the root for A and B).
You can find that two series have high correlation (statistically), whereas this happens just by chance.
OK.
Or maybe this is one of those clickbait titles that state something logically contradictory so that you'll get miffed and read the rest.
Also, apophenia. E.g. consider constellations. Stars are distributed randomly, but that doesn't stop us from making patterns out of them anyways.
To establish causation you need 3 things:
1) Correlation
2) Temporal precedence. That is, you have to show that the cause occurs in time before the effect
3) A lack of other plausible explanations
If more people knew the above the world would be a better place. Even the Udacity statistics course failed to mention the above, even though they hammered home that correlation does not imply causation.
Most troubling is point 3 -- a lack of other plausible explanations. There are many things, such as smoking cigarettes causing cancer, that have other plausible explanations. It is clear that Americans understand causality to be some form of influence, and not necessarily the only possible influence for an event. It is the degree of influence that matters.
Actually, to establish causation, in as much as this is possible at all [1], you need a predictive model and controlled experiments.
[1] following Hume, you can never definitively show causation.
Edit to add: because they are hidden.
As I linked elsewhere in this thread, if you really want to prove causality then you need to be making counterfactuality arguments. Judea Pearl's framework of causal networks is a good system for doing this.
At the end of the day, your claim of causality rests upon a network of assumptions about causality (and this is a formal description of your point (3), basically) and someone accepts this proof by accepting your assumption network and the strength of your new data.
Note: If I recall correctly, Google did show one such study.
>> 3) A lack of other plausible explanations
This means that there are not other parameters involved, satisfying the other two criteria, that can also plausibly "explain" the measured effect. In the example I gave, there may not be any such other available correlation.
That the analyst picked the plausible cause by hunting though a large number of others is by itself not a plausible explanation asked for by #3.
If we can assume that on average, a random set of people who are randomly asked about a political topic is a reflection of the population of a whole, we can start to draw meaningful conclusions about the political topic.
The OP article points to the concept of spurious correlation [1] which is a danger if you have very little domain expertise in the data that you're working with. e.g. If your regression shows US GDP is statistically significantly affected by Bangladesh butter production, you may want to discuss the results with domain experts about why the result may or may not be spurious.
1) every variable you calculate with follows the law of large numbers (also known as the central limit theorem) (this means amongst other things, that if you find correlation, you can't tell anyone involved in the variables, even yourself, because you'll act on it and change it, at which point the correlation won't hold. Also it won't work for chaotic variables, like pretty much anything involving human actions)
2) the correlation remains intact over longer time periods. The variables must be sampled without bias. You have properly separated your concepts, and made sure they actually represent what you want to be looking at. Etc. Etc. Etc. (this is essentially stating : don't fuck up the math before you calculate correlation) (obviously don't fuck up the math afterwards either, in other words : correctly check for statistical significance)
3) You have to check with every time offset, and even with variable time offsets. This is too complex to go into.
Then correlation implies a "causal relationship". Meaning corr(A, B) > 0 IFF
1) A causes B
2) B causes A
3) there is some external factor C that causes both A and B
Now keep in mind that there is no such thing as a root cause. There is simply a chain of events, and if the next event wouldn't have happened without the previous, the first event is said to be the cause of the second. "I shot him because his bees annoyed me", this "algorithm" would find the bees as guilty as the perpetrator, they are both causes of the death.
Also keep in mind that this only works AS LONG AS YOU LEAVE THE CAUSAL CHAIN ALONE. If you calculate correlation, and find, say "BAC" always goes up 2 days before "JPM" goes up, and then you proceed to buy JPM when BAC goes up, you've just invalidated your conclusion (translation : this works better the smaller an investor you are).
There is also absolutely no guarantee that killing of one chain of events won't simply lead to another. Say you have a harbor with 2 entrances, and because one is wider, every ship uses it. Therefore "getting to that harbor" correlates perfectly with "going through entrance A". Obviously blocking entrance A won't lead to no more ships in the harbor.
So this DOES NOT match the human/legal idea of "cause", and can never yield useful actions to take. It is nevertheless a useful metric.