Seven basic rules for causal inference
pedermisager.org
pedermisager.org
This is a big one that most people are not aware of. Quite often, in economics, medicine, and epidemiology, you'll see researchers adjust for everything in their regression model: income, physical activity, education, alcohol consumption, BMI, ... without realizing that they could easily be inducing collider bias.
A much better, but rare, approach is to sit down with some subject matter experts and draft up a DAG - directed acyclic graph - that makes your assumptions about the causal structure of the problem explicit. Then determine what needs to be adjusted for in order to get a causal estimate of the effect. When you're explicit about your causal assumptions, it makes it easier for other researchers to propose different causal structures, and see if your results still hold up under alternative causal structures.
The DAGitty tool [1] has some cool examples.
They offhandedly say that "correlation" means "association" or "mutual information", so why not just do the whole post in terms of mutual information? I think the main issue with that is just that some of these points become tautologies -- e.g. the first point, "independent variables have zero mutual information" ends up being just one implication of the definition of mutual information.
Judea Pearl’s book also goes into the above in some detail, as to why faithfulness might be a reasonable assumption.
They have clear dependence; if you imagine fixing ("conditioning") x at a particular value and looking at the distribution of y at that value, it's different from the overall distribution of y (and vice versa). But the familiar linear correlation coefficient wouldn't indicate anything about this relationship.
https://mixtape.scunning.com/01-introduction#do-not-confuse-...
X ~ Unif(-1,1)
Y = X^2
In this case X and Y have a correlation of 0.
Isn't it possible to contrive an example where a test of pairwise dependence causes the statistician to error by excluding relevant variables from tests of more complex relations?
Trying to remember which of these factor both P(A|B) and P(B|A) into the test
I think in this context, the issue with the spurious correlations from that site is that they're all time series for overlapping periods. Of course, the people who collected these understood that time was an important causal factor in all these phenomena. In the graphical language of this post:
T --> X_i
T --> X_j
Since T is a common cause to both, we should expect to see a mutual information between X_i, X_j. In the paradigm here, we could try to control for T and see if a relationship persists (i.e. perhaps in the same month, collect observations for X_i, X_j in each of a large number of locales), and get a signal on whether some the shared dependence on time is the only link.
How are effect lag and lead expressed in said notation for expressing causal charts?
Should we always assume that t is a monotonically-increasing series, or is it just how we typically sample observations? Can traditional causal inference describe time crystals?
What is the quantum logical statistical analog of mutual information?
Are there pathological cases where mutual information and quantum information will not discover a relationship?
Does Quantum Mutual Information account for Quantum Discord if it only uses von Neumann definition of entropy?
- controlling for a node increases correlation among pairs where both are ancestors
- controlling for a node does not affect (the lack of) correlation among pairs where at least one is categorically unrelated (shares no ancestry with that node)
- controlling for a node decreases correlation among pairs where both are related but at least one is not an ancestor
This is a very innocent sounding assumption but it’s actually quite strong. In particular it may be violated when there are control systems or strategic agents as part of the system you want to study — which is often the case for causal inference. In such scenarios (eg the famous thermostat example) you could have strong causal links which are invisible in the data.
I don't like showing the scatterplots in these examples, as "correlation" I think is more associated with the correlation coefficient than the more generic independence that the author means in this scenario. E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent.
Ok this is correct, but has nothing to do with causality. Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all. And in fact that's how the subject is taught at an introductory level basically 100% of the times.
It does. Dependence and independence have a lot to do with causation, as the article explains.
> Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all.
Yes, but this is irrelevant. It's like saying "whether or not someone is married is a conversation that can be had without resorting to the concept of a bachelor at all".
You can talk about (in)dependence without talking about causation, but you can't talk in detail about causation without talking about (in)dependence.
> NB: Correlated does not mean linearly correlated
> For simplicity, I have used linear correlations in all the example R code. In real life, however, the pattern of correlation/association/mutual information we should expect depends entirely on the functional form of the causal relationships involved.
there are various schemes for estimating mutual information from samples. if you do that and mutual information is very close to zero, then I guess you can claim the two rvs are independent. But these estimators are pretty noisy and also often computationally frustrating (the ones I'm familiar with require doing a bunch of nearest-neighbor search between all the points).
I agree with the OP that it's better to say "non-independence" and avoid confusion, at the same time, I disagree that linear correlation is actually the standard definition. In many fields, especially those where nobody ever expects linear relationships, it is not and everybody uses "correlated" to mean "not independent".
Fine, but...
> And if you are not familiar with what everyone else means by correlation, you're very unlikely to be an expert.
Perhaps, but this is not relevant. If there's a problem with this work, then that problem can be criticized directly. There is no need, and it is not useful, to infer "expertise" by indirect means.
Your point is great in that it pinpoints also the notions of agency scopes. In all the causal DAGs it feels like there are implicit regions: ones where we can influence or not, intervene or not, observe or not, where one is responsible for or not.
An intelligent agent is one capable of modelling a system, influence it, and bias it. Such that it can reach and exploit an existing corner case of it. I talk about a corner case because of entropy and murphy's law. For a given energy, there are way many more unadvantageous states than advantageous one. And the intelligence of a system is the complexity required to wield the entropy reduction of an energy source.
(paywalled link, but it's available on a well-known useful website)
https://stats.stackexchange.com/questions/85363/simple-examp...
Blew my mind the first time I saw it.
Not the same definitions one to one (author specifically talks about correlation vs linear correlation) but same idea.
But it's important to remember that dependent variables can also be not correlated. That is no correlation does not imply independence.
Consider this trivial case:
X ~ Uniform(-1,1)
Y = X^2
Cor(X,Y) = 0
Despite the fact that Y's value is absolutely determined by the value of X.
Start it in the terminal by typing:
R
first google hit if you want to try this yourself: https://www.freecodecamp.org/news/turning-vim-into-an-r-ide-...
Sooooooo much better than "notebooks". Hating on "notebooks" today.
I had to learn R for a statistics course. This was a long time ago. But coming from a programming background I never found any other mainstream language as hard to grok as R.
Has this become better? Is it just me that doesn't get it?
These are concepts i find myself constantly having to explain and teach and they are critical to problem solving.
Heckman, JJ and Pinto, R. (2024): “Econometric causality: The central role of thought experiments”, Journal of Econometrics, v.243, n.1-2.
From the paper:
> Each of the recent approaches holds value for limited classes of problems. [...] The danger lies in the sole reliance on these tools, which eliminates serious consideration of important policy and interpretation questions. We highlight the flexibility and adaptability of the econometric approach to causality, contrasting it with the limitations of other causal frameworks.
perhaps this is a quaint or wildly off base question, but an honest one, please forgive any ignorance:
Isn't this essentiallydefining the partial derivative? Should one arrive at the calculus definition of a partial derivative by following this?
The technical meaning of that sentence has to do with probability theory (probability distributions, correlation, conditionals), and not so much calculus (differentiable functions, limits, continuity).
0. The directions of all arrows not part of a collider are statistically meaningless.
(bushy trees, of which there are very many compared with linear ones, would be an instance of Gwern's model* of confounds being [much] more common than causality?)
The thing that made be read into it was a quite interesting sentence from lesswrong, saying that actually the common idea that correlation does not imply causation is wrong. Now it's not wrong in the face-value sense, it's wrong in the sense that actually you can use correlations to learn something about causation, and there turns out to be a whole field of study here.
I was at university around the millennium.
That's a specific instance of a more general problem in the "logical fallacies", which is that most of them are written to be true in an absolutist, Aristotelian frame. It is true that if two things are correlated you can not therefore infer a rigidly 100% chance that there is a causative relationship there. And that's how Aristotelian logic works; everything is either True or False and if there is anything else it is as most "Indeterminate" and there is absolutely, positively, no in betweens or probabilities or anything else.
However, consider the canonical "logical fallacy":
1. A -> B.
2. B
3. Therefore, A.
Many of the "fallacies" can be rewritten to be useful probabilistically, and aren't quite as fallacious as their many internet devotees fancy.
It is certainly reasonable to be "suspicious" about correlations. There often is a "there" there. Of course, whether you can ever figure out what the "there" is is quite a different question; https://gwern.net/everything really gets in your way. (I also recommend https://gwern.net/causality ).
The upshot is basically 1. the glib dismissal that correlation != causation is, well, too glib and throws away too many things but 2. it is still true you still generally can't assume it either. The reality of the situation is exceedingly complicated.
By the way, there is also an alternative to causal graphs, namely "finite factored sets" by Scott Garrabrant. Probably more alternatives exist. Though I don't know more about (dis)advantages.
Lesswrong promulgated a seriously misleading view of Aristole as some fussy logician who never observed reality and was unaware of probability, chance, the unknown, and so on. It is entirely false. Aristotle repeats, again and again and again, that we can only seek the degree of certainty that is appropriate for a given subject matter. In the Ethics, perhaps his most-read work, he says this, or something like it, at least five times.
I mention this because your association of the words "absolutist" and "Aristotelian" suggests your comment may have been influenced by this.
ISTM that there are two entirely different discussions taking place here, not opposed to each other. "Aristotelian" logic tends to be more concerned with ontology -- measles causes spots, therefore if he has measles, then he will have spots. Whereas the question of probability is entirely epistemological -- we know he has spots, which may indicate he has measles, but given everything else we know about his history and situation this seems unlikely; let's investigate further. Both describe reality, and both are useful.
So the fallacies are entirely fallacious: I don't think your point gainsays this. But I agree that, to us, B may suggest A, and it is then that the question of probability comes into play.
Aquinas, who was obviously greatly influenced by Aristotle, makes a similar point somewhere IIRC (I think in SCG when he's explaining why the ontological argument for God's existence fails), so it's not as if this is a new discovery.
See also Euclid; the fact that his geometry turns out not to be The Geometry does not diminish what it means to have blazed that trail. And it took centuries for anyone to find an alternative; that's quite an accomplishment.
If I have a backhanded criticism hiding in my comment, it actually isn't pointed at Aristotle, but at the school system that may teach some super basic logic at some point and accidentally teaches people that's all logic is, in much the same way stats class accidentally teaches people that everything is uniformly randomly distributed (because it makes the homework problems easier, which is legitimately true, but does reduce the education's value in the real world), leaving people fairly vulnerable to the lists of fallacies they may find on the internet and unequipped to realize that they only apply in certain ways, in certain cases. I don't know that I've ever seen such a list where they point out that they have some validity in a probabilistic sense. There's also the fallacies that are just plain fallacious even so, but I don't generally see them segmented off or anything.
Pedantry: s/centuries/millennia/ (roughly 21 of the former, 2 of the latter?)
EDIT: does anyone remember the quote about problems patiently waiting for our understanding to improve?
Judea Pearl is in some respects a little grandiose, but I think he is right to be express shock that it took almost a century to develop to this point, given how long the basic tools of probability and statistics have been fairly mature.
# variable A has three causes: C1,C2,C3
C1 <- rnorm(100)
C2 <- rnorm(100)
C3 <- rnorm(100)
A <- ifelse(C1 + C2 + C3 > 1, 1, 0)
cor(A, C1)
cor(A, C2)
cor(A, C3)
# If we set the values of A ourselves...
A <- sample(c(1,0), 100, replace=TRUE)
# then A no longer has correlation with its natural causes
cor(A, C1)
cor(A, C2)
cor(A, C3)