Causality in machine learning

I know that Petersen and van der Laan are well-respected researchers in causality that know exactly what they are doing. E.g., Petersen has a course (I think at Berkeley) that uses causal graphs which were pioneered by Judea Pearl (see comments below). I can only second the recommendation to dig into his work.

Lets return to your equation. r^2 = m1*m2/F is a statement of correlation that's valid to many digits, that also says that "these are all the variables that affect anything here." It allows for calculations of quantities by plugging in numbers. In combination with other equations, it allows extrapolitaion and physical thinking. But equations alone are not enough for a physicist to make progress on problems, they don't combine arbitrarily, they still use that famous "physical intuition" that turns out to involve a heck of a lot of causality that they talk about in the cafeteria but don't often write about.

In the world outside of physics classrooms, most situations involve a huge number of variables interacting with each other, and the variables themselves are leaky abstractions. This means that the correlation of quantities does not go out to many digits, or even one digit. This does not make an equation a useful abstraction. People have developed probabilistic models that stand in for equations in these situations. Sure, it would nice to have an equation that modeled things like they do in the simple situations that Physics publishes on, but when the equations of physics are so weak that they can't even fold a protein, how are we going to use them to model the interaction of a potential drug with a protein?

Equations are great, when they apply. By introducing simplifying assumptions, we can sometimes extend their utility into new domains, but at the cost of introducing fragility because of those simplifying assumptions. They allow for calculating new things with only a very few computations. But nearly every new field of science invents new math. One of the more recent inventions is causality. It works fantastic in those sciences that use experimental interventions. Simple algebraic equations do not.

Someone is new to the idea of relativity. This is nonsense. The frame of reference describing the sun rotating around the earth is not inertial (https://en.wikipedia.org/wiki/Inertial_frame_of_reference) and thus does not follow the laws of physics we know and love. In short, "Nope, you cannot say the sun orbits the earth."

To be clear: Pearl is not just philosophizing. He provides a formalization that takes causality from being an ambiguous concept to being applied mathematics. The bulk of the book is not a light read, but the mathematics involved are relatively basic.

And to be absolutely clear: Pearl won the Turing prize for his work. It is not simply handwaving or non sequiturs. The common online conversations about causality are several decades out of date with real academic work on the topic.

For example in this review: https://arxiv.org/pdf/1605.01138.pdf, a model-based approach is compared against a deep learning approach. It is very interesting to me that even if deep learning performs better, its failure cases are very different than human failure cases (deep learning is not sensitive to "physics illusions"). Whereas the model-based approach is very much in-line with human performance characteristics. Also, on this task (physical scene understanding), deep learning generalises very poorly.

You appear to be saying that causality is not quantitative, however that is not true.

But my point was to show that causality is important, which the OP appears to doubt altogether.

P(Dog barks | do(kick the dog) )

The `do` modifies your model to treat "kick the dog" as observed, breaking its dependence on any other variables, e.g. the doorbell ringing or the cat hissing. With those links broken, the model becomes simpler, and often the causal question can be answered using observational rather than experimental quantities.

At least, that's the idea. Not sure how broadly accepted it is, but Pearl is a towering figure and his work seemed robust at least to my not-so-expert eyes.

Hidden context, uncontrolled experimental variables, pleotropic effects, experimental noise all conspire to produce conclusions that cannot be reproduced or if reproduced, cannot be generalised. The contrast with physics is stark - theoretical physicists predict experimental results (eg Higgs boson) many years before the experiment is performed. I don't think that in biology, we merely need to prove a succession of hypotheses to achieve the kind of explanatory causal models we want. A theoretical framework is needed, and at the moment, there is not even an outline of such a framework.

It's possible to detect the direction of causality with observation with some additional assumptions that are reasonable general.

For example Additive Noise Models (ANM) assume that there is an additive noise structure in observational distribution. The key assumption is that if X causes Y, the noise in X can have an effect on Y but not vice versa.

Additive noise model is near 80 per cent accurate in correctly determining cause-and-effect across large number of datasets.

---

Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks Joris M. Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, Bernhard Schölkopf; 17(32):1−102, 2016. http://jmlr.org/papers/v17/14-518.html

Center for Causal Discovery web site http://www.ccd.pitt.edu/