Hacker News Reader

Comment by ghj | Hacker News Reader

As a layman, is causality worth studying?

For example, can I answer questions like what are the main causes that are moving corona stats?

With intervention I suppose you can conduct an experiment where you (randomly) pick cities and make half of them wear masks and half of them not. But this is of course unethical! And some stuff you want to know the effects of (e.g., how did gatherings at protests affect the infection rate) are one time events that can't be replicated again.

So all we have left are lots of natural experiments. Different countries/states/communities are handling the situation differently with a wide range of outcomes. No two communities are directly comparable since they differ along many other dimensions other than corona policy. But as a human I am still drawing plenty of conclusions on what caused what and to what degree. So it seems like a solvable problem. How do I make it rigorous?

View on HN

The theory of causal inference tells you how to do this -- what is the nature of the relationship between correlation and causation? When can we draw conclusions from natural experiments? There is actual rigorous theory that has very clear answers to this. The conceptual tools are powerful.

Pearl's "Book of Why" may be a good starting point as a popularization, although sometimes Pearl is not so easy to read. And Pearl presents his own perspective on causal inference only -- there are other schools and techniques, although there are generally equivalences between them.

It is definitely possible to understand the core ideas as a layman, if you have a mathematical grasp of basic probability. If you want to go a little further, understanding linear regression helps too.

The problem is that actually applying the theory to draw causal conclusions on real-world data seems to be a very subtle and difficult process and even experts who specialize their entire careers in causal inference frequently make mistakes and disagree with each other. So I think a lot of humility is warranted.

It’s interesting to study, but it doesn’t magically make hard problems tractable. It seems to make certain common mistakes less likely. When you do statistical analysis or machine learning to model p(x,y) or its friend p(y|x) it’s possible that you mistake correlations for causation (because there is no way to express directionality of dependency). If I have data about the measurement on a barometer and atmospheric pressure then I may learn a model that predicts higher pressure when the barometer reading is high. But if I manually force the barometer reading higher that doesn’t increase the atmospheric pressure. Causality provides a mathematical framework to express that idea.

The problem I have with causality is how to find accurate causal diagrams from unstructured observational data. We _could_ guess and check every possible causal relationship, but that’s at least exponentially hard—in which case causality is useless.

People seem to have reasonably good capabilities for generating candidate causal hypotheses from observational data (basically all of modern science), but most of the material I’ve found on causality focuses on the theoretical benefits it provides rather than on practical applications at scale. (I don’t care if we can automate finding the causal graph for barometer & air pressure; how do I find a causal graph for classifying fake/real news from plain text data?)

I guess it's tricky because the real world is full of feedback loops. If you want a causal model for fake news then your model needs to include some representation of incentives for ad revenue and clickbait. How does the causal inference framework handle feedback loops?

I believe DAG-based causal inference isn’t able to handle feedback loops (acyclic) or nonlinearity (linear). Nonlinearities include stuff like deadbands and delays.

Control theory models handle these things just fine. But control models are hard to apply to sociological/epidemiological domains, where causal inference dominates.

From what I gather, causal inference is useful for designing studies. I’m not sure if they’re used for prediction — would appreciate if someone in the know could chime in.

Interesting. I wonder if someone has tried to combine the two. I guess modern deep reinforcement learning is one such combination because it combines feedback (reinforcement) and probabilistic descriptions but maybe there are other interesting combinations of probability, causality, and feedback.

> DAG-based causal inference isn’t able to handle feedback loops (acyclic) or nonlinearity (linear)

I don't think this should be true, and if it is, then "causal inference" should be qualified to refer only to a specific modeling framework. As a counter example, it's possible to formulate a nonlinear differential equation model of Covid spread and infer the parameters to construct a plausible, causal, generative model.

The sole way would be to have global knowledge of every datapoint; omniscience. It’s impossible.

So, the solution is to get comfortable with the lack of rigor. What can be known in a system without rigor? That’s the question to make rigorous, I think.

> What can be known in a system without rigor? That’s the question to make rigorous, I think.

Who is working on making that rigorous?

These folks used a synthetic control to do the exact quasi experiment you described: https://www.iza.org/publications/dp/13319/face-masks-conside...

Reply on news.ycombinator.com