Hacker News Reader

Comment by nonbel | Hacker News Reader

Causality has never been of much interest to me. I never understood why others make such a big deal out of it. Maybe it is a real thing, maybe illusion, definitely some kind of heuristic... but shouldn't we be searching for useful "laws" (eg F ~ m1*m2/r^2) rather than "cause"? I know some have denied causality a place in science:

"In the following paper I wish, first, to maintain that the word "cause" is so inextricably bound up with misleading associations as to make its complete extrusion from the philosophical vocabulary desirable; secondly, to inquire what principle, if any, is employed in science in place of the supposed "law of causality" which philosophers imagine to be employed; thirdly, to exhibit certain confusions, especially in regard to teleology and determinism, which appear to me to be connected with erroneous notions as to causality." http://www.hist-analytic.com/Russellcause.pdf

View on HN

Rule mining[1] is a thing. It's very useful.

But it has a weakness - it doesn't always understand causality. As you note, much of the time this doesn't matter.

A good counterexample is the story of scurvy prevention[2]. Back in 1747, the British Royal Navy proved that scurvy could be prevented by eating citrus. The Navy started serving all sailors lime juice, and sure enough scurvy stopped occurring.

Fast forward to 1911. Scientists had developed rules which matched the observed behavior of scurvy very well:

Atkinson inclined to Almroth Wright’s theory that scurvy is due to an acid intoxication of the blood caused by bacteria.. There was little scurvy in Nelson’s days; but the reason is not clear, since, according to modern research, lime-juice only helps to prevent it. We had, at Cape Evans, a salt of sodium to be used to alkalize the blood as an experiment, if necessity arose. Darkness, cold, and hard work are in Atkinson’s opinion important causes of scurvy.

Basically, what had happened was that widespread steam travel and a process for canning lime juice were developed roughly simultaneously. The canned lime juice no longer provided vitamin C, but no one realized this was a problem because sea voyages were much shorter so scurvy never developed.

That lack of a correct causality model meant that when polar exploration happened, scurvy started occurring again and no one knew why.

[1] https://www.mapr.com/blog/association-rule-mining-not-your-t...

[2] http://idlewords.com/2010/03/scott_and_scurvy.htm

But your explanation of the problem here is that nobody recognized the other relevant correlations well enough to rule them out.

His example demonstrates that determining causality avoids problems such as the inability to recognize other relevant correlations.

Yes.

But my point was to show that causality is important, which the OP appears to doubt altogether.

As usual, Pearl has some good writing on this:

http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf

>Take, for instance, Newton’s law:

>f = ma.

>The rules of algebra permit us to write this law in a wild variety of syntactic forms, all meaning the same thing – that if we know any two of the three quantities, the third is determined.

>Yet, in ordinary discourse we say that force causes acceleration – not that accelera- tion causes force, and we feel very strongly about this distinction.

>Likewise, we say that the ratio f/a helps us determine the mass, not that it causes the mass.

>Such distinctions are not supported by the equations of physics, and this leads us to ask whether the whole causal vocabulary is purely metaphysical, “surviving, like the monarchy . . .”.

>Fortunately, very few physicists paid atten- tion to Russell’s enigma. They continued to write equations in the office and talk cause–effect in the cafeteria; with astonishing success they smashed the atom, invented the transistor and the laser.

How about: In ordinary discourse we say that "the earth orbits the sun" – not that "the sun orbits the earth", and we feel very strongly about this distinction.

It just seems like discussion of "what causes what" is missing the point, just like the above argument. That one persisted for centuries (millenia?) until it was pointed out the answer is neither! It depends on your reference frame.

Causality is quite different from frame of reference.

Lets return to your equation. r^2 = m1*m2/F is a statement of correlation that's valid to many digits, that also says that "these are all the variables that affect anything here." It allows for calculations of quantities by plugging in numbers. In combination with other equations, it allows extrapolitaion and physical thinking. But equations alone are not enough for a physicist to make progress on problems, they don't combine arbitrarily, they still use that famous "physical intuition" that turns out to involve a heck of a lot of causality that they talk about in the cafeteria but don't often write about.

In the world outside of physics classrooms, most situations involve a huge number of variables interacting with each other, and the variables themselves are leaky abstractions. This means that the correlation of quantities does not go out to many digits, or even one digit. This does not make an equation a useful abstraction. People have developed probabilistic models that stand in for equations in these situations. Sure, it would nice to have an equation that modeled things like they do in the simple situations that Physics publishes on, but when the equations of physics are so weak that they can't even fold a protein, how are we going to use them to model the interaction of a potential drug with a protein?

Equations are great, when they apply. By introducing simplifying assumptions, we can sometimes extend their utility into new domains, but at the cost of introducing fragility because of those simplifying assumptions. They allow for calculating new things with only a very few computations. But nearly every new field of science invents new math. One of the more recent inventions is causality. It works fantastic in those sciences that use experimental interventions. Simple algebraic equations do not.

Sorry, all these arguments seem like non-sequiturs to me.

Physicists might pray together in the cafeteria to come up with good ideas. It doesn't mean God answered their prayers when they have one.

Useful laws need not be exactly correct. You can keep adjusting to stay on course to account for the error due to any simplifications.

Sure causality seems like it must be at least some kind of heuristic, that doesn't mean looking for causes is the best, or even a good, use of time.

Well, your arguments seem equally non-sequiter to me. I spend a lot of time in the world of correlations, in biology. They're not so useful. It's much better to get a causation: a knob that when you turn it you get a response, rather than no response. If you can't see the utility of the two, then perhaps there's not much to discuss?

>"Well, your arguments seem equally non-sequiter to me."

Fair enough.

>"I spend a lot of time in the world of correlations, in biology."

I left that field. One reason was because I realized so much of my time was wasted after I attempted to attach numbers to things like "# of receptors required to fit the narrative".

Sure "A causes B", but it must be for some reason totally different from what is usually proposed (and could probably be considered a boring experimental artifact). I just don't care if "A causes B". I want to know for "f(A) ~ B", what is f?

>"It's much better to get a causation: a knob that when you turn it you get a response, rather than no response."

Reminds me of this paper:

  "Biologists summarize their
  results with the help of all-too-well recognizable diagrams,
  in which a favorite protein is placed in the middle
  and connected to everything else with two-way arrows.
  Even if a diagram makes overall sense (Fig. 3a), it is usually
  useless for a quantitative analysis, which limits its
  predictive or investigative value to a very narrow range.
  The language used by biologists for verbal communications
  is not better and is not unlike that used by stock
  market analysts. Both are vague (e.g., a balance between
  pro- and anti-apoptotic bcl-2 proteins appears to control
  the cell viability, and seems to correlate in long-term with
  the ability to form tumors) and avoid clear predictions."

https://bml.bioe.uic.edu/BML/Stuff/Stuff_files/biologist%20f...

I think that you would really really like modern causality research if you bothered to look into it. It's all about f(A) ~ B, but recognizes that there are different ways of varying B. Do you disconnect B from the rest of its influences when you vary it, or are you letting them vary.

If you try to model f(A) ~ B in biological systems without simultaneously modeling causality, then it's going to go wrong. This happened to me in year 2 of grad school. The bayesian network built around data without modeling the causality was wrong, the one that took into account the causality from the knockout experiments was right.

Sure causality seems like it must be at least some kind of heuristic, that doesn't mean looking for causes is the best, or even a good, use of time

I wonder if you'll recommend your doctor use that bit of wisdom the next time you're sick.

After graduate training in biomed, I am horrified of going to a doctor. After seeing what goes on (ie, lots and lots of NHST + p-hacking), it is really difficult for me to understand how anyone can tell if modern medicine is helping or hurting people overall.

It would actually be miraculous to me if it is the former, to the point I would be tempted to believe there is some occult driving force leading humans in the right direction.

However, I was thinking about similar the other day regarding the dentist. Say I go get a teeth cleaning. Did the polishing cause my teeth to be cleaner, or did the cleaner teeth cause the dentist to do the polishing? I feel like there is some kind of semantic game being played there.

Did the polishing cause my teeth to be cleaner, or did the cleaner teeth cause the dentist to do the polishing?

Wow...

I don't think it is that dumb. If the teeth were not going to get cleaner, the dentist would not have taken the action of polishing them. Therefore, cleaner teeth caused the dentist to polish the teeth.

Eh, maybe it is a dumb example, but I'd appreciate if you could put your finger on it rather than sarcasm.

> but when the equations of physics are so weak that they can't even fold a protein, how are we going to use them to model the interaction of a potential drug with a protein

Minor nit: the equations aren't weak at all — they predict the gyromagnetic moment of an isolated electron to ten decimal digits of agreement with experiment. The weakness is in the computational implementation of those equations, i.e., solving the equations in a reasonable time. You can perform a relativistic quantum monte carlo simulation of protein folding, and I guarantee the results will match those obtained in the lab to whatever level of accuracy you desire, but you'll have to wait longer than the age of the universe for the computation to finish running.

Also, for work on small proteins, the calculated chemical properties match the experimentally tested ones very closely nowadays. The problem is all of the unknown interactions that a new drug has with the rest of the human body outside of one protein. And without a large atomistic model of a human body, it's going to be a long time before simulation can solve that problem.

> The weakness is in the computational implementation of those equations, i.e., solving the equations in a reasonable time. You can perform a relativistic quantum monte carlo simulation of protein folding, and I guarantee the results will match those obtained in the lab to whatever level of accuracy you desire, but you'll have to wait longer than the age of the universe for the computation to finish running.

This is precisely what I mean by "weak." The equation looks great on paper, but isn't so useful in practice. And the problem isn't predicting just one protein-drug binding, it's predicting the interaction with all known proteins, all molecular complexes, and the biofilms, etc. I would go so far to say that Physics with a capital P is not so useful for many physical problems at all. Where Physics is useful, it's unbelievably useful, but its domain in the physical world is vanishingly tiny.

Physics kind of has the reverse problem of AI. With AI, when you finally solve a problem, it's no longer considered AI, and AI is continued to be considered a failure. With Physics, a problem isn't part of the body of Physics until it's been solved, at which Physics is now considered a great success.

the answer is neither! It depends on your reference frame

Someone is new to the idea of relativity. This is nonsense. The frame of reference describing the sun rotating around the earth is not inertial (https://en.wikipedia.org/wiki/Inertial_frame_of_reference) and thus does not follow the laws of physics we know and love. In short, "Nope, you cannot say the sun orbits the earth."

I thought that using a non-inertial reference frame just introduced a "fictitious force"?

Is that not the case?

You're correct. The laws of physics get very strange in a stationary earth reference frame. Imagine the forces that would be whipping the distant stars around earth.

Well, I've made a solar system simulator before. Nothing great, but it was sufficient to convince me that is just a matter of perspective. The "law of physics" used was:

  d = sqrt(dx^2 + dy^2 + dz^2)  
  F = G*m1*m2/d^3

That would just be summed up over all the objects included in the system. However, when visualizing the results, I could set (0,0,0) wherever I wanted. I could put it at the solar system barycenter, I could put it at the center of the sun, I could put at at the center of the galaxy (this leads to interesting vortex-like trails...), or I could put it at the center of the earth. The easiest was actually to use the solar system barycenter on an arbitrary date.

I would encourage you to look into Judea Pearl's work. I'd suggest starting with the epilog of his book: http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf to get the general idea.

To be clear: Pearl is not just philosophizing. He provides a formalization that takes causality from being an ambiguous concept to being applied mathematics. The bulk of the book is not a light read, but the mathematics involved are relatively basic.

And to be absolutely clear: Pearl won the Turing prize for his work. It is not simply handwaving or non sequiturs. The common online conversations about causality are several decades out of date with real academic work on the topic.

I read that last time I thought about this topic, and went over it again earlier when epistasis linked to it. The arguments against Russel's position really do not do anything for me. As mentioned elsewhere, they seem to be a bunch of non-sequiturs about what we are told growing up.

Regarding the Turing prize, etc. That type of thing doesn't really convince me. What engineering feats or surprising predictions has Pearl's methodology lead to? When I look at what is successful in machine learning these days, it is RNNs, CNNS, gradient boosting, etc. Correlation-based methods seem to be king, at least for now.

Have you run into bayes nets or probabilistic graphical models? Or read the old Bayesian Automated Taxi paper from the 90's that got people thinking about autonomous vehicles based on these concepts?

Pearl's work is heavily used: https://scholar.google.com/scholar?cites=1099626011922949961...

The Russel quote you posted is in no way some sort of rebuttal of Pearl et all's work. They aren't even in the same ballpark. I don't intended that as a slight agains Russel who I'm generally fond of.

I'd say that causality seems useful for intelligence. Josh Tenenbaum has pushed the idea of causal models and inference, and I find it compelling

> Recent progress in artificial intelligence (AI) has renewed interest in building systems that learn and think like people. Many advances have come from using deep neural networks trained end-to-end in tasks such as object recognition, video games, and board games, achieving performance that equals or even beats humans in some respects. Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways.

> We review progress in cognitive science suggesting that truly human-like learning and thinking machines will have to reach beyond current engineering trends in both what they learn, and how they learn it. Specifically, we argue that these machines should (a) build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems; [...]

https://arxiv.org/pdf/1604.00289.pdf

Some others papers related to these ideas:

- Learning a Theory of Causality http://stanford.edu/~ngoodman/papers/LTBC_psychreview_final....

- Theory-Based Causal Induction: http://cocosci.berkeley.edu/tom/papers/tbci.pdf -

I do accept engineering feats as evidence people know what they are talking about. Are there any tasks where one of these "causality-aware" robots [algorithms] outperforms "pattern-recognition" robots?

From the brief summary it sounds very hypothetical, especially compared to the causality-agnostic pattern-recognition approach which I have seen work surprisingly well. That group has convinced me they are on to something quite big.

I agree that in practise, deep learning tend to perform very well. With that said, it is not without limitations. In general, model-based machine learning approaches do not perform as well, but we should not limit our assessment only to performance.

For example in this review: https://arxiv.org/pdf/1605.01138.pdf, a model-based approach is compared against a deep learning approach. It is very interesting to me that even if deep learning performs better, its failure cases are very different than human failure cases (deep learning is not sensitive to "physics illusions"). Whereas the model-based approach is very much in-line with human performance characteristics. Also, on this task (physical scene understanding), deep learning generalises very poorly.

Causality is important when you want to design interventions.

A non-causal regression model on {Y={CA, no CA}, X ={Blonde hair, dark hair}} may be able to replicate the observed effect that blonde hair and skin cancer go together. However you cannot use that regression coefficient for coming up with an intervention. Dyeing the hair is not going to reduce cancer risk although the regression model will predict exactly that.

It's critically important whenever you are being asked to develop a policy of some kind, whether it's public policy or a business procedure. If you only understand relationships and not the causal direction relationships, you can end up making very bad decisions. E.g., "black men drop out of school more often than white men, therefore we shouldn't waste time educating black men."

Sure, but causality is ridiculously easy to get wrong, and very rarely aligns with intuition. Even the expectation it exists can mislead you.

But the alternative I proposed is coming up with quantitative laws, not vague astrology like "Race A does this more often than Race B, therefore we need to do less of C".

That is an interesting connection to make though. If people view wild speculation as the alternative focus of research, incorporating the concept of causality would at least slow them down. This would reduce the amount of misinformation generated and number of destructive policies implemented. Still, I think causality offers more than that.

> the alternative I proposed is coming up with quantitative laws, not vague astrology like "Race A does this more often than Race B, therefore we need to do less of C".

You appear to be saying that causality is not quantitative, however that is not true.

No, they're saying the causal relation does not add much more (quantitatively speaking) to an existing correlative relation. They're suggesting we throw out the part that distinguishes causal from correlative, not correlative reasoning altogether.

If that's that what nonbel is actually saying, then nonbel does not have much experience with modern science. The difference between P(Y|X) and P(Y|do(X)) are, in my scientific experience, the difference between doing science and doing statistics. The difference between understanding and guessing. This is precisely the difference between quantitative understanding and "astrology," as they phrased it.

How do you distinguish a "law" from "astrology" in this case?

The same way you distinguish rational belief from rationalization.

Namely, in that a rational belief is justified on the basis of prior experiences, while a rationalization is justified on the basis of impracticality due to ignorance.

A law will make some precise quantitative prediction, "astrology" will make vague predictions like "A is correlated with B", or "A is positively correlated with B".

Here is a spielraum[1] of possible experimental results, with the * indicating the region consistent with a law:

  1) |------*------|

Here it is for a vague speculation that there is "some correlation" (ie astrology):

  2) |******-******|

You can see it will be much easier to find evidence consistent with #2 than #1, even if there is nothing to the ideas at all.

[1] Paul Meehl. 1990. Appraising and Amending Theories: The Strategy of Lakatosian Defense and Two Principles That Warrant It. Psychological Inquiry 1990, Vol. 1, No. 2, 108-141. http://rhowell.ba.ttu.edu/meehl1.pdf [figure 3]

Reply on news.ycombinator.com