The New Science of Causation
aaronsw.com
aaronsw.com
The problem with social science is that it tries to be a science but in reality its closest relative is product management. Using even the best statistical methods still runs into the keys/lamp post problem. There are too many variables, the variables are not well defined, the variables may have strange interactions, there may be too much noise in the data, etc. In almost all social science situations, you cannot find truth solely through regressions.
For instance, recently I was studying the issue of whether the crash in the stock market caused the fall in consumer spending, or the slow down in consumer spending caused the fall in the stock market. If you run a straight regression you'll have a lot of problems because a) even if the stock market falls first, it could have been because of expectations of future consumption b) the data on expenditures is not collected frequently enough, and c) there are other potentially confounding variables.
But if you take the product management approach you can listen to what people say. And when you listen, you find that people say stuff like, "After the stock market fell, I had to cut back my plans to travel." or "Since the fall in our endowment, we have had to suspend spending on all new projects." By simply asking people you can figure out the causality.
Note that this is how every company in the world makes most of its decisions ( except perhaps Google, because they actually enough controlled data to meaningfully do regressions). When we're trying to figure out what features to build, we do customer interviews, follow-me-homes, surveys, feature requests, talk to customer support, etc. We make very modest use of statistics, because we do not have large amounts of controlled data. Did the customer not use this part of the app because it was not useful, or because we got the label wrong, or because of something else? We could do a bunch of statistics, but what's the point? It won't answer those questions. Asking people may be subjective, but it's the only way (keys/lamppost).
The problem is that social scientists do not get promoted for making accurate judgments and being right. A product manager does. A social scientist/academic can only be judged by his methods. Thus there is the preference within the professions for methods that can be objectively evaluated, and methods that only a select elite can perform. Being right is not part of the selection process for being an academic.
I think it is different. Do you understand the smoking/tar/cancer example? See here for the example: http://bayes.cs.ucla.edu/IJCAI99/ijcai-99.pdf (pages 34-38 and 46), and here for a better introductory text: http://cscs.umich.edu/~crshalizi/notebooks/graphical-models.... .
Basically, if the situation allows you to reason that some causal arrows are impossible (e.g. due to time or space separation), you can often infer which of the remaining causal arrows are real and which aren't, using only non-experimental correlation data and some formal inference rules. Of course it doesn't always work - sometimes there really isn't enough data and you have to do a controlled experiment. But on the other hand, people often need to reason about causality in areas where controlled experiments are impossible or unethical, like smoking/tar/cancer.
As different as astrology and astronomy. Multi-variable regressions with control variables are known to incapable of supporting causal inference. The network based techniques of Pearl et. al. do permit causal inferences. Notice though that the network based techniques will quite often rigorously prove that the data you have are inadequate to make a causal inference. They do this by discovering multiple causal networks, all consistent with the patterns of correlation and conditional correlation in the data, but with causal links going one way in one net work and the other way in another network.
"You must learn to distinguish between what is true and what is real." See: http://www.edge.org/q2005/q05_8.html#kay
I think most mathematicians would disagree with your assertion that math is about "definitions". (I'm no math geek, just a programmer, so take with an appropriately sized grain of salt.)
> I think most mathematicians would disagree with your assertion that math is about "definitions".
An analogy: math is about definitions in the same sense that go is about placing stones on a grid.
There's certainly an abstract similarity, but etymologically, constructing a proof is very different from running an experiment. A proof declares a statement as True. An experiment (at best) shows that reality behaves as your model hypothesized it would. Experiments can provide evidence of something, not prove it as True. As Alan Kay says (following former link):
When we guess in science we are guessing about approximations and mappings to languages, we are not guessing about "the truth" (and we are not in a good state of mind for doing science if we think we are guessing "the truth" or "finding the truth"). This is not at all well understood outside of science, and there are unfortunately a few people with degrees in science who don't seem to understand it either.
Now comes the cuasality connection -- if the best model is P(A)P(B)P(C|A,B) you read it as A->C<-B. If best model is P(A)P(B|A)P(C|B), you read it as A->B->C. (ie, A causes B, B causes C)
I think causality connection is questionable because the model that corresponds to A->B->C will have the same fit as model for A<-B<-C or A<-B->C. In fact, you could take any causal network without loops and "unshielded colliders" (connections of the form A->B<-C), pick any node as a root, and re-order the arrows to face away from the root to get a model with a different semantic causal structure, but the same mathematical structure, meaning it'll give identical fit to data.
What would be really interesting is if someone deduced causality using Pearl's approach, then verified it using a direct experiment
It's not useful to have a pure causality representation because with the real world and other complex systems, you don't know all the variables. Your variable 'A' in 10 years might be shown to be a combination of variables.
I, for example, only 99.99% believe in gravity. One day we might float off the planet. Lack of 100% proof about observations was one of Hume's basic arguments and I think the point of the "do-calculus" for AI that is discussed in this link. We all have an internal "do-calculus" for operating in the world: work with current held beliefs/rules until they're proven to work differently, etc.
I normally hate to hear people say no one knows how certain things work but in this instance it seems apt. No one has created a self autonomous consciousness yet and so no one really knows exactly what goes into it.
I guess there's also a difference in advancing the study of robots and advancing the study of artificial intelligence.
huh? I have no idea what you're talking about.
I suspect the theory of causality will be the same way; yeah, a quick approximation that works most of the time is readily available (almost enough information in that post for a mathematically-inclined person to work it out, after all), but working through all the nuances will probably take us somewhere unexpected. For example: What will it mean when someone proves there is no one theory of causality, like there is no one set theory? (Obviously, I don't know this will happen, but from a strictly mathematical point of view it sure wouldn't surprise me much; for a similar example, see Arrow's Impossibility Theorem. (If you (in general, not dfranke specifically) are not math savvy, still, Google it; it's not that hard to follow. Wikipedia's a bit heavy on the math, there are less formal introductions around.)) A branch of math to keep an eye on, I think.
tl;dr: for truth to make any sense you have to postulate how "X being true" would cause a different sensory experience vs it being false.
Afraid it's going to be terribly brilliant with lots of maths and I won't understand it.
I know, I should just give it a try...
(I would support a hypothetical ban on those too, FWIW)
Video lectures has a workshop from NIPS '08 with talks by Pearl, Spirtes, et al at http://videolectures.net/coa08_whistler/
The UAI and NIPS conferences are a common venue for this research to be published in if you're interested in the latest results.