Comment by joe_the_user | Hacker News Reader

Comment by joe_the_user | Hacker News Reader

Correlation + plausible based on your knowledge of the world implies causation (obviously to the appropriate degree).

It's the flip-side of extraordinary claims require extraordinary evidence.

Facebook driving Greek debt is implausible and two vaguely shaped curves aren't enough. A formula that predicts to many decimal places over a fair period, prospectively, would be really weird but hard to ignore.

Spanish debt, gold prices or something related on the other hand, would require proportionately less evidence for causation.

This disturbs people because it implies you can get stuck in a pathological view of the world and statistics and evidence won't get you out. Sorry about that.

Edit: And this is pretty the Bayesian Approach. It's just that Bayesian Statistics and Pearl arguments, are themselves just models of the world. You can have others but, even more, you need more of an arguments than "stuff that seems implausible needs more evidence" and all of the more elaborate stuff is going to be specific to a given situation and thus might be applicable to a different one.

"Facebook driving Greek debt is implausible"

Yes, but in reality you don't know that

Plausibility is a subjective measure, and while I would say that, yes, it can be a hint, you cannot disregard something merely because it's implausible

> a hint, you cannot disregard something merely because it's implausible

Coming back to Bayesian statistics, the word for this is prior, and its not that you disregard evidence, its that you can quantify both your existing beliefs about reality and the change in your beliefs according to the evidence you see.

Firstly, our hint: we have a prior assumption of the probability that the popularity of Facebook is driving up Greek debt (call it P(Fg)). Then, we observe a correlation between these two things. For the sake of argument, I'm going to make this 0.001 (I'd probably estimate less).

Now, once we see this correlation, we now need to calculate two things: 1. The probability of observing that correlation (call that P(C)). Note that the more extraordinary the correlation, the less probable it is, and the smaller this term would be. In this case, the graph matches vaguely, I'm going to give it a probability of 0.1.

2. Given a world where there is a causation, what is the probability we'd see this correlation (Q: I'm not 100% on this part). Now, Greek debt could plausibly be driven by other things, which would mask the Facebook effect, so there's no guarantee there would be a correlation. This term is called P(C | Fg), and I have no idea what value to give it. Let's try 0.5.

What we want to know is: P(Fg | C), that is, the probability of a connection given we have observed a correlation.

Boom! P(Fg | C) = P(C | Fg) x P(Fg) / P(C)

So our posterior probability (after observing this correlation) changes from 0.001 to 0.001 x 0.5 / 0.1 = 0.005

Right.. but when you find correlation, you just see that two things are obviously related in some way - there is a common factor. It could be direct causation, or it could be any number of other things. The question is whether or not you have a theory that can be analyzed to see if this is a CAUSE or not... and we can't test every correlated thing out there exhaustively.

Plausibility is a measure based on less than exact knowledge of the world. I indeed don't know with certainty the relationship between Greek debt and Facebook value. Not all less-than-perfect knowledge of the world is subjective however - subjective involves personal prejudice but there's more than personal prejudice operating in the judgement that Facebook prices won't affect Greek Debt - there's an understanding of finance, a doubt about action at a distance, etc.

Also, if you read my post, I'm not "disregarding" anything. I am simply saying that some things have a higher bar than others.

Reply on news.ycombinator.com