Hacker News Reader

Comment by rrmm | Hacker News Reader

The result is three-sigma, so not yet there (5 sigma). As usual, interesting if true.

I will give you $999 if it's just noise in exchange for $1 if it's not.

Looks like some people didn't like you putting the concept in more accessible terms. "Three sigma" has a real interpretation, it's not just a little score where you pull out LaTeX when it reaches 6.

But LaTeX seems to take a whole lot longer to run when it's 6 sigma than when it's 3.

Of course, to perform any valid statistical reasoning, you need to consider the prior. The standard model has held up very well, and we do a lot of experiments-- any of which has a small chance of popping out a result that looks highly significant by chance. There's a reason why 6 sigma is the line to take things super seriously (in fundamental physics).

Break-even for three sigma would be 997:1, so the parent should actually take this bet.

The result was past three sigma, not exactly at three sigma. But good point, because if it was exactly at three sigma I would have snookered myself. :)

How about 999 subscriptions to Penthouse in exchange for 1 subscription to Private Eye?

Plenty of 3 sigma results have come and gone, (there's obviously an xkcd for this about doing a whole lot of experiments). It's certainly something to watch and I would personally be stoked for some new physics.

I only posted because it was the first thing I wanted to know, and I had to dig around in the article for a bit before I found it at the end.

My mentor once said "3 sigma effects happen in physics a lot more often than they should." I've since come to realize that we should do a multiple-hypothesis testing correction on all such results, where the number of hypotheses tested is approximately equal to the number of Ph.D.s granted in particle physics.

The model indicates 1000:1, but it may not capture all aspects (i.e. correlations) in the system. So what seems like excessive sigmas is also margin for those unknown-unknowns.

Hence the qualifier "if it's just noise," ruling out other falsifications.

You are joking, right? The experiments around the LHC have produced a handful of three-sigma "non-standard-model" results already. So far they have all gone away.

This is not a bad thing, by the way - just an inevitable consequence of doing many searches.

Yes. I think it's a joke.

As in 3-sigma ~= 1/1000

Yes but he brings up a good point. They are more likely to talk about a 3 sigma result that challenges the standard model than one that doesn’t yield anything special. So given you know a 3 sigma result is being discussed, the probability that it’s a false result at higher sigma is more than 1/1000.

Shouldn’t there just be around 1 in 1000 experiments the state reach 3 sigma for a result which then turn out to be noise?

I guess that only applies if there are no mistakes or biases at all?

You're not seeing all the searches that didn't pan out.

Reply on news.ycombinator.com