Hacker News Reader

I am reminded of an observation made by my former boss, a planetary scientist at NASA, about that same statistical effect:

When a new Earth-crossing asteroid or comet is discovered, it's orbit is rarely known with any great certainty. The cross sectional area of the Earth divided by the area of space the object might possibly pass through yields the chance of a collision. With each successive night of observation, the orbital characteristics are determined to greater accuracy, and the cone of potential trajectories is reduced. As a not-so-intuitive result, the reported probability of a doomsday scenario grows geometrically higher with each successive night of observation. In some cases it makes national or world-wide news as the probability of mass extinction grows from 1 in a billion, to 1 in a million, to one in 10,000... Obviously there's a trend there, right? Some journalists think so, then mass hysteria ensues and NASA receives correspondence from mothers asking if they should euthanize their children to protect them from the horrific event (a true story, sadly).

Then at the height of mass hysteria, the next night's observation brings the circle of uncertainty just small enough to exclude the passage of the Earth, and instantly the probability of Armageddon drops to zero. Those media outlets who hyped the doomsday scenario then accuse the scientists of fear mongering (ironic), and the public forgets until the next big object is found looming our way, and the cycle continues.

It's sad, but not unexpected that the general public lacks the mathematical literacy to understand these statistical quirks. It's truly sad, and perhaps even criminal that journalists, and especially science journalists fall victim to those same misconceptions.

It might be because how it is communicated, where an uncertainty is communicated as a chance.

"It's truly sad, and perhaps even criminal that journalists, and especially science journalists fall victim to those same misconceptions."

"Journalists"? You are being moderated in your high praise of 'journalists', that is, professional members of the highly professional journalistic profession?

Cruel. SO cruel. How can you be so CRUEL?

I mean, while we're discussing particle physics, don't you believe that the English, drama, theater, and communications majors also need to eat?

And it's even worse, you are being even more cruel: You are suggesting that the journalistic profession should pay attention to technical information and, thus, push out the main technique of journalism back over 100 years: Use communications of human experience and emotion to grab people by the heart, the gut, and below the belt.

So CRUEL!

Gee, some people really do NOT like facetious comments, parody, and humor! What is it particle physicists: Don't understand humor?

Gee, looks like a LOT of dedicated, devoted English majors here. Who would'a thought?!

Maybe I'm missing something, but how is this argument flawed? Bayesian probability says that eliminating possible universes increases the probability of other ones (given everything else as constant). So reducing the probability that the Higgs Boson lies in some energy range (which is what this experiment did) increases the probability that a) the Higgs Boson doesn't exist and b) the Higgs Boson exists at some other energy level, which is what this article is saying.

I think we should be careful with Bayesian inference for the usual reason -- there is only one universe with one (or zero) Higgs particle. You can't really collect statistics over universes, so what is the meaning of probability in this case?

More concretely, what prior probability would you assign to the distribution of the Higgs mass? This depends on your theory. For instance, supersymmetric theories tend to favor a lower Higgs mass, so if you are a proponent of such theories you wouldn't expect a heavy Higgs anyway.

Note that physicists themselves do not phrase their results in these terms. The statement "Higgs excluded above 130 GeV at 95% confidence" does not mean there is 95% chance the Higgs isn't there. Rather, this statement has a precise meaning based on frequentist reasoning.

But perhaps more importantly, the tone of the article is sensationalist and seems to imply that we are about to give up hope. This isn't true at all. I mean, look:

     "while CERN will continue it's search at least until the end of this year, if no positive results about the Higgs should come out, Stephen Hawking ... would be able to cash in on his wager."

Bayesian inference is applicable here, because we are talking about the subjective probability - ie, confidence - that one single fact is true. There is no way to reduce a single "event" (that there is Higgs particle or there isn't) to a frequentist, or "objective", probability.

As you correctly say, the influence that the results we already got about the possibility that Higgs boson exists with a high mass have on the global possibility that there is a Higgs boson depend on the a-priori (to the current experiment) confidence that we give to the high mass/low mass hypotheses. So there is for sure not a 95% drop in confidence, but there is indeed a drop, unless you gave 0 confidence to the high-mass hypothesis before the experiment.

I'm not saying Bayesian inference isn't applicable, I'm just cautioning against an careless interpretation of its results. But if you insist on interpreting it in this way then yes, it means the probability of finding a Higgs is lower.

However, as I said above the term "95% confidence" is not related to this reasoning at all. Saying for example "the Higgs mass is not 140 GeV at 95% CL" means precisely: If the Higgs were at 140 GeV, it would have 5% probability of producing the results we measured experimentally. It does not mean "we are 95% sure the Higgs isn't at 140 GeV".

Of course not, but we should discount our belief that the Higgs has a mass at 140GeV by a factor roughly proportionate to the 95% confidence of the result. And I don't think anyone in this thread was actually claiming that we are 95% sure the Higgs is not at 140GeV, that's usually precisely the sort of mistake that relying on Bayesian methods helps you avoid.

    "we should discount our belief that the Higgs has a mass at 140GeV by a factor roughly proportionate to the 95% confidence of the result"

I'm sorry but I don't understand what this means in practice. The first part is a Bayesian belief, while the 95% confidence result comes from frequentist analysis. I'm not sure how you can mix the two.

The difference is that frequentist practice would be to stop at the 95% confidence interval and leave it there, whereas a Bayesian would use that observation to update their probability estimate of the theory being true.

"If the Higgs were at 140 GeV, it would have 5% probability of producing the results we measured experimentally" is the same as P(Observation | Higgs at 140Gev) = .05

So we can say that

  P(Higgs|Observation) = P(Observation|Higgs) * P(Higgs)
                         -------------------------------
                         P(Observation)

So given that getting your new belief about the probability of a Higgs Boson at some energy is going to be updated based on your observation, you can see that it ends up being scaled by that exact confidence result. That's sort of an oversimplification, since really you end up calculating the P(O) scaling factor based on P(O|H) among other things, but I hope you can see how they're closely related in practice.

Thank you, now I understand what you meant. I concede (again) that using Bayesian analysis the new results do lower the probability that the Higgs exists. Personally I don't subscribe to this point of view since, if the Higgs exists and has a low mass, the most likely chain of events is: Bayesian probability for Higgs existence starts at some subjective value, goes down (with a subjective slope that depends on your priors), then goes up and reaches 1. Not only is it subjective, this just doesn't feel to me like it is describing anything "real"; it seems like we're just playing with numbers. But I guess this is already way off topic for this discussion.

For me the important point to communicate was that the article is, let's say, mostly nonsense. Just consider the title:

> A Higgs Setback: Did Stephen Hawking Just Win the Most Outrageous Bet in Physics History?

Never mind the superlatives. There was no "Higgs setback", and the answer to the question is "No". The article does not leave out the correct details, but I'm quite certain it leaves the layman with the feeling that the Higgs search is all but doomed.

I agree with you about the article being just sensationalistic non-journalism.

About not using a Bayesian approach, though, I don't understand how could you infer the existence or not of the Higgs without it, considering that we can only measure something that is probabilistically correlated to what we want to find.

In other words, what should happen for you to say that we have verified that there is a Higgs boson? I'm pretty confident that it would be some application of Bayes theorem :)

I'm not saying there's a 95% chance the Higgs isn't there. I'm saying there was some non-zero probability that it was in the specified energy range before, and given these results the probability that it's now in that range is now 5% of the value it was before. Or more precisely, the probability that we're in the universe where the Higgs is in the specified energy range and it's detectable with this experiment just went down to 5% of it's original value. Even if we can't accurately identify what the value of the probability is, we can still say it went down to 5% of its original value on the basis of the evidence from this experiment. I would also argue that prior to this experiment the probability of the Higgs appearing in this energy range given our state of evidence of how the universe works was significant (more than 15% I'd say) otherwise we wouldn't have built the LHC. I understand you can't say this with a frequentist approach because we don't have a sample of universes to draw on to estimate priors.

Finally we're not talking about the probability of some state of the universe in the absolute (as frequentists claim, we can't meaningfully talk about such things), we're talking about some state of the universe given the evidence we have.

I'm not sure I follow your first argument. The probability of finding the item in the remaining 5% of the search space is independent of the time required to search different sections of the search space (unless of course you have additional prior information about where the item is more likely to be located).

The fact that it may take you, say, 90% of the time to search a particular 10% section of a search space doesn't increase the probability of finding the item in that section.

The difference is that physicists have a good idea of where the Higgs should be, and the search isn't quite intuitive. They don't just have to find a single particle, they have to look at a large collection of events and perform statistical analysis. Also, the time it takes to search for lower energy particles may not be the same as for high energy particles.

It's a little more complicated, but "all" the energies ranges are searched in parallel.

It is easier to discard the ranges that are very far from the mass of the Higgs' boson. So initially the expected range is very broad, something like:

* More than 114GeV, because if it's smaller we would have seen the Higgs boson in another smaller accelerator (LEP)

* Less than 185GeV, because we saw small corrections that are probably due Higgs boson in the (LEP)

So you get a lot of money to build another accelerator, and you are confident that the maximal energy of the collider is enough to see something.

In some ranges, the experiments doesn’t show anything interesting, so it is possible to discard that the mass of the Higgs boson is in that range. It is easier to do this when the range is more far away from the "real" value. So you can discard with a 95% of confidence that range, get some papers published, perhaps a few Ph.D. thesis, compare the data to another more indirect calculations, show some progress, and ask for more money because the accelerator is really big.

The problem is that some energy ranges are more difficult to test, because other well known particles appear but the final results is very similar to what is expected from a Higgs boson. So it is important to choose some strange phenomenon where there is easier to see the difference between a Higgs boson and another particles. So with more experiments you can discard with a 95% of confidence a new range, ..., because the accelerator is really big.

But some ranges are more difficult to discard. There are a lot of interesting phenomenons. Some of them are due to other particles. The other particles are well known, so it is possible to calculate how many of these are expected to appear sadistically. But the experiments show that there are more than expected. It can be a statistical fluke, or it can be corrections that appear because the mass of the Higgs boson is near that range. The real problem is that if the Higgs boson really exists, a lot of corrections appear in the nearby mass ranges, so these ranges are more difficult to discard with a 95% confidence ...

The most difficult range is the one that includes the actual mass of the Higgs boson. It should be impossible to discard :). But it is not enough to not be able to discard it with a 95% confidence, because it can be a statistical fluke. To "prove" that the Higgs boson exists wits a 99.99997% confidence, more and more experiments are necessary to see the difference between a statistical fluke produced by the background process of the other particles and a real signal produced by the Higgs bosons. So you need more time to find it where it really is, than to discard it where it isn’t. (So you need more money to run the accelerator, so you need to dhow some preliminary results.)

So, next year we will probably see something like "RIP^2 Higgs Boson, 99% of the initial search range discarder forever with 95% confidence, only a minuscule 1% remaining." and it will be really a good new. And in a few (5?) years something like "Zombie Higgs Boson found, returned from 99.9% dead." and it will be a really good new.

> On the other hand, not finding the Higgs would be an extremely exciting result, since it would open the way to less well-explored ideas about the origin of mass. The goal of the LHC is to teach us about the world, not stroke physicists' egos and tell us how clever our existing theories are.

Maybe the Higgs boson is the new aether[1] which, after many experiments failing at observing its existence, was replaced by relativity and quantum physics. By which I concur that an experiment built for, yet failing to demonstrate a point is not necessarily a failed experiment.

So whatever the results of the LHC, I'm confident that it's a worthwhile task to undertake.

[1] http://en.wikipedia.org/wiki/Luminiferous_aether

There is a big difference. The experiments showed that the ether didn’t exist. But there is no experiment that shows that there is no Higgs boson. IIRC the expected roadmap is to find the Higgs boson in 3-5 years (from now) in the LHC, so don’t expect to see it sooner. (It is expected to see it in sooner in 2-3 years in the Tevatron, if it were kept open.)

The experiments show that if the Higgs boson exists, it is not very heavy (not heavier than 145GeV). And there is some indirect and inconclusive evidence that it exist and the real mass is near ~120GeV (probably +-10GeV), but it can be a statistical fluke, so we have to wait a few years.

Didn't the author of the article shoot himself in the foot with this statement:

> These assessments carry a probability measure, such as 95%, 99%, or—as traditionally required in particle physics for a “definitive” conclusion about the existence of a new particle: 99.99997% (this is the infamous “five-sigma” requirement).

So...the five-sigma requirement means that your run-of-the-mill physicists won't accept a claim until there's statistical evidence accurate to 99.99997%. From 95% to ...that number is still a significant number of tests, no? Doesn't this paint the title as sensationalist with the author's own words?

For discovering an effect the standard is indeed 5-sigma. But here we are discussing excluding an effect, namely deciding that a suggested effect isn't there. For this the standard is lower.

"The goal of X is to teach us about the world, not stroke Ys' egos and tell us how clever our existing theories are."

This a million times. I want this on a shirt, with the millions of X's and Y's it satisfies. Thank you for posting.

This comment thread is more interesting than the article.

The article did not hide any of these facts though, so it's not "utter nonsense". The title of the original article is "A Higgs Setback: Did Stephen Hawking Just Win the Most Outrageous Bet in Physics History?", nothing like "RIP Higgs". The article, at best, is trying to be provocative. But, more importantly, you believe in god?