Neutrino experiment affirms faster-than-light claim
blogs.nature.com
blogs.nature.com
But it is an important step towards understanding what's happening, and it's great that the OPERA group were able to put together this followup experiment so quickly.
You might be thinking of "confirm". This certainly is an example of the OPERA group affirming their results.
> Affirm. v. 1. State as a fact; assert strongly and publicly. 2. Declare one's support for; uphold or defend.
Confirm means to check again.
This test confirms the accuracy of OPERA's timing measurement, ruling out one potential source of systematic error. The new measurements do not change the initial conclusion. Nevertheless, the observed anomaly in the neutrinos' time of flight from CERN to Gran Sasso still needs further scrutiny and independent measurement before it can be refuted or confirmed.
For example, there were small errors in the Met Office's climate change software that I detected. The scientific papers were correct, but the translation into code was not. This could have happened here and it would be better if they released the code.
http://blog.jgc.org/2010/02/something-odd-in-crutem3-station...
Furthermore, I doubt anyone at OPERA is in a position to even fulfill your request. The minimum number of separate systems with exclusive sets of code is three, and probably more than ten.
However, physicists do make extensive use of unit tests, so that may be some consolation.
Ehm, well some do. And those that do, endlessly complain about most of them that don't.
Fact of the matter is, most physicists aren't trained in proper software engineering methods. Quite a few, but not nearly a majority are very interested in computers and computing so they may pick up a thing or two or actually bite their teeth in doing things the right way, and then there's most of them who love Physics for being Physics and Science for Science and see the computer as just another tool next to their pencil and paper, and just "get things done" and screwed be the next PHD that has to pick up on their research.
At least, that's what I hear from a friend doing a PHD involving analysing huge datasets coming from apparatus measuring high energy cosmic particles.
Of course the LHC is a much more prestigious project, so you might expect somewhat higher quality, but you can't change the culture. Physicists are generally not Software Engineers.
At the very least it should be as available as the papers themselves.
Laziness, lack of time, and the fact that most scientific code is not suitable for public consumption. For instance, my code includes error messages such as "What the fuck????" and "This never happens" which I'd need to take out in order to prepare it for publication.
In addition, there's the assumption that source code is a pretty trivial implementation detail; we publish the algorithm but not the details of the implementation, just as the experimentalist tells us what he built but doesn't tell us what brand of screwdriver he used to build it.
One "trivial" detail in the implementation, one tiny little floating point rounding error, can throw the entire thing. We all know this. So why is the scientific community so unwilling to face this elephant in the room? And why is nobody else confronting them about it? Of course there's jgc, with his admirable record of confronting people about things, but that's the only example I can think of.
You would've thought the "Al Gore made up AGW to devalue my exxon shares" angst-brigade would've jumped on this, but they seem more concerned with digging through departmental email gossip.
If you use a different implementation of the algorithm, on the same data, and get a different result, you will find that there was an error. That's a lot easier than combing through someone else's code base.
A simple example: If Lab A says they took the square root of 16, you don't need their source code to know there's a problem, if their result is 5 and your result is 4. If you use their code, you might not notice that it's borked.
Again, there is no reason to be embarrassed of comments like "this should never happen". My own code has parts that raise "ThisShouldNeverHappenError" when something that should never happen disregards my opinion and happens anyway. ;-)
If the first lab used C++, and the second lab used Matlab and its libraries, and the third lab used trusted code based on NumPy, I would be surprised if their errors were strongly correlated. Brand new code? Sure. But not code they've used for several years through multiple experiments, or code that's part of a widely-used software package.
The thing is, labs working in similar areas most likely have ready, tested, trusted software toolkits for doing similar tasks, especially when it comes to data analysis.
It may seem to a programmer that the fastest, easiest thing to do would be to read the code, in fact the fastest, easiest thing to do is probably for another lab in the same field to do an independent analysis of the data with their own code.
Further, the Kolmogorov complexity of a simulation is typically a wee bit larger than the Kolmogorov complexity of a square root implementation. That metaphor is beyond useless, it's deceptive.
Not at all. The questions are 1) does the method described sound reasonable and 2) if I implement the same methods, with my toolset, and process the same data, do I get the same result?
If the results are different, using different implementations of the same methods, then it's possible that either party could have the error. But if the second party is reusing well-tested code of their own, or something from Matlab or whatever, then it is more likely that the unknown code of the original lab is the problem.
You might have a point about a simulation, but an awful lot of science isn't simulations, it's analyzing recorded data.
(Ex: http://www.plexon.com/product/Offline_Sorter.html#Features)
Given a terabyte of recording data, I'd think it should be sufficient to specify the parameters they used for spike sorting. Someone else ought to be able to use a different package on the same data and obtain results similar enough to know if the first person fudged their paper.
Given a set of data, and a named algorithm, it ought to be possible to obtain the same result with any correct implementation of that algorithm.
Is your paper nothing but off-the-shelf "spike sorting", or is that used as a component of something larger? Rolling back around to the original point of source code release being a desirable thing, if it's the latter, just knowing that this one library was used from what is probably literally a single sentence in the paper that reads "We used spike sorting software X to obtain sorted spikes", when presumably the library has knobs whose settings you don't know and you still don't even have their raw data to check the settings with, you still know virtually nothing about what was actually done. Source release ought to be standard.
The degree to which people who are putatively scientists will go to bat to defend making it difficult or impossible to replicate their experiments boggles my mind.
Besides, the code is almost always worthless compared to collected data. What would actually be most valuable and practicable is for groups to be running their proprietary code on each other's public data for confirmation.
Take the data, apply the described algorithms, get the same result. If Lab B doesn't get the same result, and Lab B has confidence in their implementation, then there's probably something funny with Lab A's code or data.
Reusing Lab A's code on Lab A's data just reproduces Lab A's errors, if any.
However if I can just look at Lab A's code and spot an error in it, that will allow me to discount their results without having to redo their experiment, potentially saving years of work and millions of dollars.
Maybe that's what researchers are really afraid of?
In machine learning research, for example, most evaluations consist of running a new program on soem data, getting results back, and from these results computing some aggregate measure of performance. A bug on the code that computes this measure of performance is _really bad_ and can invalidate all your conclusions. If that code is right, however, a but on the code that trains your model is completely meaningless, because as long as your results are good you can argue that you actually meant to write a paper about the model actually implemented rather than the model you were supposed to implement.
I'm sure other scientific areas have similar distinctions, and a naive code reader might fail to notice that a bug is harmless (and there's also the fact that scientific code carries within it a lot of assumptions about the data which, if broken, can be buggy, but are not broken by real data).
Or you could just analyze their data using their described methods and your own implementation.
Also, a lot of the code may well be specific to the apparatus used, which may be unique to a lab, consisting of custom-built hardware.
If biologists could copy mice from other experiment to give them their own drugs and compare results, they would surely do this along with using their own mice.
The more ways to compare experiments, the more sure we will be about results.
I understand cleaning up code is a lot of work, so don't do it, just release it as it is. I know it will be ugly, will include obscenity, etc. If somebody need it, it will do the rest of work.
No. The more exactly you duplicate the original experiment, the more likely you duplicate any confounding issues. If someone observes something through a telescope, it's best to verify it through a different telescope somewhere else, to exclude the possibility of an optical quirk of the first telescope.
Some other people will run the same code on different data (maybe from simulation, or when checking their own theory). They will get some other results, or the same.
Some people will recreate experiment from scratch, and get code B and data B.
Some people will run code A on data B. Some people will run code B on data A.
Now we can't do that. The code is already written, it's waste to hide it, it's not like publishing code prohibit other people from writing their own code.
That's how you check for errors - by changing as little as possible and observing results.
In a sense, they actually do.
Saying you used a particular strain of mouse is like saying you used a particular algorithm on your data. Asking to use the exact individual mice used by another lab might not reveal an error such as mixed-up strains.
Besides, they're only inbred strains, not clones.
How does that come about? Patent shenanigans?
Pretty much anything 'lab grade' is going to be expensive. The lab I worked in ordered a lab grade laser for optogenetics experiments, and that cost something like $18k. Presumably that gets you a very precise, stable wavelength.
Somewhere, somebody screwed up... and if the problem is in my code then I want to find it and correct it myself, rather than have it found and corrected by some random stranger who emailed me, and will then post the bug all over the internet so the world can point and laugh at the exact line of my source code that sent the world on a wild goose chase for FTL neutrinos.
So I don't blame this guy for not sending you his source code.
The problem is that suppose it is a bug and suppose the code is being kept behind so they can find the bug themselves. A huge amount of time and money is wasted whereas a programmer could look at the code and compare with the paper pretty quickly.
Also in the case of high-energy physics its not exactly easy to duplicate experiments because of the eye-watering cost of the equipment.
Maybe. It might be faster and easier to analyze the data using their described methods and your own code and see if the results are the same.
Some code has to be trusted, and scientists can't check every assumption every time, so I don't blame them. But why don't they allow others to check the code? It's the opposite of scientific method.
How is that any different from a random programmer assuming the physicists are incompetent and presuming that what is really necessary is a code review?
EDIT: publishing code don't require dismantling equipment. It's not my fault electrician isn't very good analogy.
EDIT: True, not a great analogy.
That said, a brief search turned up this paper ( https://facultystaff.richmond.edu/~ebunn/vanelburg.pdf ) which argues that that explanation is faulty (i.e., the paper made a mistake, and when corrected, the original researchers' results stand).
(Disclaimer: although I once read a book on this stuff, I shouldn't be confused with an expert, and I have no idea who's right and who's wrong. But it is exciting.)
IIRC someone from OPERA responded to questions about that paper by saying, basically, that it wasn't even worth debunking because they had real work to do, and publicly disproving every amateur with a theory would suck up all their time.
[1] http://www.astronomy.ohio-state.edu/~pogge/Ast162/Unit5/gps....
"Particle Physics for Non-Physicists: A Tour of the Microcosmos"
http://www.thegreatcourses.com/tgc/courses/course_detail.asp...
But measuring the simultaneity of an event between two different points with a moving clock is a lot trick.
But they weren't. By my understanding, they were merely using the satellite signals to synchronize the ground clocks in the ground frame, and after that was done, the satellite's were never heard from again.
It discusses how the distance traveled by the neutrino beam was calculated. It covers GPS measurements, tidal effect considerations, the Sagnac effect [2], etc.
[1] PDF: http://operaweb.lngs.infn.it/Opera/publicnotes/note132.pdf
Now we have neutrinos that may travel faster than light. That would throw Special Relativity out the window, sure, but not the locality principle: those neutrino do not show infinite speed, unlike quantum entanglement under the Copenhagen interpretation.
To his deathbed he believed in his alternate explanation that the particles were set opposite "at birth" and initiated that way until observation (if I understand that correctly).
So now we have at least TWO phenomena that appear to be "faster than light" - so get cracking scientists (and engineers) on a practical application for communications to probes that are light-hours or light-years away.
I'm still trying to keep my expectations down for now, though.
For what it's worth, this part of the first paper is pretty accessible - at least, as to how they did it - it's more difficult to poke holes in how accurate it was :)
1) The distance between the labs is smaller than they think
2) Tachyonic neutrinos (not likely)
Another possibility that occurred to me (which is probably way off, and has likely already been trivially disproven by some simple experiment that I'm not thinking of at the moment) is that the "speed of light" that we tend to measure is actually a bit lower than the geometrical "c" that appears in equations (I'm not sure what the state-of-the-art is on measuring that geometrical quantity directly), because for some reason photons are slowed as they move through the vacuum (similar to how we can very easily slow photons by making them move through glass, except this would have to be via interactions with the vacuum or some other form of matter). It could be that the vacuum itself has an index of refraction for photons not quite equal to 1 due to some unknown interaction.
Neutrinos interact with almost nothing, so it's conceivable that they even if they have a small amount of mass and travel slower than geometrical "c", they could still end up going faster than photons because they slip through background interactions with ease.
Again, this is wild speculation, "c" is used all over the place, and if we had the wrong value for it, that would require a lot of coincidences, that this sort of vacuum index-of-refraction effect also had the side effect of making other measurements involving the speed of light (energy measurements, for instance - I would tend to think a discrepancy would be way more obvious here, involving, as they do, factors of c^2) come out looking correct, and that seems unlikely to me.
Then again, so much of what we directly measure is grounded in e/m interactions, so there's a chance (vanishingly small, as it is) that a fifth decimal place discrepancy in photon-in-vacuum-c vs. geometrical-c could escape notice because of the relative difficulty of directly measuring geometrical-c, especially if whatever vacuum effects that slow photon travel had other side effects that dunged up our energy measurements, too.
Not that I understand that.
He said it in his recent "Ask me anything" session on reddit.
They know the distance to within +-20cm, so the 76cm of Earth's rotation would be very significant (but not significant enough, again with my rough calculations, to bring these results back in line with c).
The distance is significant enough that to get an accurate measure of c over that distance you would have to take it into account. ie. it would be obvious
I suppose my question is - all things being relative, why aren't the neutrinos bound to the same frame as the emitter and detector?
I also see now that I misread the grand-parent post. Of course these motions would not affect the measurements, and if they did, the rotation of the earth alone would be obvious.
I'm sure that the effects are tiny for the LHC, but are they accounted for at all?
Actually no. The speed of light is constant in ALL frames of reference, whether inertial or not. The math just gets more complicated in non-inertial frames. (That's the difference between special and general relativity.)
UPDATE: it's actually a little more complicated than that. See http://en.wikipedia.org/wiki/Propagation_of_light_in_non-ine...
> I'm sure that the effects are tiny for the LHC
Actually, gravity is a pretty significant factor. See e.g.:
http://www.astronomy.ohio-state.edu/~pogge/Ast162/Unit5/gps....
The sense I've gotten is that the most likely culprit is some sort of systemic error in the software somewhere, as that stuff is the most labor-intensive part of the experimental setup to analyze for errors.
4) High energy neutrinos spontaneously teleport themselves (casualty saved, new, very odd physics needed)
...
n) Subtle measurement problem (impossible to falsify until the result is replicated on independent setup)
Physics, as I'm sure you know, has undergone many revolutions in its understanding of the world. This could be the start of a new one, or it could fizzle and be a quirky measurement error. In either case, highly worth following to see what happens.
But while it's not unthinkable that relativity requires revision under some circumstances, it's one of the most thoroughly and repeatedly proven tenets of modern physics. It's going to require something much more definitive and reproducible to overturn that mountain. I wouldn't expect a positive answer to this inside ten years, but a negative answer could turn up any time since it depends on someone finds the bug in the gears.
Many problems in science are considered to be "twenty year problems," by which we mean that they'll probably be solved eventually, but that it's unlikely to be any time soon. Many such problems go on for much longer without a solution despite significant progress (see also: fusion power plants). This isn't quite on that scope - but it could evolve into one if it actually ends up with a result which can be reproduced on command.
There are actually three types of neutrino (and they spontaneously change from one type to another), they are all spin-one-half, and they interact with other matter only via the weak nuclear force, which has a much smaller effective distance than the electromagnetic force, so they don't interact very often. They have non-zero mass, but it is so small that we don't actually know what it is.
Everything in the universe exhibits the wave-particle duality.
Einstein's self professed "biggest blunder" was the cosmological constant. That term has never referred to the assumption of lorentz invariance.
I have seen Apache ignore updated PHP files, and serving old code, which should never happen. This a real bitch to debug.