Our quantum problem
aeon.co
aeon.co
http://www.wired.com/2014/06/the-new-quantum-reality/
Also, having people with expertise in other fields (fluid dynamics in this case) look at the problem is exactly what can create progress.
But mostly the reason is that if you are going to develop a better theory (namely how relativity and qm work together), then it may be helpful to start on a clear foundation where irrelevant confusions have been eliminated.
For example, the role of operators is derived in pilot wave theory, not assumed. This greatly simplifies the issues of putting quantum mechanics on curved space where the Fourier transform may not be so easily defined, if at all. You do have to worry about the Hamiltonian and its boundary conditions, which is part of the physics of the space, but the relevant measurement operators are derivable from the ported theory.
I think the idea of pilot wave theory is really interesting, and that a variant of it could turn out to be more fundamental than our current understanding of quantum physics.
I did a thesis on it which I am quite proud of. But I also left academia proper though more due to my disgust with various aspects of the system unrelated to the discrimination associated with Bohmian mechanics.
To be fair to your point, the successful ones pursuing this either hide out in mathematics departments or keep their mouth shut until well-established.
One of the reasons is that it's considered to be an "Aether Theory".
This goes back to Einstein, whose main beef with QM actually wasn't indeterminism. Instead, determinism was supposed to be a means to restore locality. Bell's theorem tells us this is doomed to fail, and indeed Bohmian mechanics restores determinism only at the cost of locality.
Even though experiments with walking droplets are an impressive demonstration that wave-particle duality isn't necessarily something mysterious, they don't help to address the issues at the heart QM.
Our brains are an emergent properly of the large scale behaviour of the universe. There is no reason we should even assume we're capable of comprehending the small-scale universe's properties, let alone that they should "make sense" to us.
I don't think we can have an argument about whether we can generate a description of the universe that agrees with reality until we agree on what "making sense" means in the first place.
Then there are things which we cannot imagine intuitively, like 5-Dimensional hypercubes, but we can describe them in math quite easily.
Most people can't even intuit basic statistics properly. "Intuition" is no substitute for "theory, experiment and disproof".
Based on previous experiences, I suspect that the universe is governed by relatively simple rules that lead to complex emergent behavior, which would certainly be conducive to our understanding it.
The nature of that fundamental limitation may escape our grasp by definition. The article at least presents the idea that the strange observations of the quantum mechanical universe may sit outside the range of science. I think there's some merit to the idea that humans, not as "willful" entities, but as groups of particles swept up in a cosmic chain reaction, may face fundamental limitations to the "scope" of what we can grasp about the nature of the universe.
Of course, that could be totally wrong as well.
As for systems where it's hard to come up with a complete and consistent model...
- If coin flips were decided by a cryptographic random number generator, it would be intractable for us to extract the seed or even to distinguish the output from true randomness or go-both-ways indexical uncertainty.
- Probably-approximate-correct learning is not possible for all models [1].
- If we're unable to eliminate Boltzmann brains [2] from cosmological predictions, that would seem to imply agents should constantly assign near-certainty to being surrounded by heat death and ignoring that is more of an optimization-for-the-cases-where-you're-not-and-things-matter.
- If the number of rules is larger than the number of atoms in the universe, we're never going to be able to remember them all or write them all down.
1: http://en.wikipedia.org/wiki/Probably_approximately_correct_...
The post does a good job of roughly outlining several interpretations, and explaining why they're each a little weird or at least what the common objections are.
Personally, I'm hopeful that quantum computers will shed light on the issue. They make a lot of large-scale experiments possible, or at least significantly easier.
Gravity has yet to be successfully included
in a theory of everything. ... Theoretical
physicists have not yet formulated a widely
accepted, consistent theory that combines
general relativity and quantum mechanics.
The incompatibility of the two theories
remains an outstanding problem in the field
of physics.
Your momma so fat even if I'd entangle
with her no information would be able
to leave her event horizon.
Nobody has managed to put gravitation
and QM together yet, and you want to
do it in a your-momma-so-fat-joke? Wow.
http://lesswrong.com/lw/q8/many_worlds_one_best_guess/
I'm not sure it adequately supports its claim that "the debate should already be over", but it's worth reading. It's not science, but when all the interpretations other than collapse models predict the exact same observations, there's only so far the math can take you...
The author of this article is in a very small minority. Of course, the silent majority is silent because we generally have better things to do than philosophize.
Just like because we can't predict turbulence, doesn't mean that fluid dynamics is now completely useless.
Science is still useful - both practically and theoretically, even if the universe won't ever let us see past quantum "stuff".
There are a lot of questions we just can't answer until we have the instruments to take measurements at Planck scales, and it's likely that, at those scales, everything that doesn't fit quite right will seem obvious, and everything that niggles at the back of a physicist's brain will be put to rest.
"Shut up and calculate" - physicist David Mermin, on philosophizing about quantum mechanics.
What's the correct (or more correct, anyway) way to mathematically describe reality? Beats me; I'm no physicist.
Seriously. The Standard Model has held up in spite of numerous attempts to break it. Physicists would love to break the Standard Model as it would lead to new, cool stuff.
Even things like hidden variables have been tested for. There aren't missing variables.
Just because something doesn't match your personal experience does not mean it isn't true.
One such explanation might be the wavy nature of spacetime itself: any excitation of the medium called 'spacetime' maybe introduces ripples in that medium, causing the behavior of particles to seem as probabilistic, whereas in reality it is not, it is simply chaotic in nature.
This makes much more sense than the concept of a particle being a wave until the wave collapses.
And this theory ties in perfectly with relativity and gravity: particles create spacetime ripples just like any other body does. With this explanation, there is no need for multiple universes and other strange things. The hidden variables might be the universe itself after all.
The expansion of space time proves there is a medium.
Gravity proves there is a medium.
Frame dragging proves there is a medium.
So, this medium may be the one that causes the ripples and the waves; the particles themselves are not waves in any case.
That said, these problems are probably not going to be resolved by HN posters thinking about things abstractly. They're going to be solved (if they are ever solved) by subject-matter experts working very hard.
I didn't; I wrote that I suspect that it is.
As another notes, the inability to merge quantum theory and relativity indicates that there's something we're not accounting for. Given how comprehensive both of those theories are, I suspect that the fix which accounts for all the observed evidence will be something else entirely.
Ptolemaic epicycles looked iron-clad for centuries; the successfully predicted observed reality for quite a long time, and (IIRC) even when folks started supposing that they might need tweaking, it looked like still more epicycles might save the day.
And then they were replaced with a better theory.
Also QM (in many-worlds form) is literally the most elegant scientific theory ever. It is a thing of staggering beauty, and to my eyes far more obviously true, far less epicycle-ey, than e.g. relativity.