Hacker News Reader

Comment by im3w1l | Hacker News Reader

Is she suggesting us throwing out Occam's Razor? That's pretty radical.

On the contrary. She suggests adhering to it. The notion of whether a theory is elegant or beautiful is exogenous to the question if it describes nature, hence should be cut by the razor.

In technical terms, there is no need for a prior on theory space:

https://arxiv.org/abs/1801.02176

---

5.2 Occam’s Razor

A probability distribution from which to calculate the most likely choice of parameter adds unnecessary structure to the theory and is thus in conflict with the dictum of simplicity. We could have chosen a parameter and be done with it. The probability distribution and all the not-observed values of the parameters are unnecessary for the derivation of any observable and they should therefore be stripped by Occam’s razor.

That seems to be part of it.

However, I don't think disliking parsimony is that radical. Andrew Gelman is not a fan of parsimony.

http://andrewgelman.com/2004/12/10/against_parsimo/

My current view is that parsimony is a heuristic you can use, but weak evidence at best. Plus, parsimony is not unambiguously defined, so comparing hypotheses in terms of it can still be subjective even with objective criteria as there is no agreement on which approach is best. It's most justified to say that extremely complex models are unlikely if the complexity is not necessary.

I’d use another, less charitable, word. Finding the laws of physics isn’t any different than finding a winning play in Go, except the search space is vastly larger. If we aren’t going to pay for things like the Superconducting Supercollider, we aren’t going to pay for a brute-force search to find the laws of physics. We have to use heuristics to guide the search. “Beauty” has worked in the past (Gell-Mann has a TED talk on this) so there’s no reason to abandon it - until it doesn’t work. Then we’ll try something else. In addition it took, what, 300 years to prove Fermat’s Last Theorem? Good grief. For hard problems, 40 years is nothing.

Brute force search isn't what solved Go. What was needed was a model with the right information, not all the information or the most elegant or compressive version of the information, but the right information. What "right" means in this context is what we need to figure out!

To be clear, Go is not "solved". We just have AI they is strictly "very good" at it (superhuman).

fair point, poor language from me there.

What is special about Occam's Razor? Why not get rid of it? Do you know why Scotus and Occam and the rest of the gang came up with it? Very few people have actually thought about why and what it was for, so why do so many people cleave to it?

Is that a rhetorical question, or do you really not know?

I have strong ideas that are somewhat better informed than most people I talk to, but then again I am probably talking to the wrong people. As far as I know Occam's Razor was invented 800 years ago to solve the problem of the holy trinity. Basically medieval philosophers were stumped by the number three - one was fine; unity was perfection... but more than one was a problem - why would god have a certain number of aspects? So school of Scotus formulated the idea that the number of entities was necessary; just right because God knew what was up with the universe and therefore would produce aspects to necessity. This brought in the idea that there were elegant and sufficient numbers of things in nature which was quite contrary to the old way of looking at the infinite and mystical aspects of the world, inspired by ideas of real numbers and right angled triangles with length 2 hypotenuse. This idea has been a handy heuristic for science (not maths) ever since, but it has no actual basis apart from avoiding getting put on a bonfire if you were a Christian philosopher in the thirteenth century ( I think that William of Occam was nearly burned in Avignon, but talked his way out of it).

When people use Occams Razor today they typically mean "prefer the simplest, most elegant explanation". The reason we have it is that for any kind of observation you can make there are an infinite number of different theories that could explain your observations. So we need some kind of rule that tells us which of these theories we should prefer.

It's not only a heuristic though. If you have a scientific theory modeling 10 parameters, but a theory modeling the exact same phenomena can be formed using only 7 parameters, the 7 parameter theory is preferable for a number of very rational and non-aesthetic reasons.

I think it's an eye grabbing title, but not really honest, as it changes from 'beauty is bad' to 'why has it failed us'. It's a question worth asking, because obviously answering it would be progress. But, I don't think anyone is actually arguing against Occam's Razor, or that simpler question solving more, is the goal.

I read the article, and I don't see any insightful answers, perhaps we're supposed to buy the book? Or perhaps people should devise stricter criteria for theories to be experimentally demonstrable...

Why has beauty failed us? Such mathematical beauty is produced by the conscious mind, and the mind is created (via the brain) from the action of the laws of physics. Perhaps it's because the self-referential circuit Math -> Physics -> Mind -> Math can't exist that Math can't fully explain Physics, and e.g. GR and QM will never be united via a "beautiful" mathematical theory.

I don't think it's failed us in a sense that there is no finality to progress.

Reply on news.ycombinator.com