Hacker News Reader

Comment by Robotbeat | Hacker News Reader

This is a misconception. It’s more fundamental than that. There’s a fundamental connection between (Shannon) information theory and thermodynamics. The Landau Limit, whether blackholes can destroy information or not, quantum mechanics, etc.

Information is actually tangible. It’s not just an analogy or a coincidence that the word “entropy” is a word used in both physics and computer science (information theory). Thermodynamics, mind you, is perhaps THE most fundamental elements of physics and how the universe works.

View on HN

Information and computation are not the same thing. I was very specifically talking about Turing machines and other theoretical models of computation.

That said, you are still making the same mistake I pointed out, elevating human symbolic information to a higher plateau than it deserves. I think it's because you're being too vague about the connection, when it's fairly mundane. The "fundamental connection" is that physical quantities are information, and on the other hand information is always a large collection of semi-independent semi-stochastic physical objects and can be profitably modelled by some sort of statistical mechanics. Information theory is relevant all over physics because human agents collect all sorts of physical information. The universe "doesn't care" about information in and of itself. Shannon entropy and Boltzmann entropy have similar formulas because they measure precisely the same thing; put another way, a goofy but formally equivalent way to model a gas would be a noisy radio channel communicating each molecule's kinetic energy.

The problem with the black hole information paradox isn't that information is destroyed per se, but that it appears to be destroyed in a way that violates quantum mechanics (destroying quantum state without a measurement). The theoretically predicted destruction of information points to a more general problem.

The Laundauer limit is no more fundamental to the universe than "a mechanical crane cannot violate the laws of pulleys." It says that no matter how you design your binary (or whatever) computer, it must involve an ensemble of binary states, and statistical mechanics puts an absolute floor on how little heat is required to alter such states. Whether these states are gas molecules or the written symbols "0" and "1" is immaterial.

> Information and computation are not the same thing.

Shannon information, sure.

However, algorithmic information (Kolmogorov complexity, etc.) is based on computation.

But that just takes us even further from physical relevance / universal fundamentals. Kolmogorov complexity is fundamental in computation, but its relevance in physical science is pretty selective.

I’ve always hated the use of the word “information” in relation to things like spin.

Information and order are effects of human perception and preference. They exist as abstractions in the mind and not in reality.

Best way to show this is to implement information in multiple physical substrates. Make a book out of paper and make it out of clay. Same information. The physical substrate doesn’t matter.

A parallel concept to this is from functionalism/theory of mind in Philosophy is that of Multiple Realisability

It’s not as simple as for example degrees of freedom and the states for a system is “information” and those one can argue are physical.

This. I'll never understand people who reify "information" as though it is some object with independent existence from man.

No information without an interpretation. The "amount" of information is completely dependent on the observer.

You'd think people who work constantly with abstraction wouldn't fall prey to reifying abstractions but they actually seem more susceptible to it than anyone else.

I think you are prematurely dismissing something deeper here.

The more I learn about the fundamental nature of the electron, probability in quantum mechanics, and the wave function in general... the more information being fundamental substrate makes sense.

I'm not saying it is... just that it makes more sense the deeper you get.

Yeah, but my point is that "information" is not really a substantive concept. It is an abstraction. It is a useful technical tool. For the record, I am a nominalist when it comes to mathematics. Thinking that information is some "real thing" with meaning independent of theoretical context and interpreters is a definitely Platonisist perspective (the idea that our mathematical representations are not merely useful technical models but that they correspond to some real substantive elements of the real).

Claiming that "information" is the underlying basis of the real is the same as effectively saying that "stuff" is the underlying basis of existence. It's essentially Platonism. It is not informative. What is informative is the operational use of the mathematical concept of information as defined by Shannon to understand real things or to structure theories. But this is a very different thing than identifying a substance that exists, independently of us in the universe. This "making real" of what is ultimately a mathematical abstraction is precisely reification.

This stance is very tempting because it appears so straightforward and no-nonsense. But there are issues which undermine it in my mind. Because, at a closer inspection, it itself doesn't seem informative, in the end coming down to a question of semantics, i.e. how we define the word "reality". Which then determines whether it encompasses the realm of mathematics or not. I wonder what definition you favor, but the language you use points towards something like "the totality of all things" or "everything that's made from a physical substance". Yet platonists and other realists wouldn't argue that numbers are made from a substance as you imply.

Physical reality is a slippery thing. Yes, at the level of our daily life it all seems very solid and obvious, but inspecting that foundation revealed a bewildering world of relativity and uncertainty. And one of the most intriguing ways out of this bewilderment seems to be the "it from bit" trend in physics, which makes information the fundamental reality.

I personally don't feel the need to tie the notion of reality to physical phenomena. I prefer something more along the lines of Philip K. Dick's "reality is that which doesn't go away when you stop believing in it", or, as you put it, "something which exists independently of us". The fact that there are an infinite number of primes, is such a fact. Surely any alien civilization would know this fact. Maths doesn't go away when we look away, and neither does the notion of computation. Sure, Turing machines are a human construct, and there can be others as well. But the Church-Turing thesis speaks about how these constructs all ultimately describe the same underlying reality.

"it from bit" is just "shut up and calculate", positivism. There are no bits in physics, and no boolean phenomena.

Let me try again. We use miles to measure distance. Hopefully you agree with me that, miles, being a system of measurement to express relationships of distance, have no ontological existence themselves. I would find it completely bizarre, for example, for someone to argue passionately that, "no miles are the real deal and something that exists in the universe and meters are a fabrication". Distance surely is a real phenomenon, but "miles" are a system of expression.

Bits and information are the same thing. They are a system of measurement to express relationships of difference. Difference and change as physical phenomena exist, but bits do not.

So, yes, pointing out that the limits of computation give us some useful facts about the way things change in reality is a useful perspective. Going further and saying something like "computation is the basis of reality" or similar extremes is too reductive and too abstract to really be a useful piece of knowledge in my view. Finally, to claim that information is somehow the important fundament of existence is patently absurd to me, since it would've similar to arguing that miles have some very special epistemic and ontological status.

That's just it though.

Yes, separating symbols/descriptions from the physics is a common conflation, rightfully avoided and called out here, but I think you might be overcorrecting unconflating the concepts too much.

Quantum information is strange precisely because the theory treats information-like relationships as physically consequential.

Entanglement, path dependent stuff, cloning theorum, black hole information paradox... There are so many areas you run into where its information-centric descriptions appear to fit unreasonably well.

Again, I'm not saying the symbolic representation is "real" per se (not saying it's not either, I simply don't know what it is).

I agree bits are just nomenclature, but I don't agree that "bits" and "information" are the same thing. Like distance, "information" has a deeper and general architecture to it.

And it's also weirdly physical in places that I wouldn't expect it to be, for reasons I simply don't understand at the moment.

I don't know enough about the quantum level to really understand whether or not that would change my view of things.

From Shannon's theory, it's abundantly clear that "information" is just a mathematical description of difference and expectation. I can frame anything as information. If I know who my parents are, a statement about who they are is not informative to me because it constitutes no difference. The entropy and expectation/surprise is low. If someone approaches me with a compelling argument that no, actually, my parents are not my parents I was adopted and my real parents are so and so, this is informative. The entropy is high.

Crucially, this all depends on me my personal history, and my status as an observer. It is in this sense that I find information to be a communications concept (which is how it started!) or semiotic concept more than a useful abstraction for organizing our knowledge about other parts of reality. Yes, the abstraction might still be useful if e.g. quanta end up behaving in some way according to shannon's laws, that is interesting, but to me uh at is not an argument that turns information into some kind of "universal substrate" of the real—it is just a surprising applicability of laws in one domain to another. Is it surprising that empirical data fit the model? Maybe. Idk, lots of empirical data can be made to fit all kinds of models (there are ways in which hI can reasonably take a Newtonian view of various aspects of my life and the data generally won't contravene the laws).

> From Shannon's theory, it's abundantly clear that "information" is just a mathematical description

That's a very narrow and technical definition of information. It's what people refer to by the word when they discuss things like: communication, medium, surprise, compression, etc. in a technical context. Otherwise one should refer to the dictionary definition, so we don't end up talking past each other.

If you'd like to get a better intuitive sense of what "it from bit" points at, check out David Deutsch's Constructor Theory and the Stephen Wolfram Physics Project. Also, Bernardo Kastrup, on the philosophy front.

> Finally, to claim that information is somehow the important fundament of existence is patently absurd to me, since it would've similar to arguing that miles have some very special epistemic and ontological status.

As the sibling comment says, no, it is like arguing that distance/space/extension has a special ontological status — which it does. "It from bit" is just using the word "bit" for the rhyme, the theory is quite profound and compelling, at least to me.

Then there's also Max Tegmark's "Mathematical Universe" idea, which makes the symbolic world the fundamental one. It goes a bit far for my taste, but it has its own internal coherence.

This is the way. But what is interesting to contemplate is _why_ smart people try to reinvent religion. I think it reflects their deep need for meaning and immortality. Religion in the naive sense, a bunch of stories, myths and bearded gods, if obviously silly and childish. The modern rationalist who has not managed to shed his reflex/need for religion, then reifies (or perhaps better said... "deifies") information, and thus introduces all kinds of potential immortality and an ideal realm. Just like the myths of religion did, in a more naive sense, since time immemorial.

Landauer’s Principle (a direct consequence of the Second Law of thermodynamics) shows there’s a very clear physical manifestation of information and information theory, it is not just human perception and preference. https://en.wikipedia.org/wiki/Landauer's_principle

A more concise explanation is that, computation is the transformation of information. Any transformation of information is computation, and is thus subject to some theory of computation. A physical process involves the transformation of information, so can be studied with a theory of computation.

> A physical process involves the transformation of information, so can be studied with a theory of computation.

A rock rolling downhill has state-dependent future behavior you could describe informationally, but nothing is gained by modeling it with automata theory instead of Newtonian mechanics, and it isn't computing in any substantive sense.

Quantum mechanics is pretty fundamental but you wouldn't use it to describe a rock rolling down hill either.

> It’s not just an analogy or a coincidence that the word “entropy” is a word used in both physics and computer science (information theory).

Shannon asked von Neumann what to call it, and von Neumann said "You should call it entropy, for two reasons: In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage."

I’m aware of the von Neumann joke.

There is, however, a direct equivalence between them derived from the 2nd Law of Thermodynamics. https://en.wikipedia.org/wiki/Landauer's_principle

Um, only the second part was a joke. The first part was confirmation of your statement that it's not just an analogy or coincidence ... and Shannon followed von Neumann's advice (not joke).

Thank you. I’ve been thinking about this for a while now and couldn’t express what I was thinking about… but when you linked information theory to thermodynamics, it clicked.

Given the quantum fields were collapsed in the early universe, I’m now wondering how information theory looked like during the different phases on the universe

You can say things like, “In the domain of physics, X,” “Assuming the scientific method, X,” or “Assuming the premises of logic, X.”

But “computation is a fundamental aspect of the universe”, in the way it's being understood in this thread (as opposed to the article) does not remain within any of those frameworks. It makes a claim about reality as a whole.

Once you make that kind of universal claim, all the assumptions built into “computation” - about identity, distinct states, lawful transitions, causation, logic, etc. - become part of the claim. You cannot use those assumptions silently and then present the conclusion as framework-independent.

And those frameworks come with consequences. Using the language of computation, they come with "lossiness" wrt modelling "reality".

(see Aristotle Metaphysics -> Kant CoPR -> later Wittgenstein)

The Heisenberg Uncertainty Principle and the Laws of Thermodynamics are pretty d*mn fundamental.

The Church-Turing thesis (the fact that all models of computation we've been able to come up with are equivalent) gives the necessary weight to that claim.

The Church-Turing thesis is a statement about symoblic processes, and symbolism is simply not fundamental to the universe. It is fundamental to how humans understand the universe. That's the distinction.

Did you mean landauer?

Yes. Unable to edit it. https://en.wikipedia.org/wiki/Landauer's_principle

Reply on news.ycombinator.com