Hacker News Reader

Comment by im3w1l | Hacker News Reader

So one philosophical issue I've had with entropy is that it is based on probability. But it is applied even to non-quantum systems that evolve in a deterministic fashion. How can we talk about the uncertainty of a system that is in a single, well known, predictable state? And with many-worlds interpretation, even the quantum wavefunctions evolve in a deterministic manner.

How can we reconcile this? We can say that the system is not probabilistic, but it is approximately probabilistic. If it is only approximately true, does that mean we can find a loophole in the laws of thermodynamics?

View on HN

> How can we reconcile this? We can say that the system is not probabilistic, but it is approximately probabilistic.

It is indeed probabilistic based on your existing knowledge of the system. The idea that (say) a roulette wheel evolves according to deterministic laws of physics is not useful to you since you do not know the current micro-state necessary to make the prediction.

This is similar to the difference between frequentist and Bayesian interpretations of statistics. Bayesian statistics is more comfortable taking probability distributions over notional facts (such as 'is this coin unfair/weighted?') because it treats the experimenter's confidence as the fount of probability.

If you treat entropy in the same way, it makes more sense. The entropy of a system is related to the number of microstates it can occupy consistently with our known macrostate (e.g., measurement of temperature and pressure -- inherently averaged quantities).

The issue I have with this is that the system is in a given microstate. We may not know which one, but there is one and exactly one. And it will evolve deterministically from that one. And if we look at how it evolves it may do things that seem very unlikely from a random macro point of view.

Assume for instance there is a macrostate with a huge amount of energetically favourable microstates. But our little microstate could be destined never to enter any of them.

Typically there is an assumption right, that any state could transition into any state at a probability given only by the energies but that isn't really true.

Grandparent is right.

Entropy is best thought of as a two-parameter function.

Not entropy(system), but rather entropy(system, your knowledge of system).

The system is in a microstate, but you don't know which one. If you did, it would have zero entropy. Since you only have a few general measurements like "one litre of gas at room temperature and one atmosphere of pressure", the entropy is much higher, because many microstates meet that description.

If you think of entropy as an objective property of the system regardless of a given observer's state of knowledge, you'll likely get confused.

How can we connect this subjective entropy back to things like Gibbs/Helmholtz free energy, or temperature (du/ds)?

This way!

https://bayes.wustl.edu/etj/articles/theory.1.pdf

Entropy is only kinda about probability.

>How can we talk about the uncertainty of a system that is in a single, well known, predictable state.

The system we are usually talking about is orders of magnitude larger than the uncertainty. Think a liter of gasoline in an engine vs the position of a single atom. While we might not know the trajectory of the atom, we have a pretty good idea how the collective atoms that make up the gasoline will function.

> we have a pretty good idea how the collective atoms that make up the gasoline will function.

Yes, and that idea is based on theories assuming things are probabilistic. Thus bringing us back to my second paragraph. That falsely assuming things are probabilistic works very well in practice. But it may break down under certain conditions, and there may be loopholes.

Edit to add:

I think this whole thing is very related to pseudo random number generators and keystretching. Maybe computability theory too. If you have a 1gig file generated from a 128bit seed, then the apparent uncertainty (1gig) is much higher than the actual uncertainty (128bit). But since it's so very hard to undo the prng, it will behave very closely to something with 1gig of actual uncertainty.

Edit to add again:

A fun thought experiment is someone "encrypting" chemical energy into heat, putting the key in a box, only to later go back and decrypt it back into chemical energy.

Systems where there are multiple sequential collisions are not fundamentally deterministic. There are quantum-mechanical effects in the dynamics of the collisions between the electrons in the atoms, whose positions are thoroughly probabilistic in their nature.

Quantum physics is not compatible with determinism, so any dynamical system is only precisely describable in probabilistic terms,

There are deterministic qm interpretations though so this can't be absolutely true.

You're not wrong, but the connection's already explored well; we call it "chemistry". Cook a piece of chalk and it'll crumble into powdery lime; mix it with water and air and it'll heat up and set back into hard stone.

The bigger idea here is that, while it's highly improbable that heat is applied to chalk, we can choose to apply heat to chalk. We have the choice of where to direct our energy. This is, by the Free Will Theorem, the same choice that subatomic particles make about how their energy will be measured. The loophole is precisely that if enough particles choose to synchronize their choices, then they can make highly structured choices with rich data for a long period of time; they can integrate information.

When we crumble the chalk, we lose the specific information about the shape of the chalk, and it would be very hard to recreate; this is entropy. But we also gain the specific information about the shape of the cement, and this would be very hard to determine from the prior context of the universe; this is choice and integrated information.

I think this is not what I meant. Let me restate your point as I understand it with a more well known example.

Melt a piece of ice and it will turn into water. Cool it and it will turn back into ice. The bigger idea here is that, while it's highly improbable that heat is applied to ice, we can choose to apply heat to ice. (...)

When you are heating the ice, energy and entropy from the environment is stored in the ice. When you are freezing it, energy and entropy from the water is transferred away to the environment.

This is a different thing from what I was talking about.

> If you have a 1gig file generated from a 128bit seed, then the apparent uncertainty (1gig) is much higher than the actual uncertainty (128bit).

More than that, since you also need to specify the algorithm used for the prng. Your true entropy is the 128-bit key plus the Kolmogorov complexity (https://en.wikipedia.org/wiki/Kolmogorov_complexity) of the generating algorithm.

This is also a good example of how information-theoretic entropy can change based on our prior knowledge. If we already know the generating algorithm, then the entropy is just the 128-bit key.

> But since it's so very hard to undo the prng, it will behave very closely to something with 1gig of actual uncertainty.

No, not really. A one-time pad with 1Gb of actual uncertainty is not made weaker by disclosing 128 bits of entropy. However, 1Gb of PRNG data is made weaker (broken) by disclosing the 128-bit key if the algorithm is known.

We can practically equate the two because inverting the 128-bit system is just as impossible (absent a quantum computer large/fast enough to take 2^64 operations) within the age of the universe as inverting the 1Gb system.

I think the concept I mentioned, apparent uncertainty, can be formalized by referencing efficient algorithms. Something like "no efficient algorithm can tell apparent uncertainty and actual uncertainty apart".

Sure, but the thought experiment is more that you write down 128 bits, generate the 1gb, then eat the paper containing the 128 bits. You now have 1gb that behaves a lot like it is 1gb of entropy, even though it still isn't.

>A fun thought experiment is someone "encrypting" chemical energy into heat, putting the key in a box, only to later go back and decrypt it back into chemical energy.

Unfortunately, I think it's less like encryption and more like compression. And it's actually less like compression and more like deletion. It's impossible to compress information past a certain point (right?), and the amount left in heat seems like it's just been outright deleted.

The Universe wants to debloat its codebase of information: https://youtu.be/8N1BxHgsoOw?t=119

You have a common misunderstanding. Heat contains a lot of information. Chemical energy contains almost none. That's a problem because you can't destroy information, so you can't go back to (much more useful) chemical energy.

But if you have "illusory heat" that seems to contain a lot of information, but it's actually just a tiny amount of information (say 128 bit) that has been used stretched into a lot (say 1gig), then with the right key you can see that it actually is not a lot of information. And then you should, in theory, be able to go back to chemical energy.

Reply on news.ycombinator.com