Hacker News Reader

Comment by dr_dshiv | Hacker News Reader

There seems to be a great, unresolved difference of opinion about whether information entropy and thermodynamic entropy are commensurable.

In trying to connect Shannon entropy to thermodynamic entropy, I always get stuck on the fact that you need to have a defined alphabet or symbol set.

View on HN

It seemed clear to me at the time when I read (a few of - but especially https://bayes.wustl.edu/etj/articles/theory.1.pdf) ET Jaynes' papers on classical thermodynamics that they were commensurable. I'm mildly surprised there is some question about that!

The alphabet is to distinguish states. But this is the case in thermodynamic entropy as well, that each state is distinguishable. What is important is quantifying the set of possible states.

vs an arbitrary defined number of dimensions?

Their are more exotic mechanisms to define entropy: https://en.wikipedia.org/wiki/Parastatistics

In a thermodynamic sense, information input (say, of photons) is generally going to increase the entropy of a receiving system. Only certain kinds of information can reduce the entropy of the receiving system. Too much information (i.e., too many photons) can literally burn your eyes.

I don't see how to square those manifest physical effects with the immateriality of Shannon information entropy.

Is this not the problem of Maxwell's Daemon, if an intelligent agent observes and interferes with a particle system to reduce the system's entropy then overall entropy is still increasing because the agents needs to store and calculate information in some physical medium, be it brain matter of silicon.

Not exactly, I don't think.

In the Shannon formulation, there isn't a notion of "too much information" -- it is all symbols.

But in reality, too many photons can burn our eyes or a denial of service attack can break a website. Too much information is a reality --because we are thermodynamic entities.

I don't see how to understand that with Shannon.

Yeah right, and too much heat can burn my toast, forget about global warming it about universal warming, I don't understand that with Einstein either.

Reply on news.ycombinator.com