Hacker News Reader

Comment by ThePhysicist | Hacker News Reader

The second law of thermodynamics isn't really a fundamental physical law, but rather a promise based on statistics that says "disorder will increase or stay constant in a closed physical system".

That said, it's entirely possible for entropy to spontaneously decrease in a closed system, the probability of this happening is just astronomically small for typical macroscopic systems.

Example:

If you have a system consisting of two compartments that are separated by a wall, where one of the compartments contains N particles. The system has low entropy because all the particles are on one side of it. Now, if you remove the barrier between the two compartments, particles will distribute evenly on both sides, increasing the entropy of the system. If we assume that the position of the particles is random, the probability of finding all of them on one side of the system is (1/2)^N, which quickly converges to zero for most macroscopic systems, which often contain > 10^23 particles.

View on HN

aren't people (and all living organisms) the quintessential example of a local decrease in entropy that results in greater overall entropy? we are highly ordered groupings of matter but we're really great at churning about the matter and energy around us and we eventually decompose too.

No more so than the wall in the previous example that created a local decrease of entropy.

Sure, but the above point was indicating that even a closed system is capable of decreasing in entropy, according to all other physical laws; it's just that the more complex the system is, the probability of the decrease declines rapidly to near zero.

> aren't people (and all living organisms) the quintessential example of a local decrease in entropy that results in greater overall entropy

Complexity is orthogonal to entropy. The aforementioned barrier-separated box is high in entropy but simple. Upon lifting the barrier, a description of the gas front moving into the vacuum is enormously complex. It is also lower in entropy than the previous, barrier-separated state. Finally, when the gas is diffuse and entropy at its maximum, complexity dips back down.

Life is complex. Plants turn solar radiation into complex living structures; they also diffuse waste heat. Animals eat those plants, make more complex stuff, and generate more diffuse waste heat.

Is there a way to quantify complexity in that sense?

No, because the GP is wrong. He claimed

> The aforementioned barrier-separated box is high in entropy but simple.

This isn't true in any meaningful objective sense; representing the exact state of the system in some basis (e.g. the position or momentum states of all particles in the system) requires a huge amount of information, corresponding to the entropy. The GP only said it's a "simple" system because we (for arbitrary reasons) don't really care very much about the position of each gas molecule. We as humans are satisfied with describing the system in terms of bulk statistical characteristics like temperature and pressure. However, we would be unsatisfied doing the same thing with a similarly information-rich system such as a microprocessor, because now the bulk characteristics of the system aren't sufficient information to describe the characteristics that humans care about.

"Simplicity" or "complexity" as described by the GP is not a physical quantity; it's a reflection of which precise dynamical behavior we happen to care about. You could probably find all sorts of heuristics that generally match with human intuition, but in the end it's up to opinion.

Shannon entropy is the expectation of Kolmogorov complexity, although I don't remember how Shannon entropy relates to Boltzmann entropy.

Yes absolutely! Life is actually a way to increase entropy as it harnesses chemical energy in systems that otherwise wouldn't be able to reach a lower energy state by themselves.

Isn't it the other way around? Life decreases local entropy?

Yes but globally entropy increases faster than would be possible without life :)

A living organism is of course a noble endeavor standing against the tide of entropy. But it can do so only with significant influx of energy extracted from its external environment. The cost of constructing such an elaborate order of matter is paid by disorder elsewhere.

As we know, the living stance can be maintained for only a relatively short time, after which it rapidly decomposes to background entropy.

Interesting to speculate, as living beings we are chock full of "micro-environments", there could be exceptions to the laws of nature as we've understood them that could account for certain mysteries of biological nature.

Any living organism is high entropy there are considerably lower entropy states for the molecules that make up your body to hold than the arrangement that makes you, you.

A living organism also increases the entropy of the environment.

We're not standing against the tide of entropy. If we're going to use a water metaphor, let's talk about a dam or a water wheel. Life is capturing energy and redirecting it-- we're still increasing entropy, but we're directing it in such a way that allows us to have some semblance of order. We aren't trying to resist anything, we dance in the expenditure of entropy.

aren't people (and all living organisms) the quintessential example of a local decrease in entropy that results in greater overall entropy?

Yes, but that's not what the person you replied to was saying. They were pointing out that there can be a global, absolute decrease in entropy, it's just statistically very unlikely.

The second law of thermodynamics is as fundamental as the uncertainty principle: The former is a result from markov chains and information theory, the latter is a result from fourier analysis of conjugate variables.

What would you consider a "fundamental physical law"?

Any law that's not based on a stochastic process, since stochastic processes can be gamed with a little bit of cleverness. If you don't believe me, just look at HFT.

You may want to read 'Thermal Physics' by Kittel and Kromer. It is the BIBLE of statistical mechanics and should straighten you out on why a statistical law is just as good as all others. And yes, it goes into great detail about when and how the statistical laws 'emerge', so to speak. Physicists have been 'worried' about what you mention in you comment for about a century and the science is very mature these days. Used copies start at ~$25.

https://www.amazon.com/Thermal-Physics-2nd-Charles-Kittel/dp...

What is HFT?

High-Frequency Trading, I'd think.

You chose a really, really bad comparison there.

Nature is under no obligation to comply with mathematical analysis.

Especially as the subject is quantum physics and thermodynamics, this is a very interesting viewpoint. Though I agree with you in principle, many centuries of inquiry about our universe by some of the most intelligent humans to date seems to say that mathematical formulations and ideas are the best (if not only, as with QM) way to think about our universe. No other way has really been as incredibly useful in terms of prediction as a mathematical one. I think you do have a point, but I will continue to take the so-called Copenhagen Interpretation of QM: Shut Up and Calculate.

Would you mind sharing a link to a proof of the second law using Markov chains and information theory? I'd like to learn more about this approach.

I'm surprised that Markov chains would be involved when the laws of physics are deterministic.

The Poincare recurrence theorem has always suggested to me that the second law is not as fundamental as other laws. For a finite system with finite phase space, the state of a system will traverse closed loops, repeating forever with no steady increase or decrease in entropy. (Edit: to be clear, I'm not claiming that what I just described is the Poincare recurrence theorem or that it applies to our universe. But it is worth considering systems where the second law doesn't apply and trying to figure out how and if they differ critically from reality.)

https://en.wikipedia.org/wiki/Poincar%C3%A9_recurrence_theor...

Not that my background is worth anything, but just so you know where I'm coming from, I have a PhD in physics, spent years thinking about the entropy of computation, and wrote parts of the Wikipedia entry on Maxwell's demon. I think much of the disagreement over entropy and the second law comes from how we frame the problem.

I am on mobile now, and can't provide a simple link, but it is given in Cover&Thomas "Elements of Information Theory", in the episode that discusses entropy of markov processes. I can find pages in google books but they won't zoom big enough to read...

IIRC, the proof requires the markov chain be irreducible, and extends to the general case by summing over the irreducible parts; and that entropy will stay the same or increase while converging to the stationary distribution over states.

(Although it has now been 20 years since I dealt with these things so I might be misremembering. Time to retread Cover&Thomas I guess...)

Cover & Thomas, 2nd Edition, Jul 2006, pg 81, section 4.4 - entropy rate of markov processes, I did not remember all the conditions needed for this to hold, please read if you are interesting.

[0] staff.ustc.edu.cn/~cgong821 /Wiley.Interscience.Elements.of.Information.Theory.Jul.2006.eBook-DDU.pdf seems to have a copy indexed by Google. I suspect it is not legitimate

Leff & Rex (both editions) is fun reading.

I believe a distinction could be made around the second law of thermodynamics being emergent rather than, well, fundamental.

Put another way: the second law of thermodynamics is not required to fully describe a universe that acts like ours appears to. It will "take care of itself" based on more fundamental descriptions of matter and its interactions.

I'm also comfortable with what appears to be fundamental today no longer being fundamental tomorrow.

This list of what makes a physical law is a pretty good start: https://en.m.wikipedia.org/wiki/Physical_law#Description

> The second law of thermodynamics is as fundamental as the uncertainty principle: The former is a result from markov chains and information theory, the latter is a result from fourier analysis of conjugate variables.

I don't think enumerating the pedigree of a law gives it the seal of approval of validity. Newtonian physics also has pretty solid pedigree, and yet breaks down in relativistic conditions.

Please note I do not disagree with your conclusion, only with the way you present it.

A. One that doesn't derive from any other laws.

B. Was around at the beginning or prior to the big bang, as opposed to being the result of conditions early on.

C. What other laws, forces, constants, etc derive from.

The uncertainty principle is an absolute and inviolable consequence of pure mathematics; the second law of thermodynamics just makes predictions that are "very very overwhelmingly likely". The probability of the application of the uncertainty principle producing an incorrect prediction (internal to the theory) is zero, whereas the probability of the application of the second law producing an incorrect prediction (again internal to the theory) is non-zero. There is an infinity of difference between the internal "absoluteness" of these theories.

The probability of either theory producing an incorrect prediction external to the theory is much higher, and will be about the same for each theory. That is, it's more likely we're wrong about all of physics than that the second law makes a bad prediction in a bulk system.

Explanation of "internal" vs "external" probability: http://lesswrong.com/lw/3be/confidence_levels_inside_and_out...

The statement of the uncertainty principle that I am aware of is that the product of standard deviations of conjugate variables (e.g., time and frequency; position and velocity) is bounded from below. This is a statement about the sample space. Note that standard deviation is expectation (over the ensemble) of the 2nd moment.

I did not remember the exact statement when I posted earlier, but here it is: the statement of the (markov) 2nd law of thermodynamics is also a statement about the sample space: It says that in expectation (over the ensemble) the relative entropy decreases towards that of stationary distribution (which in most systems is the highest entropy distribution possible in that system, thus absolute entropy is non decreasing). That is, unless the system starts at a state with a higher entropy than that of the stationary distribution, entropy will not decrease.

Both are mathematical statements, consequences of pure mathematics, neither of which gives a prediction - they both give ensemble averages, and both are both internally perfectly correct and consistent. See Cover & Thomas, 2nd Edition, Jul 2006, pg 81, section 4.4 - entropy rate of markov processes. Unless, of course, you are referring to a different version of the uncertainty principle which I am not familiar with.

No, the uncertainty principle is a statement about the behavior of non-commuting operators in a Hilbert space. It is not a probabilistic statement. It doesn't even have anything to do with probability until you apply it to a probabilistic interpretation of quantum mechanics, where vectors in the Hilbert space have something to do with probability. The understanding you are referring to is more or less correct, but is a specific application to certain interpretations of quantum mechanics. It's also useful to think about it this way experimentally (in terms of "uncertainty"), which is why most people learn it this way in early physics classes and where the name comes form.

Perhaps I have made the same mistake as you, and was thinking about the practical but non-generalized way the second law of thermodynamics is usually taught, which includes concepts like "the system will be in this state". The only part of your statement that is still probabilistic is "entropy will not decrease". That's not really true; it probably won't decrease.

>neither of which gives a prediction

"entropy will not decrease" is a prediction. It is possible (albeit overwhelmingly unlikely over large time scales) that this prediction is sometimes false.

> The only part of your statement that is still probabilistic is "entropy will not decrease"

The markov/IT version of the 2nd law is a statement about macrostate entropy (taking the entire microstate ensemble for each macrostate), and in that sense it is not probabilistic. See the Cover&Thomas reference I gave earlier for the exact definition. It is indeed different than how it is usually taught in physics, in which the microstates are differentiated.

I guess we both need to be more careful about mathematical definitions in the future ...

After reading other posters, I believe the difference is similar to how terms "axiom" and "theory" function in mathematics.

This argument that the Second Law is a simple logical consequence of basic probability always seems glib to me. Why would probability demand entropy increase as one extrapolates forward in time, but probability not similarly demand entropy increase as one extrapolates backwards in time? In other words, what of Loschmidt's paradox (https://en.wikipedia.org/wiki/Loschmidt%27s_paradox)?

There fact that is not a consequence of statistics is that we observe the universe to not be in the state of maximal entropy. That is sufficient to explain asymmetry of time, I think. The mystery is -- why wasn't the universe in a state of maximal entropy at time 0?

Elon Musk hadn't started his simulation yet /s

The anthropic principle requires any self-reflective universe to not be in a state of maximum entropy. If our universe where in a state of maximum entropy, there would be no one around to ask.

Universe started from low-entropy state. From the big bang, there is nowhere go but downhill.

In a microscopic level the physics can be time-symmetric (it seems not to be the case, there are T-symmetry violations), but macroscopically universe had higher entropy in the past (cosmological arrow of time)

edit: CPT-symmetry -> T-symmetry .

Which experiment saw CPT violation? CPT is a straightforward consequence of QFT with minimal assumptions.

Sorry, I meant T-symmetry violation, corrected.

Kaon decay and B mesons decay break CP-symmetry and a CP-symmetry violation is equivalent T symmetry violation (CPT symmetry is preserved only if CP-violation is paired with T-symmetry violation)

That's one of the fundamental questions of physics. Hawking's Cambridge lectures audiobook explicitly cover the topic of what's up with the second law being statistical, why the arrow of time is apparently directional, and possible explanations for how you can answer those questions and explain the existence of the universe without requiring a "cause" for the big bang.

You are right in that we need to assume that the universe started in an extremely small entropy state. This is basically the past hypothesis. (Which can be justified to some extent using anthropic principle and eternal inflation, however these two ideas are very controversial among physicists.)

Look at every possible two particle collision. To simplify, consider one of the particles as the initial frame of refernce so you now just have impact angle, which is random.

Now play back time by looking at that particle's post collision path as the frame of reference and play time backwards. You will find a bias in input angles for these collisions.

This is very true, and is the case with many 'laws'. Another example is with centre of masses: Take a hollow sphere, clearly the c.o.m. should be in the very middle, yet an object placed inside has not force whatsoever on it so it isn't attracted to the c.o.m. and the idea breaks down (NB objects outside are attracted predictably).

This is because this idea is purely a tool to make calculations of large groups of particles as easy as one, but it does break down occasionally.

I don't think that's true. An object would be attracted to the inner surface of the sphere because that's where the mass actually is. The shape you're describing doesn't have a center of mass the way we traditionally think of it.

It's absolutely true, and it's known as the shell theorem. It's an old result actually.

https://en.wikipedia.org/wiki/Shell_theorem

An object inside the hollow sphere would in fact be attracted to each individual mass-ful particle on the surface of the hollow sphere. But (assuming uniform density on the sphere) the net effect is 0 (it feels no gravitational attraction whatsoever).

The best way to prove this is to compute the gravitational force between your object and any arbitrary particle on the surface, then do the integration over all the particles (across the 3 dimensions).

The best way is to use symmetry. Ask yourself, which way would the net force be directed?

I don't find this satisfactory - someone could just answer with "towards the center of the sphere", as if the center were some equilibrium.

At the center of a uniform hollow sphere the force will be precisely 0.

Not just at the center; everywhere inside it.

Of course, Gauss's law. :)

Every pyhsics problem with a uniform sphere reaches the escape velocity necessary to escape the complex calculus problem well.

Which law are you thinking of? There isn't a physical law I'm aware of that claims gravitational attraction toward a center of mass, you might be conflating several laws or thinking of an approximation rule that was never intended to cover one object inside or even near another.

Newton's law of gravitation is stated in terms of a particle to particle relationship.

Center of Mass is the balance point of an object, and objects will always rotate around their center of mass unless constrained, but this doesn't relate to gravity.

You can approximate gravity at a distance from any object by using the object's center of mass, but that approximation breaks down when you're close to it.

Doesn't evaporative cooling work against the second law somehow? There is a system with some average temperature, and spontaneously enough entropy is created to cause some of the liquid to reach boiling temperature and leave the system, which decreases the average temperature of the liquid.

"closed physical system" being the operative word.

Mass and energy is leaving, thus the average temp decreases but so does the particle count.

I am not sure but have a question, is the barrier a kind of potential implied in the system?

Or say, just the particles are the "insiders" of the "established system", while the barrier not.

Reply on news.ycombinator.com