A 50-year quest: My personal journey with the second law of thermodynamics
writings.stephenwolfram.com
writings.stephenwolfram.com
The book "Algorithmic Randomness and Complexity" by Downey and Hirschfeldt goes into all of this. It is way beyond this rule 30 stuff.
while (true) {
if (random() % (random() % 19)) {
printf ("me ");
} else {
printf ("i ");
}
}Am I misunderstanding something?
I get the feeling that the criticism of his work is because he hasn’t found anything new yet, but that doesn’t mean his computational paradigm isn’t useful and/or simply interesting?
One significant aspect of his program is also to restate all of the laws of physics in terms of computational transformations, instead of differential equations.
Also, I believe he is looking for some base rules that are different from those of QM, thus leading to a theory that is not consistent with QM, but makes similar experimental predictions.
Yet here’s an entire book…
Wolfram's ability to create interesting visualizations is fantastic. But I seem to feel underwhelmed at the significance of his discovery.
In short, he says that the Second Law of Thermodynamics is a consequence of the fact that we as observers are computationally bounded and cannot perceive all the details in the the lower-level, computationally irreducible physical systems around us.
Yet this does not seem enormously different, or perhaps not different at all, than the standard account in statistical mechanics in which the observer is considered to be coarse-graining over the fine details of the system which are too complex to track.
What is new in Wolfram's approach other than using computation-related labels to describe the situation? For example, his 'computationally bounded' observer is just the standard 'coarse-graining', right?
I think part 2 is much more interesting. If anyone starts with part 1 and decides to give up on it I would recommend that they give part 2 a chance. The subject and the style are quite different.
I can remember being bored out of my mind and so practicing Bach on guitar for hours in 1990. There is just no way I would have done that in 2015. That feeling was gone forever the first time my modem connected to the internet.
It wouldn't be shocking that the intellectual giants of old became so because they had nothing much else to do but read and think.
I do put Wolfram in the same league as Von Nuemann in terms of not being ashamed to admit they are beyond me. I can distinctly remember getting New Kind of Science from the library, getting home and then quickly realizing there is no chance in hell I can read a 1000 pages of this.
We think von Neumann was smart not because von Neumann himself said so.
Read it the following order:
1. https://writings.stephenwolfram.com/2023/01/how-did-we-get-h...
2. https://writings.stephenwolfram.com/2023/02/computational-fo...
3. Skip https://writings.stephenwolfram.com/2023/02/a-50-year-quest-... if you are tight on time. It helps tying things between 1 & 2.
Given the context, the thesis gives sound foundations to the second law in terms of computation. It ties in remarkably well with Cellular Automata, and feels like "how did I not see this?".
Here is my first attempt at summarising the whole thing (badly): The Second Law holds when systems are computationally irreducible for computationally bounded observers. The applicability changes based on the observers' computational capacities.
Though it might seem like "it is obvious" to many, the restatement and interpretation of the second law is quite novel. The claim is that the second law is in fact a property of computational universe.
Myself not being a physicist, I don't know how much and how well it ties with Physics Project. From a shallow perspective I am thinking "IF the universe is computational...". I have talked with some people who told me that without having established Physics Project as foundation, some interpretations of this thesis might be a bit stretched. But that is to be expected since the Physics Project is "his bet".
In any case, it seems to me that the thesis presented in https://writings.stephenwolfram.com/2023/02/computational-fo... stands on its own for all of its core ideas.
But also important, the program did not produce mental tools (e.g., mathematical structures or other concepts) that maybe somebody else might adopt and try a new, more productive, approach.
This doesnt mean the original idea will forever fail to be relevant. (Though other "deep insights" of Wheeler, like geometrodynamics, also did not pan out.)
The life of ideas evolving in our collective mental spaces is poorly correlated with the biological lifespans of the 'carriers' and even more so with their human needs and feelings.
500x? Shouldn’t the speedup over 40 years be something between 10,000x (increase in single-threaded performance) and 1,000,000x (increase in number of transistors).
Julian Barbour's solution to this puzzle is to point out that the second law only applies to a system 'in a box'. The universe is not 'in a box' and so the second law doesn't apply to the universe as a whole.
Also, the 2nd law of thermodynamics is only about thermodynamics. Other forces, like gravity squishing things or strong force making fusion can provide order and extra energy. But it is temporary, there will eventually be no more gas to collect and fuse. Then all there is cooling off into the void.
> At the time, everyone involved denied having been responsible for the appearance of the article. But years later it emerged that the source was a certain John Gage, former political operative and longtime marketing operative at Sun Microsystems, who I’d known since 1982, and had at some point introduced to Brosl Hasslacher. Apparently he’d called around various government contacts to help encourage open (international) sharing of scientific code, quoting this as a test case.
> But as it was, the article had pretty much exactly the opposite effect, with everyone now out for themselves. In Princeton, I’d interacted with Steve Orszag, whose funding for his new (traditional) computational fluid dynamics company, Nektonics, now seemed at risk, and who pulled me into an emergency effort to prove that cellular automaton fluid dynamics couldn’t be competitive. (The paper he wrote about this seemed interesting, but I demurred on being a coauthor.) Meanwhile, Thinking Machines wanted to file a patent as quickly as possible. Any possibility of the French government getting a Connection Machine evaporated and soon Brosl Hasslacher was claiming that “the French are faking their data”.
Yet again, one mustn't go against the "american exceptionalism"[1] dogma, that's sad to hear The result of this ego-centric americanism; Russia able to develop their supersonic weapons before anyone in the west, thanks to the work of the silenced non-US scientists [2]
(if you want a connection between mathematical physics and informatics, I'd suggest looking at quantales before cellular automata)
Lagniappe: https://www.youtube.com/watch?v=i6rVHr6OwjI
It helps me to keep in mind his own admission of being quite egocentric (something like: "After all I am part of the club of those that named their company after themselves").
> part 2 in a 3-part series about the Second Law:
> 1. Computational Foundations for the Second Law of Thermodynamics
> 2. A 50-Year Quest: My Personal Journey with the Second Law of Thermodynamics
> 3. How Did We Get Here? The Tangled History of the Second Law of Thermodynamics
conundrum:
1. can i win by starting with part 2?
2. will I have a better chance of breaking even if I start with part 1?
https://writings.stephenwolfram.com/2023/02/computational-fo...
3. do I have to read part 3?
time to find out...
You can't win. You can't break even. And you can't get out of the game.
[maybe think of this like building a perpetual motion machine]
(1) you can't win [if you build a perpetual motion machine, don't expect it to generate extra power to run other things]
(2) you can't break even [you can't even build a perpetual motion machine cuz friction etc.]
(3) you have to play the game [your entire life is like a failed perpetual motion machine, you only survive by killing other things and even so you're going to run down and die, along with the rest of the universe]
but it suffuses everything, more than just perpetual motion machines, like if your coffee water has sugar dissolved in it, don't expect it to climb out and reform crystals, and if you have sugar crystals, it's just going to dissolve itself in humidity in the air, and you can neither stop nor make these things happen. If you think you can, if you figure out a way to get the chaotic dissolved sugar back out of the water as organized crystals (rock candy!), that only happens if you have created even more chaos (evaporating the liquid water, burning carbon to heat the water, etc.) along the way.
This is a dangerous pattern of thinking that led to scams like FTX or Bernie Madoff.
He failed to recognize that he was primarily exceptional in his preparation for, and early participation in, a brilliant collective of individuals. When he left the collective, his limited individual capacity left him chasing instead of leading.
Guy is trapped inside his own mind and can't get out. Even independent researchers need scientific community to spar with. If you can't explain your thought simply to your peers (followers don't count) maybe you are lost.
Same thing with Erik Weinstein. Left academia early to do something else, became rich. When he finally published the outline of the great thing he had been working for decades, someone easily spotted the errors and that was it.
Einstein's best friend Marcel Grossmann was a mathematician specializing in differential geometry and tensor calculus. Those were exactly the tools he needed for his breakthrough. Grossmann was important facilitator for Einstein. There were others. "Olympia Academy" and his wife for example.
Oliver Sacks experienced Indigo on ahem some substances.
What is left of him as a scientist is just clever crank and eternal wannabe. All his books and projects in the last 20 years, from "A new kind of science" onward are just obfuscating, hand-waving and playing with "digital mud" getting nowhere.
Excellent visualizations of overly complex things hide the fact that he finds no deep new science or explanations.
Yeah I am as unimpressed by his automata theory now as I was 20 years ago.
It’s like when avionics use independent systems to make decisions. If all the systems agree, they’re probably right. The value is in independent parallel construction of the same result.
As someone who doesn’t know enough to really know if he’s right or wrong, it’s hard to fully believe the HN view that he’s a transparent Charleston (and easy to spot at that) considering his history.
It’s helpful to see a “positive” read of him and gives me more to think about so thank you!
Can you link/mention the podcast?
This sounds like the scepticism here is unfair.
You need to consider that if somebody establishes a reputation for using their reasonably well-known (and justifiably respected) position as an academic software developer to publicise dubious and grandiose scientific theories to gullible techies and pop-sci readers, rather than submitting them to be evaluated by experts through the normal scientific process, it seems pretty miniscule beer in comparison to have some pushback in threads like this.
Who else has ever achieved that level of success on the back of a programming language, let alone a Lisp? Our host here is the only one in the same ballpark and his success was more on the application side. Arc is no Wolfram language.
Think of this as a stretch exercise for HN.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
(We detached this subthread from https://news.ycombinator.com/item?id=34658960.)
When Wolfram writes a long screed about his own genius as an introduction to his work that's a claim to intellectual authority. Scientific work isn't supposed to be evaluated based on the authority of its author, because that's not a good guide to whether it is valid. As I understand it (I haven't personally checked) Wolfram doesn't submit any of his stuff to peer review either.
Your post has me thinking about 'independent scientists' more generally.
Some defy characterisation as their work is just outside normal categories. An example could be George Spencer-Brown with his Laws of Form, a book that appeared as a complete and dense argument. Spencer-Brown was a colourful character to say the least.
Others are sort of 'semi-detached' and do publish papers in mainstream academic journals and do collaborate. Julian Barbour would be my example here.
I would imagine that a fair amount of self-confidence is needed (along with resources) to work in this way.
There's a great deal of similar practice in other places too but the inverted cult of personality in Wolfram threads is sui generis, I think. I tried once to make the case that it's a mirror image of the the thing it's criticizing, but IIRC that didn't go over too well.
I suppose the cynically exaggerated version of this would be to say bad science is fine as long as it doesn't lead to crap threads. Note that I didn't say that.
However, it could be that common opinions about authors are not tropes but warranted. In the first decade of my exposure to Stephen Wolfram, I was in awe of his intelligence. Mathematica is a wonderful tool and its step-by-step explanations of proofs are fantastic. I thought his approach to computational systems was mind-blowing.
Then I leaned more. I saw that much of his work was built on the backs of giants. His cellular automota are simplified versions of what Conroy worked on. Don't get me wrong, Wolfram formalized 1D automata in his own works, but it wasn't revolutionary. He didn't "discover" these features; he built upon the works of others.
My biggest critique of Wolfram is that he will come up with SOME new interpretation of a physical or computation phenomenon and claim that it is THE ONE TRUE INTEPERATION of how the world works. He feels like hubris personified.
But to simplify it, I think he is a smart person who often posts self-congratulatory things and I don't thing it's wrong to call him out when his writings are full of fluff.
I'll tell you one thing though: anytime someone uses the phrase "call him out", that's a sign the needle is in the red, or at least twitching to get there.
The thing to realize is that even if you're 100% right, it doesn't make repetitive threads any less tedious. Avoiding tedium is more important than being right—at least on HN. No doubt other websites, with other mandates, have other priorities.
The second law is very easy to understand when presented as Boltzmann formulated it, and Shannon later generalized it: the entropy of a system is the (log of the) number of states that system could possibly be in given a set of values we can actually measure. In a gas, for example, we can measure its volume, temperature, and pressure, but that still leaves a lot of possible actual underlying configurations of the gas particles. The more possible configurations there are, the greater the entropy. And this explains the second law: given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.
That's really all there is to it on a conceptual level.
Why?
Edit to add: In statistical physics you rely on the idea of equivalent microstates. In thermodynamics you do not introduce anything about the microscopic structure of the system under study (an extremely valuable way to generalize your results to cases not covered by typical stat physics assumptions), rather you introduce the zeroth law of thermodynamics as an axiom.
That's true, but the historical development doesn't always make for the best pedagogy. Quantum mechanics, to take my favorite example, is vastly simpler than it is commonly made out to be [1]. It just took about 80 years for this to be understood.
The development of thermodynamics followed a similar path because thermodynamics predated the widespread acceptance of the atomic theory. Indeed, statistical mechanics played a major role in getting the atomic theory accepted. But it does not follow that the thermodynamic treatment is superior in any way. I challenge you to explain to me what entropy actually is in fewer words than I used without reference to atoms or microstates. I'll bet you can't even explain what "temperature" is under those constraints, and that is much simpler.
---
[1] See https://en.wikipedia.org/wiki/Quantum_Computing_Since_Democr... chapter 9
Your (statistical) definition of entropy is perfectly reasonable and fairly intuitive and it should be taught first (as you alluded twice and as it is relatively common these days). But "understanding" entropy involves understanding multiple facets of it and why it is useful, not just knowing the easier definitions. Hence, it seem unreasonable to claim "understanding" entropy without having familiarity with the independent thermodynamic definition of entropy and *why are they equivalent*. That would be like saying one understands programming language theory while being familiar only with object oriented programming.
Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it *more* complex because it now depends on the definition of *atoms and microstates*. The thermodynamics explanation of entropy does not actually require more words, but crucially, it requires fewer concepts (heat, useful work, and heat transfer).
Concerning the development of thermodynamics: defining entropy *without* statistical physics requires *fewer* assumptions about the physical system. Statistical physics did not make thermodynamics simpler, nor is it necessary to explain thermodynamics. To your example about quantum mechanics: this is different because statistical physics is not some better more-fundamental explanation of thermodynamics. The two are separate sets of axioms with equivalent explanatory power.
Concerning the definition of temperature: are you aware that there is an implicit assumption about temperature scales when defined in the statistical physics way? That is much more explicit in the thermodynamic treatment. And it is why I brought up the zeroth law of thermodynamics: it is where temperature is defined. If you think temperature is easier to define in the statistical physics setting, you are simply hiding a ton of assumptions under the rug, which is not a very good way to build a mathematical theory.
I am curious what do you claim is a simple statistical explanation of temperature? If it is something about average energy, that is a *wrong* explanation of temperature, valid in only a few special circumstances.
As to actually giving you the thermodynamic definition of temperature: The joke is that it is whatever the thermometer says it is. The rigorous version of that joke is: According to the zeroth law of thermodynamics, if bodies A and B are in thermal equilibrium (no heat transfer) and bodies B and C are in thermal equilibrium, then A and C are also in thermal equilibrium. Use this equivalence property to tag all bodies that belong to the same equivalence class. This tag is your measure of temperature. It is an arbitrary (ordered) scale. If you want to, you can then use the second law of thermodynamics to pick a neater canonical scale.
Finally, the definition of entropy: enumerate the (macroscopic) variables describing your system and draw them as a coordinate system; compute the value of ΔQ/T as you travel through some path in that coordinate system; the second law of thermodynamics says that value is zero on a closed path, thus we have a potential function; name that potential function entropy.
Mathematically, in many aspects that is both a simpler and a more powerful definition. One could hardly claim to understand entropy if they do not understand the enormity of the previous paragraph and its relationship to the independent statistical definition.
TLDR: "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent, not about being able to recite one or the other. Especially given that statistical physics is not more fundamental than thermodynamics (unlike special relativity which is indeed more fundamental than Newtonian gravity)
What’s wrong with the usual arguments for the identification of beta as being (proportional to) the inverse of the thermodynamic temperature? You mean that it’s based on an arbitrary choice of constant to identify the entropies?
But I did discuss the consequences: my definition (well, Boltzmann and Shannon's) leads directly to the second law, and in a way that is much more general and intuitive than the thermodynamic approach. What, for example, is the entropy of a cryptographic key? On the thermodynamic approach that question doesn't even make sense to ask. It's a category error because the thermodynamic definition relies on the concept of temperature and cryptographic keys don't have temperatures.
> Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it more complex because it now depends on the definition of atoms and microstates.
Complexity is in the eye of the beholder. The classical billiard-ball model of atoms is perfectly adequate to explain the second law -- both the thermodynamic and information theoretic version -- and most people have no trouble wrapping their brains around the idea of "lots of tiny billiard balls wiggling around".
> I am curious what do you claim is a simple statistical explanation of temperature?
Higher/lower temperatures mean that the billiard balls are, on average, moving faster/slower relative to each other, all else being equal. And yes, I know that is not entirely accurate, but neither is the idea that atoms are billiard balls in the first place. This model is perfectly adequate as a first-order approximation. It allows you to describe the qualitative features of a heat engine purely in terms of the Newtonian mechanics of colliding billiard balls, which most people have a pretty good intuition for. No, it's not 100% accurate in the thermodynamic case, but the benefit is that it is obvious how to apply the concept of entropy defined in statistical terms to computation and information. I think that's a worthwhile tradeoff. Knowing how to calculate the efficiency of a heat engine is not very useful to most people in today's world. On the other hand, knowing how to estimate the entropy of a password or cryptographic key is very handy.
> The joke is that it is whatever the thermometer says it is.
Yes. But that joke conceals a deep truth: actually describing (let alone defining) thermodynamic temperature without reference to atoms, and without resorting to the thermometer joke, is very, very hard.
(It is analogous to trying to explain quantum mechanics without reference to entanglement, which is how it was done for decades. The problem with that approach is that it leaves you waving your hands about what a "measurement" is, because measurements are entanglements. Once you understand that simple fact, all of the mysteries of QM simply evaporate.)
> if bodies A and B are in thermal equilibrium (no heat transfer)
And how can I tell if A and B are in "thermal equilibrium" without resorting to the joke?
(BTW, your definition as you've given it is actually wrong. I can have two systems that are different temperatures but with no heat transfer between them simply by separating them with a perfect insulator. So now you have to incorporate that into your definition somehow, which means you have to define "insulator", and I don't see how you're going to do that without going around in circles.)
> It is an arbitrary (ordered) scale.
No, it's not. If it were arbitrary, the concept of specific heat would be meaningless.
Ultimately you need to make a connection between heat and motion. That is, after all, the thermodynamic project. You can convert heat into motion, and you can convert motion into heat, but the latter is much, much easier than the former. Why?
The statistical answer is: because heat is motion. It's a particular kind of motion. But if you deny yourself a referent to the things that are moving in the case of heat (atoms) then you deny yourself access to this simple straightforward explanation of the fundamental question of thermodynamics.
> "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent
I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems. I've never seen that done successfully.
There's a reason that the atomic theory is a thing.
My main claim is: building the notion of entropy without relying on atomic theory is crucial if you want to apply it to "interesting" things like quantum mechanics or black hole physics. Admittedly that does not contradict your earlier posts, but it does significantly contradict the philosophy of your last post.
An assortment of issues I have with the second half of what you wrote as related to our discussion of what "understanding entropy" means:
- Passwords and cryptographic keys do not have entropy. What we call password entropy is a useful heuristic, but it differs from the rigorous information theory notion of entropy (or the thermodynamic one). I claim that it is important to understand that difference as it leads to understanding the limitations of the heuristic.
- The definition I have given is not wrong, it is one of the most standard ways scientist have been defining temperature for the last century. "Perfect insulators do not exist" is an extremely important theorem that you "derive" in theoretical physics and use for a vast array of incredibly important thought experiments on which much of our understanding of nature is build. Not taking seriously the thermodynamic definition of entropy thus limits your power to explain Nature.
Edit: "perfect insulators do not exist" is not a statement about how difficult it is to solve some engineering problem. It is an absolute statement with certainty equal to that of "√2 is irrational" or "you can not solve the halting problem".
- Temperature is indeed an arbitrary scale, again something that has been established in pretty much any textbook on statistical physics or thermodynamics. At this point I really think it is worthwhile to play the authority card: you are claiming multiple things that contradict standard physics textbooks on which the last hundred years of science are based.
- The heat engine efficiency is indeed not something most people care about day to day, but most people do not care about entanglement either. Moreover, I do not care about the heat engine as a question of engineering. However, thought experiments that involve heat engines are crucially important for the derivation of many no-go theorems (including in quantum mechanics).
- This is incredibly important: heat is NOT motion. This is incredibly limiting worldview that is insufficient for anything but the simplest of toy problems. Very useful early pedagogical tool, but it has to be discarded early on if you want your student to grow.
Sure, it is reasonable to say that it is a bad pedagogy to start with the thermodynamic definition. I teach graduate classes both in physics and in quantum information science (theoretical CS) and I actually follow your way of presenting entropy in the first few lectures. But the atomic theory interpretation of entropy is actually limiting when it comes to advanced physics. Really, the only thing I disagree about with you is your insistence that one does not need the thermodynamic definition in order to "understand" entropy. Please excuse my use of the word "insistence" if that is not the case - the limitation of text-based conversation are creeping in.
> I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems.
100% yes
> I've never seen that done successfully.
Most of the graduate classes I have taken have done that relatively well, but I would agree with you that undergrad classes need more work in the US. For what is worth, the French Class Prepa curriculum on this topic is absolutely outstanding. I particularly like classes/textbooks that take the differential geometry approach to thermodynamics. As long as the textbook treats stat physics and thermodynamics separately, it is probably a textbook that does it well.
Now, it's possible that my insistence on this is a reflection of my ignorance. I'm not a physicist, I'm a computer scientist. But there are a few things that I know (or think I know) that are at odds with what you are telling me. For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. You get to pick two points on the scale, but the remaining points are then determined for you by actual physics. There is a whole field of study called calorimetry which relies on this. Your profile says you are a physicist so you must be aware of these things.
I'm not pointing this out to challenge your bona fides, only your pedagogy. You say, for example, "Heat is not motion" and that this is "incredibly important", and that "perfect insulators do not exist is a theorem", but then you don't provide any explanation or references. You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.
BTW...
> Passwords and cryptographic keys do not have entropy.
You are mistaken. Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".
(Imagine how annoying it would be if I stopped there and said nothing further.)
Entropy is only a coherent concept relative to a state of (generally incomplete) knowledge. It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas in cryptography it is determined by withholding information (i.e. keeping secrets). If I flip a coin, and I peek at it but I don't show you, that coin has zero entropy relative to my knowledge, but one bit of entropy relative to yours. The only difference between that and thermodynamics is that in the latter there are O(10^23) coins and it is impossible for anyone to peek at them.
My password has zero entropy relative to my knowledge, but (one hopes) a lot of entropy relative to an adversary's knowledge. Indeed, the amount of entropy in my password (measured in bits) is precisely equal to the base-2 logarithm of the number of possible passwords that an adversary would have to try in order to guess it.
Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable.
> For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary.
If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more.
> I'm not pointing this out to challenge your bona fides, only your pedagogy.
But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand".
> You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.
Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though:
- https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F...
- https://en.wikipedia.org/wiki/Thermodynamic_temperature
- Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V...
> Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".
No, they are different concepts, neither one is generalization of the other. That is the point I am making all along.
"Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that.
> It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas.
This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information.
Few more fun references if you are into extreme mathematical rigor:
- https://web.ist.utl.pt/berberan/data/68.pdf
- https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T...
Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship...
> Perfect insulators do not exist
Then it is not possible for any systems ever to be in thermal equilibrium before the heat death of the universe. So your definition of temperature is based on something that you yourself have said is physically impossible to achieve.
This level of hierarchical abstraction is pretty much the only tool we have though, and it seems it works better than whatever else we have tried. Axiomatic thermodynamics (the last couple of links in my previous post) does answer this particular problem though, it is just that no-one uses it.
> no-one uses it
Gee, I wonder why.
Here is how it's actually done:
It is truly silly to share a pop-sci blurb from NIST in this situation. It certainly provides a good initial intuition, but it skips all the rigor and logical coherency you pretend to care about (and which, again, is discussed in what I shared).
Well, you are the one who introduced the term "rigor" into the discussion, but you didn't define it so I had to infer what you meant, and the only clue I had to go on is that, whatever you meant, it was at odds with "practicality". So I assumed that "rigor" meant something like, "thoroughly and explicitly attending to all of the details of an argument, defining all the terms, making all the assumptions and logical reasoning steps explicit" or something like that. So sweeping some of the details of rigor under the rug in service of "practicality" was an acceptable transgression.
On the other hand, I am the one who introduced the term "logical coherence" so I'm the one who gets to say what it means: a logically coherent argument is one which is not based on any transparent contradictions or logical fallacies. On this view, rigor and logical coherence are orthogonal. You can have a rigorous argument that is not logically coherent (here is an example: https://math.hmc.edu/funfacts/one-equals-zero/), and you can have a logically coherent argument that is not rigorous. You can even have an argument that is both rigorous and logically coherent but is nonetheless wrong. But a logically incoherent argument cannot be correct. Logical coherence is necessary but not sufficient for correctness.
It doesn't really matter. What matters is that "rigor", on your view, is something that can be casually sacrificed on the alter of "practicality". Logical coherence is not. If you sacrifice logical coherence you abandon all pretense of hewing to logic and reason, and your position is no longer worthy of serious consideration or rebuttal beyond the observation that it is logically incoherent.
What makes your position logically incoherent is that you are taking a position on what it takes to truly "understand" the second law, but you are unable even to define the word "temperature". Your first attempt was literally a joke, and your second was based on a logical contradiction. And BTW, when I pointed this out, your response began with "You are kinda right :D"
No, I'm not "kinda right". I am right, full stop, no smiley. Your stated position includes both P (there are no perfect insulators) and NOT P (it is possible for two systems to be in thermal equilibrium with each other). And this is not an inference, you were absolutely adamant about both of these things:
> [P]erfect insulators do not exist" is not a statement about how difficult it is to solve some engineering problem. It is an absolute statement with certainty equal to that of "√2 is irrational" or "you can not solve the halting problem".
> According to the zeroth law of thermodynamics, if bodies A and B are in thermal equilibrium (no heat transfer) and bodies B and C are in thermal equilibrium, then A and C are also in thermal equilibrium. Use this equivalence property to tag all bodies that belong to the same equivalence class. This tag is your measure of temperature. It is an arbitrary (ordered) scale.
I don't need to know anything else to know that what you are saying is wrong, just as I don't need to know the details of a perpetual motion machine design to know that it won't work. (Though it certainly doesn't hurt to further observe that what you are saying is at odds with the SI definition of the Kelvin, to say nothing of the fact that I can accurately predict how much electricity it will take to boil a cup of water.)
BTW...
> It is truly silly to share a pop-sci blurb from NIST in this situation.
That citation was more for the benefit of lurkers than it was for you. You have clearly buried any desire to seek the truth under layers of defensiveness that are far beyond my ability to penetrate.
NIST is quite literally in the business of defining physical quantities in ways that allows them to be accurately measured, and also of communicating to the public how their work is done. If you think it is "silly" to cite one of their official publications in the context of a discussion of the pedagogy of the second law, well, all I can say is that I feel sorry for your students. I really urge you to try to develop a little more empathy for your audience, because if you don't you are going to do a tremendous amount of damage.
All of the things I claim you said are verbatim copies of things you wrote, copied and pasted directly from your comments, with one exception: I changed a lower case p to an upper case P, but even there I indicated the edit with [brackets] as is customary. No reasonable person could consider that a "willful misquote".
> check the actual rigorous logically coherent textbook references I shared
What textbook references? You linked to one paper by Pogliani and Berberan-Santos, and one mathexchange article. All of your other links were to Wikipedia.
BTW, I just browsed through the P&B-S paper, and it is also logically incoherent.
> ... his famous second axiom, that constitutes the real novelty of his work ... “In the neighborhood of any equilibrium state of a system (of any number of thermodynamic coordinates), there exists states that are inaccessible by reversible adiabatic processes”. ... It can be noticed that in the axioms and definitions of the new method there is no mention of heat, temperature or entropy whatsoever
But that is not true. The axiom is framed in terms of "adiabatic processes" and "equilibrium state", both of which depend on the concept of heat.
I'm sorry, but if this is the best you have, then the whole field of axiomatic thermodynamics is utterly bankrupt.
I've found this quote amusing and I believe it explains to some extent problems with the Second Law.
> The second law is very easy to understand when presented as Boltzmann formulated it
When I read prominent scientists I often become impressed by their creative ability to not understand "simple" ideas and to spend years in attempts to understand them. It seems for me that this ability and willingness to think is the main driver of a scientific progress.
Einstein for example rejected a very simple idea of time, and reinvented it to create the theory of relativity. How on Earth someone with a brain can not understand time? But scientists can and they find this useful. I think they have special secret training on incomprehension.
To apply your mind to a problem you need first to find a problem to attack, and this is the most tricky part of science. If you do not see problems with existing theories you are out of luck, you have nothing to attack. So your first reaction to any idea presented to you better be "I do not understand" then "it is very easy and even obvious".
But that _really_ isn't what he did! He started from the seemingly obvious notion of linear time. Added the _observation_ of the finite speed of light. And _noticed_ that the "obvious" notion of linear time leads to contradictions (basically saying "at the same time" has different meanings for different observers). And from that he _had_ to go out and construct relativity in order to reconcile these contradictions. Does it matter to anyone? In practice? Probably not. As long as you do not get close to the speed of light (either in velocity or in mass. Yes.. that statement makes sense as is) relativity really is indistinguishable from our notion of linear time. This is _necessary_ for relativity to be a useful theory.
But it is _also_ a massive shift in how we think about the world. That's the danger with "intuitive" understandings. They can only ever be based on experiences and those do not lend themselves to extrapolations. The 2nd law of thermodynamics is one of those things. It seems completely obvious if learn about it the first time. Then you go a bit deeper, start picking it a part, thinking about the second and third order consequences of it. Then you argue a bit about whether the probabilistic formulations are worth anything in the first place, then you try to reconcile it with information theory. At this point you get into the whole quantum mess and ask yourself how this "law" can hold in all these circumstance, even if it doesn't seem to have any reason to mean anything anymore.
Like what?
Why not? By whose standard is a different formulation more "fundamental"?
This is an argument for observing high entropy macrostates at any time, assuming all microstates are equally probable. But not for evolution in time that makes entropy higher in the future, or at least non-smaller in the future. You assume the closed system evolves its microstate in time "by pure chance", i.e. any microstate is equally probable as the next microstate, which ignores many constraints there are due to physics laws like conservation of phase volume or energy. This probabilistic argument could be applied to past just as well as the future. But observations show entropy of an isolated system is lower in the past than in the present. So your argument does not really derive all aspects of 2nd law.
Well, obviously the system is subject to the constraints of its dynamics. It's not just picking a new microstate at random at each instant in time. It can only evolve to states that are accessible from its current state according to its dynamics. But given that set of accessible microstates, you are more likely to get a macrostate corresponding to a larger number of microstates than a smaller number.
The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.
This is due to mixing in phase space, which is plausible for many particle systems. However, it works in both directions of the parameter t.
The only way to derive 2nd law including its time asymmetry (which is the major point of 2nd law) is to restrict considerations to those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state. This means we consider only subset of all microscopically possible processes. E.g. Jaynes selects those macroscopic processes that bring one macrostate A, via some irreversible evolution, to another macrostate B, repeatedly. Then, assuming this reliability of the resulting state, he derives non-decrease of entropy from maximum entropy principle and Hamiltonian statistical physics. If we didn't assume this restriction and allowed all mechanically possible processes, entropy could both increase and decrease.
> The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.
This often repeated idea is in vein with Clausius' ostentatious statements on energy and entropy of the Universe, which have been out of place and plagued discussions of thermodynamics with needless mystical assumptions ever since he wrote them. There is no evidence for them, as there are no reliable measurements of entropy of the Universe. There is no need for them, as there are no good reasons to believe Universe has defined entropy at all. Thermodynamics, including 2nd law, is about finite sized systems free of strong gravity effects. It does not extrapolate easily beyond compact systems like planets/stars to ever bigger systems. There is no Universe in it anywhere.
2nd law that we see in daily life or a lab is not a statement about entropy of Universe being lower in the past, nor does it hang on such an assumption. It is (in one of its variants) the statement that a finite isolated system can change its macrostate to some other final macrostate, but this final macrostate is of higher or same entropy, so in real process entropy can't decrease.
I don't think that's true, though I'm not sure I understand what you mean by "those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state". (The same as what? The same in what sense?)
The reason you can't unscramble an egg is not because there is anything fundamentally irreversible in the laws of physics, it's because the time-evolution of the scrambling egg is governed not only by the motion of the whisk but also by the random thermalized motion of the constituent particles. To unscramble the egg you would need not only to precisely reverse the motion of the whisk, but also all of the thermal velocities of the constituent particles. The latter might be possible in principle (though not in practice) in a classical world, but throw in quantum mechanics and it's not possible even in principle. So the second law happens simply because the dynamics of non-trivial systems (like eggs) are chaotic, and quantum mechanics always throws in some randomness at the lowest levels.
And yes, quantum mechanics is also time-reversible. If you want to quibble about that, read this first:
https://blog.rongarret.info/2014/10/parallel-universes-and-a...
> There is no evidence for them
If you believe that the 2nd law is universal (and I see no reason to doubt it) then the inescapable conclusion is that the further back you go, the lower the entropy of the universe was, reaching a global minimum at inception.
In the formulation of 2nd law for adiabatically isolated systems, there is the initial equilibrium macrostate A, and the different equilibrium macrostate B that the system gets into eventually after the intervention, where either some constraint is released, or some macroscopic work is done. 2nd law states that if the same intervention in A results in B reliably, then S_B >= S_A.
This statement can be derived from mechanics and maximum entropy principle only under the assumption that B is a reliable result of intervention in A (always or with close to 1 probability). This is more than mechanics or logic can provide, and it means we have a restriction on the allowed set of microscopic processes considered.
It is not possible to derive 2nd law from mechanics or logic for all microscopic processes. Some microscopic processes bring the state B to state A, and those break 2nd law.
[emphasis added]
Your use of "the" here, with the implication that there are two unique microstates A and B, is problematic. There is no such thing as "the" equilibrium microstate. A system at thermodynamic equilibrium is not static.
(In fact, I think it is fair to say that the whole point of thermodynamics is to sweep vast numbers of the physical degrees of freedom of a system under the rug so that you can treat a system at equilibrium as if it were static even though it really isn't. It is frankly astonishing that this is even possible at all without losing any fidelity in terms of measurable outcomes.)
> Some microscopic processes bring the state B to state A, and those break 2nd law.
Yes, that's true. The second law is probabilistic. It is "possible" to break the second law in the same sense that it is "possible" for a baseball to quantum-mechanically tunnel through a catcher's mitt. You can do the math to figure the odds in both cases. In fact, actually going through this process is an informative exercise. It isn't difficult.
There are some events whose probabilities are so low that the chance of them occurring anywhere in the universe before heat death is barely distinguishable from zero. The odds of actually observing such an event here on earth are vastly smaller still. Both macroscopic tunneling and 2nd law violations are events of this sort. Neither is categorically impossible, but (to put it mildly) it's pretty safe to bet against them nonetheless.
The problem with your handwavy argument about macrostates with higher multiplicity is not due to 2nd law being a probabilistic claim. The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states. To address that, another assumption is needed, such as the asymmetric relation between my macrostates A,B.
So, having now re-read what you wrote upstream more carefully, can you elaborate on what you mean by "intervention"?
> The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states.
I don't understand this. What do you mean that it "does not apply to past states"?
> another assumption is needed, such as the asymmetric relation between my macrostates A,B
Well, yeah, there is an asymmetric relationship: one precedes the other, and not the other way around. But that's not an "assumption", that is just how you defined A and B.
I really don't see the problem you are trying to describe.
> given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.
This argument is that future macrostates of an isolated system passing through non-equilibrium states (due to some intervention to the system that was in an equilibrium state before) should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.
However notice the word "future". We can formulate the same argument using "past": past macrostates of an isolated system passing through non-equilibrium states should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.
Now, why does the argument give correct answer when applied to future macrostates but not when applied to past macrostates? We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.
OK, that's what I thought.
> We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.
Ah.
No, you don't need any additional assumptions. All you need is to observe that the present constrains the past differently from the future. There are fewer pasts compatible with a given present than there are futures compatible with that same present.
You can't just say that there are many high-entropy microstates, and so you are as likely to encounter one in the past as you are in the future, because any candidate past state has to be able to evolve into the present state. And if the present state is a low-entropy state, the precursor for that is much more likely to be an even lower-entropy state. Yes, there are high-entropy microstates that will evolve into low-entropy ones, but the fraction of those states chosen from among all high-entropy microstates is indistinguishable from zero, and so the odds of a high-entropy microstate in general being the precursor of a present low-entropy state is likewise indistinguishable from zero.
By way of very stark contrast, the successor states of a low-entropy state (or any state for that matter) is much more likely to be a high entropy state than a low-entropy one for the exact same reason: the present constrains the past more rigidly than it does the future.
What you wrote after that is such an assumption. You can't derive asymmetry from nothing, logic or mechanics. The only way to have it, is to assume it in addition to those.
Not quite. There is an underlying assumption, but that's not it. I can demonstrate the asymmetry between the past and future. For example:
Suppose I flip a fair coin, and I make a video of the coin flip. Let's say the coin comes up heads. I can predict with near 100% certainty (modulo shenanigans) that if I watch the video I will see a coin being flipped and coming up heads. By way of very stark contrast, I cannot predict the outcome of actually flipping the coin again with better than 50% odds.
I claim that this asymmetry in my prophetic abilities (100% ability to predict what will be seen on the video vs 50% for re-flipping the actual coin) is a result of the asymmetry between the past and future.
Note that this is not a circular argument; this demonstration would work equally well in a universe where it is possible to unscramble an egg. All it requires is for there be some experiment whose outcome I can record but not predict, even with arbitrarily advanced technology. The existence of such an experiment is an assumption, but it is a very weak assumption, much weaker than the second law itself. Indeed, it is a necessary assumption for doing thermodynamics at all, indeed, for doing anything. If I can reliably predict the outcome of any experiment, that means I know everything that is going to happen before it happens, including my own actions.
So yes, I have to assume that I cannot be omniscient. Or maybe a better word here would be "hope" because being omniscient would render my entire existence meaningless.
No, this and other similar memory-induced asymmetries have much more prosaic reason. It is more successful to predict systems we have enough knowledge and control of (video playback) than those we don't (coin flip).
Predicting and retrodicting systems we understand and have memory of is successful, and predicting and retrodicting systems we don't understand and don't have memory of is not. This has nothing to do with universal asymmetry of past vs future.
Your memory of what has happened (e.g. some system had lower entropy in past) does not explain why your original argument does not apply now to retrodiction of past macrostates; the memory only contradicts the result of that argument.
> If I can reliably predict the outcome of any experiment, that means I know everything that is going to happen before it happens, including my own actions.
That would in no way prevent validity of 2nd law.
Nope. Consider this variation on the experiment: I flip a coin, I make a video recording, but I do not look at the coin, I only reveal it to the video camera.
Now I cannot predict the result of watching the video despite the fact that my "knowledge and control" of my video recorder is exactly the same as before.
And, BTW, my ability to make a reliable prediction in the first case is in no way contingent on my knowledge of how video recorders work, nor is it contingent on my having control of the camera. I could be completely ignorant of the inner workings of video recorders, and the camera could be completely outside of my control, and I could still make an accurate prediction, but only if I saw the coin after if was flipped.
> That would in no way prevent validity of 2nd law.
I never said otherwise, but that is neither here nor there.
Yes, knowledge of details of how the recorder works is indeed largely irrelevant, but that was obvious. The point was that being successful with predictions or retrodictions of some system is correlated with having enough information on its state and the laws it obeys. This difference of success for predicting playback vs predicting result of a future experiment sensitive to unknown conditions is due to sufficient information vs insufficient information. I don't see anything in your responses that shows some universal asymmetry between past and future that would explain why your original argument does not work for past macrostates, but does work for future ones.
No, I lack information about the state of the coin. All of my observations and knowledge of the operation of the recorder are exactly the same as before. I can still see the camera, see that it is pointing at the location of the coin (even though the coin itself is hidden from me). Nothing has changed in that regard.
> I don't see anything in your responses that shows some universal asymmetry between past and future
Maybe it would be clearer if I change the scenario slightly: instead of one camera, I'll use N>>1 cameras, all pointing at the same coin. I flip the coin in view of all the N cameras, but hidden from my view, and then I put the coin in my pocket without ever looking at it.
At this point I can make no reliable predictions. But if I wach the playback on one of the N cameras, I can then reliably predict the outcome of viewing the playback on any and all of the other cameras. But under no circumstances can I ever accurately predict the outcome of flipping the coin again.
Why? Because there is a fundamental asymmetry between the past and future. It is possible to make reliable records of the past, and if you make multiple records of the same event, all of those records will agree on what the outcome was (that's what "reliable record" means). By way of very stark contrast, it is not possible to make reliable records of the future.
And that implies you lack information on the record, which is decisive for predicting successfully its playback.
> But under no circumstances can I ever accurately predict the outcome of flipping the coin again.
Because you lack the necessary information, because you assume those experiments are that way. If you set them up so that you don't look at the coin flip result, but in the next coin flip you measure the initial conditions and the flipping process accurately enough, the conclusion would be the opposite - you would accurately predict the new coin flip, and would not be able to predict the playback.
> Why? Because there is a fundamental asymmetry between the past and future.
Not at all. The asymmetry is due to how you defined constraints on the experiments.
> It is possible to make reliable records of the past, ... it is not possible to make reliable records of the future.
Record is, by definition, a pattern that has been created in the past, and this is likely to correlate more with the past states than with the future states. This is a property of the concept, not of some universal asymmetry between past and future. We may introduce analogous concept of a future record, which will be created in the future, and which will correlate with the future states.
We can talk instead about prediction and retrodiction. And then this is more of an easy/hard thing, not possible/impossible thing, and the distribution of success on this axis depends on details. Predicting Moon's position on sky 2 years into future is easier than retrodicting it 20 000 years into past.
None of this explains why your original argument does not apply to past macrostates.
Yeah, that's gonna be a long row to hoe if I can't even get you to acknowledge that there's a difference between the past and the future independent of the second law itself.
Maybe I should just ask: do you think there is any difference between the past and future (independent of the 2nd law)? If so, what is it?
But in the meantime:
> that implies you lack information on the record
This turns on what it means to have information "about" something. All of the data I have first hand access to comes to me through my senses, mainly my eyesight. When I say a coin is heads-up it's because I can see the coin, i.e. there are photons emitted from the coin (presumably) that end up on my retina and create electrical impulses that go to my brain and leave me with the subjective sensation of seeing a heads-up coin. There are all kinds of background assumptions about how light works, how coins work, how my retina works, and how my brain works, but for the most part one chooses to sweep those under the rug and accept as a working hypothesis that the reason I have the subjective sensation of seeing a heads-up coin is that there is in point of actual fact a coin "out there" and that it is, again in point of actual fact, heads-up.
But the camera is a very different story. I get direct input from the camera in the same way that I get direct input from the coin. I can see the camera just as I can see the coin. But unlike the coin, the camera's state with respect to what it has recorded is not directly apparent to me. That state is part of the internal state of the camera. It is hidden from me. I cannot directly observe it simply by looking passively at the camera the way I can with the coin while my visual cortex of my brain does the heavy lifting. I cannot even tell that there is hidden internal state based on anything that I can see about the camera while it is recording.
Making a prediction about what will happen during "playback" (indeed, even predicting that there will be something interesting to observe at all!) requires a much more complicated model of the world and chain of inference than it took for me to conclude that the coin was heads-up.
I introduced multiple cameras so that the structure of the example eliminated all that complexity and everything was symmetric. All the cameras are the same, and my knowledge of them is all the same -- until I start looking at the playback, at which point things start to diverge. When I look at the playback on one camera, I now have direct information about the (previously hidden) state of that camera, but not of any of the others.
So here again the question is: why should (direct) information about the state of one camera tell me anything about the state of any of the other cameras? It is because of a very complex chain of inference, part of which includes the hypothesis that there are, in point of actual fact, events which have occurred in the past (and which now cannot be changed) and that there are things in the present (like cameras) whose states correspond to those past events, and for that reason correlate with each other in the present.
I've lived a long time and I've engaged in a lot of philosophical discussions, but you are the first person with whom I've ever had to go into that level of detail on such a foundational matter. Not that there's anything wrong with that, you're keeping me on my toes. But my point is just that if you're going to insist that I be excruciatingly precise about something this elementary, it's going to be a very long time before we can come back to talking about thermodynamics.
This lack of skill is the same towards the future as towards the past, at least in the context of reversible theories of motion like classical mechanics or evolution governed purely by Schroedinger's equation.
Flip a coin, do not look at it, and make a short video of the coin from above when it is at the top position. This video provides us with details of state of the coin.
Assuming a fair coin and a strong enough flip, from this state of things we can't predict the resulting face, and also can't predict the starting face on the thumb. This problem is symmetric towards the future and towards the past.
The same argument holds for knowledge of one microstate of gas at single time instant. Past and future are equally undetermined, and all probabilities are the same towards the future as towards the past.
To get 2nd law, we need to assume the asymmetry in addition, and it's best if it is explicit. E.g a process connecting two equilibrium macrostates, where the two states aren't defined in a symmetric way. For example, as Jaynes does, we define B as the reliable result of autonomous evolution after manual intervention in A. This makes A,B asymmetric, because it is not stated in the assumptions (and usually not considered possible) that also A is the reliable result of autonomous evolution after intervention in B. This asymmetry is what allows us to derive 2nd law.
See e.g. this Jaynes' paper, Section IV The 2nd Law. https://github.com/lawrennd/jaynes/blob/main/gibbs-vs-boltzm...
It is interesting that this derives asymmetric 2nd law from symmetric (reversible) underlying microscopic model. The asymmetry comes in exclusively due to the assumption of B being the reliable result of intervention in A.
We cannot have complete knowledge of the state of a gas. We cannot even have complete knowledge of the state of a single particle because the positions and velocities of classical particles are real numbers but our knowledge is constrained to be finite, so there will always be some measurement error, and so chaos theory applies. (And physical reality is actually quantum, so we have the uncertainty principle at work as well.)
> This problem is symmetric towards the future and towards the past.
Only because you've carefully crafted this example to have this property; it is not true in general. You have constrained your knowledge of the present to exclude that which is needed to reliably extrapolate into the past. But it is possible to change your scenario and collect data that allows reliable extrapolation into the past, for example, by looking at the coin before it is flipped and remembering (or writing down) what you saw. But for the future this is not possible. That is the essential asymmetry.
> See e.g. this Jaynes' paper
I have a lot of respect for Jaynes so I'll take a look but it may be a while before I have the time to give it the attention it deserves.
One thing I see right away that he gets right (well, mostly): in section 6 he observes that there is no such thing as "the entropy" of a system, which is correct. Entropy can only be measured relative to a state of incomplete knowledge. But calling it "anthropomorphic" is not right. It's not a function of human ignorance specifically, it's a function of ignorance in general.
Agreed. But it is true for your original argument applied to a non-equilibrium state at any single time instant, while not assuming any more on the past or the future states.
If we adopt further assumptions about the past states, but not about the future states, then we agree this problem of prediction/retrodiction is asymmetric between past and future and this may be the reason why the multiplicity argument does not have to result in past entropy being higher. In particular, there may be other facts on record about the past states and that would override/modify the multiplicity argument for the past.
> it is possible to change your scenario and collect data that allows reliable extrapolation into the past, for example, by looking at the coin before it is flipped and remembering (or writing down) what you saw. But for the future this is not possible. That is the essential asymmetry.
Yes, in any realistic scenario involving prediction and retrodiction at any time, we have some data on the past states (in some lucky cases, enough data for having certainty), but never data on the future states (we may have predictions, but those are not of the same quality as data). I agree.
But in addition, you seem to argue that this asymmetry of availability of data is an essential fact to derivations of the 2nd law.
Consider a scenario where we have an isolated system that is a mixture of miscible liquids that exists for ages but we have data on its state only for the last 2 years, and that data says its entropy was steadily rising the whole time.
Now, given this data, was entropy increasing before the collection of data started, or was it decreasing? For short enough times around the edges, we can extrapolate the entropy curve based on collected data, but for long enough times, we can't. Using the multiplicity argument, including the data, we get that entropy of very distant past states was much larger than entropy of the first recorded state. Which contradicts 2nd law.
Effect of that "essential asymmetry" has a time span that is not universal, but limited to times where collection of data was happening. Outside it vanishes, and using only that data and the multiplicity argument, the distant past becomes as cloudy(highly entropic) as the distant future is.
No, it isn't. But we need to take this one step at a time. (And, I should also point out, this is not my argument. It's Boltzmann's and Shannon's.)
> further assumptions about the past states
They are not assumptions, they are observations. There is actual evidence that it is possible to remember the past but not the future. (I can't believe I'm actually having to persuade you of that. This is a pretty basic fact about reality.)
> in some lucky cases, enough data for having certainty
Nope, you can never be certain. Even the best measurements have error bounds, and our brains and recording devices have finite storage capacity. Our knowledge of the past can never be perfect. But -- and this is the important part -- our knowledge of the past can be vastly better than our knowledge of the future.
> you seem to argue that this asymmetry of availability of data is an essential fact to derivations of the 2nd law
I don't know if it's essential. I'm just saying that 1) it is a fact supported by evidence, not an assumption and 2) the second law follows from it (plus a few other things). It might be possible to derive the second law without it, I don't know, and I don't care because the actual fact of the matter is that this asymmetry between past and future does exist in point of actual fact. I don't care about hypothetical universes that behave differently from the one I inhabit.
> Using the multiplicity argument
Sorry, what is "the multiplicity argument"?
> Effect of that "essential asymmetry" has a time span that is not universal, but limited to times where collection of data was happening.
No, it doesn't matter whether data is actually being collected. What matters is that it is possible to collect data (about the past) because the past actually happened. Whether or not anyone actually bothered to make a record is irrelevant. If a tree falls in the forest, it actually falls even if no one makes a video of it.
You certainly don't have to waste time convincing me of that, but it seems you should try to explain why this general asymmetry of concept pairs record-past, record-future is relevant to your original argument. We're at this for days and you did not write any coherent argument where this evidence (which I don't contest) would play obvious role.
When I talk about some statement as an assumption, this does not mean I contest its factual evidence or validity. I am interested in the structure of your argument, and all assumptions need to be illuminated, even if they are based in factual evidence.
> what is "the multiplicity argument"?
It's your original argument: given the system is evolving without external influences, and we know its macrostate and its entropy at some time, then most likely the future macrostates will be such that they do not have lower entropy, but higher or same, and this is so because there is immensely more microstates corresponding to macrostates of higher entropy (macrostates of high multiplicity) than there is microstates corresponding to macrostates with lower entropy (macrostates of low multiplicity).
I don't see how asymmetry of memories or records made plays role in this argument. I can see how they play role in my example with collecting data for 2years. But then my argument stands: the data becomes more irrelevant the farther into the past we retrodict, and for very distant past it can be discarded, and the multiplicity argument wins, and fails to be consistent with 2nd law.
To me, this is due to fact one cannot derive 2nd law in general, as probable behaviour for all processes. There is as many phase space trajectories increasing entropy as there is decreasing it. One has to select those macroprocesses that we consistently observe to have the same result (Jaynes ' states A -> B) and then one can derived 2nd law for those processes, only.
> Whether or not anyone actually bothered to make a record is irrelevant.
It is relevant in that it changes data based on which predictions (or retrodictions) are made. I'm sure in science, different data means possibly different predictions.
Try to formulate your whole argument (which I understand as being based on the multiplicity argument), including how "it is possible to remember the past" makes its application to distant past which nobody remembers invalid, while keeping its application to distant future valid.
No. The universe has kept a record of the very distant past. We can directly observe the past all the way back to the cosmic microwave background, which gets us >99% of the way to the big bang.
So yes, I agree with you that it is possible to have a universe where the second law does not apply. It just happens that the universe we live in is not such a universe, not because of any assumptions, but because of what we can directly observe today, and how we can explain those observations in terms of a past that actually existed.
"The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery."
And my answer to this is the same. Unsubstantiated assumption that universe started out, and that its entropy at this start was low. In addition, it is not at all clear how this assumption helps us to explain that an isolated system could not have higher entropy in the past long before we observed it.
You have elided something very important here. I did not say that the universe started out, full stop. I said it started out in a low-entropy state. It is the "in a low entropy state" part that matters, not the "started out" part. If you don't like the idea that the universe "started out" then you can replace this with "the universe was, at some point in the past (specifically, about 13 billion years ago) in a low-entropy state. We don't know how it got there, or what, if anything, happened before that."
And this is not an "unsubstantiated assumption." There is evidence that the universe was in a low-entropy state in the past. In fact, the evidence is overwhelming. Disputing it is akin to disputing that the earth is round.
> it is not at all clear how this assumption helps us to explain that an isolated system could not have higher entropy in the past long before we observed it.
That's because it isn't true. An isolated system can have a high-entropy past, e.g. https://en.wikipedia.org/wiki/Spin_echo
But any system that we actually observe is not isolated, and that is part of the reason why we never observe decreasing entropy despite the fact that decreasing entropy is not only possible, it actually does occur under the right circumstances.
In fact, we can invoke the anthropic principle to explain why we do not observe decreasing entropy: creating a record of the past is not possible in a universe where entropy decreases. There are, by definition, fewer low-entropy states than high-entropy states, and so transitioning from a high entropy state to a low entropy state requires the destruction of information by the pigeonhole principle. And so the reason such universes are not observed is that such universes cannot contain records of the past, and so they cannot contain observers.
“It cannot be emphasized strongly enough that W is not the measure of the set C of all states […] compatible with the external macroscopic constraints; rather, it is the dimension of the subset of those states that can be realized in the greatest number of ways.” [Entropy and the Time Evolution of Macroscopic Systems, Walter T. Grandy Jr.]
I wonder, though, does this actually matter to anyone but a physicist? Consider the common just-so story about electromagnetic waves: a changing electric field produces a magnetic field, and a changing magnetic field produces an electric field. Therefore, once you start to wiggle an electron around, the changing field become self-sustaining and you end up with a wave.
This story is wrong because it would lead you to conclude that the E and B fields are 90 degrees out of phase when in fact they are in phase, and understanding why that is requires going into quite a bit more detail. But does that actually matter to anyone other than a physicist or an antenna engineer?
That formula - Boltzmann's entropy - is only applicable when all the microstates are equiprobable. The question of what does it mean in general is subtle.
If we have a gas in a container in thermal equilibrium with a heatbath, for example, the energy of the microstates compatible with those constraints is not fixed and their probability depends on the energy. The energy and the pressure fluctuate.
> I wonder, though, does this actually matter to anyone but a physicist?
The people most concerned with the question of entropy on a conceptual level tend to be physicists - and philosophers. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/4341...
So, it makes you wonder does that make ChatGPT smart since Wolfram says he's smart, or does it make Wolfram dumb since he's just an LLM ?
The result of this ego-centric americanism; Russia able to develop their supersonic weapons before anyone in the west, thanks to the work of the silenced non-US scientists [2]