Entropy in the Universe | Hacker News Reader

Entropy in the Universe

johncarlosbaez.wordpress.com

115 pointschmaynard55 comments

Entropy is not disorder. It is

S ~ log #{microstates}

For hn readers, microstates you can think of as a configuration of a system, so its counting number of configurations. Disorder is not well defined. Stop using the word disorder with entropy. A bunch of more complicated elements have more configurations than just pure H.

My interpretation is that "order" and "disorder" are subjective. One person's order is another's disorder. For example, my wife and I have differing tolerances of "order" as it pertains to the tidiness of a room. That may seem far separated from the topic at hand, but if a thing is able to be somewhere, then it's purely subjective as to whether it's the right place, ie. ordered. There's a time component to it as well, I might have a bunch of in-progress things laying around, but that's the correct place for those things for the current moment, for me. In parallel, however, it's most definitely the incorrect place, at all times, according to my better half.

   Potential Configurations = n
   Configurations considered "order" by me: x
   Configurations considered "order" by milady: 1
   Configurations considered "order" by the universe: 0
   Configurations considered "order" by the universe at this point in space and time: 1

That which is, is order. That which isn't, is not.

Reminds me of this scene from Hi Fidelity:

    Dick: It guess it looks as if you're reorganizing your records. What is this though? Chronological?
    Rob: No...
    Dick: Not alphabetical...
    Rob: Nope.
    Dick: What?
    Rob: Autobiographical.
    Dick: No way.

This may be correct but its useless to make any engineering sense out of it. For engineering and practical intuition, use the following definition that we used at an Ivy-league college in US:

https://en.wikipedia.org/wiki/Entropy_(energy_dispersal)

Quoting the article:

> Even though courses emphasised microstates and energy levels, most students could not get beyond simplistic notions of randomness or disorder. Many of those who learned by practising calculations did not understand well the intrinsic meanings of equations, and there was a need for qualitative explanations of thermodynamic relationships.

Trust the formula you used at an ivy league college? Come on, that's a laughable call to some unknown expert! I don't care if your ideas come from an ivy league college. I care about the ideas and their quality.

I distrust people that think because something came from a prestigious school I should trust them.

That's extreme and I am sorry you feel that way. I didn't realize that mentioning Ivy-League college would be seen as ego-inflating thing. I simply wanted to say that it was used in a college level class. I definitely agree, truth matters and not the source.

Thanks for your civilized reply to my semi-rant ;-)

I don't mind a qualitative explanation for casual discussion, but it still must be morally correct. I think "configurations" is a pretty intuitive concept.

>imagine all the ways you can stack lego together.

It's the physicist's definition, for whose purposes it is quite useful.

And yeah, the physics students at your ivy league school learned H = log|microstates| also.

Anyway, entropy as a count of microstates has to be packaged with a couple of additional ideas to give rise to the concept as used in chemistry/etc. First, that the microstates are randomly explored by the system, and second, that the number of microstates is so mind-boggling exponentially huge that the most common ones will be the only ones you ever see in practice.

Is that the same as

H'=-\sum _{i=1}^{R}p_{i}\ln p_{i}

as per https://en.wikipedia.org/wiki/Diversity_index#Shannon_index

With a log base e for nits (or natural digits) and a log base 2 for bits (or binary digits) as per _a mathematical theory of communication_ (Shannon 1948)?

Not quite. The formula you gave is the more general one, the previous one assumes that all microstates are equally likely. Given that assumption they are equivalent though.

how does one read "log #{microstates}"? Is it simply "log of the number of microstates"?

Yes.

There's a quote by von Neumann advising Shannon on what to name his measure of uncertainty:

> You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage.

Or 'the number of possible microstates (internal configurations) that amount to a given macrostate'

Since this is interesting and nobody has commented yet, let me say as a side note that I like how Mr. Baez emphasized the difference in orders of magnitudes here:

The neutrinos from the Big Bang also carry about 10^90 bits — a bit less than the photons. The gravitons carry much less, about 10^88 bits.

A difference in exponents of 2 might not look like a lot but one can almost ignore any of the earlier numbers once a larger exponent appears.

From the comments of the author:

"You can puzzle your group by asking whether entropy increases as clouds of gas and dust in the early Universe collapse and form stars and planets. If it doesn’t increase, this process violates the Second Law. So it must increase. But why is it increasing? It seems that order is increasing… so doesn’t that mean entropy is decreasing?

If someone can’t answer this puzzle correctly, they don’t understand how thermodynamics works in our Universe."

So, who knows the right answer to the puzzle: where and how the increase of the entropy happens? (I also suggest that those who really know should wait for some hours... I'd first actually like to see some wrong answers which try to be right but don't manage to be -- just to get some better picture of the readers here -- I don't know if this will work).

The question is based on a false premise. What the Second Law of Thermodynamics states is that the entropy of an isolated system is never decreasing. Is the Universe an isolated system (meaning it cannot exchange matter or energy with its environment across its boundary)? We don't know anything about the boundaries of the Universe, or if it has any at all, therefore the 2nd law doesn't apply. QED.

Based on our observations we can see a lot of galaxies as in practice isolated systems, and for each separate galaxy second law still holds.

So this attempt for explanation fails to answer the question.

No galaxies are particle isolated, as all of them emit radiation.

Entropy increases because its much more likely than for it to decrease. And the reason higher entropy states are more likely than low entropy states are that there's more of them.

Alright it's been a few hours. I'll bite. The particles the stars formed out of where in a higher energy state or something. Ok now what's the true answer.

According to Baez, all the matter fields and radiation fields were in equilibrium at the big bang, so by elimination the answer has to have something to do with gravity (which is, after all, the force driving the formation of the gas clouds, stars, etc). So here's my guess: Evenly distributed matter has gravitational potential energy. By collapsing, a cloud of gas can convert this into thermal energy, raising its entropy. This compensates for the fact that the cloud's volume has reduced, which would reduce entropy on its own.

How did I do? :)

Here's another thing: I remember learning that all the nice usable energy we get from the sun is due to the fact that the big bang produced lots of hydrogen, but fusing four hydrogens into a helium yields a bunch of energy. (And you can still get some energy out by fusing further, all the way down to iron.) So if matter and radiation were at equilibrium at the big bang like Baez says, why wasn't more helium (and heavier elements down to iron) formed, leaving nothing for the stars to burn?

I think the answer has to be that the temperature was incredibly high at the time of the big bang: At really high temperatures, it's more favourable for nucleons to be flying around separate from each other as protons. At lower temperatures, it starts being better for them to be things like helium and carbon, and at lower temperatures still, iron. So if the universe's temperature had lowered very very gradually, we would be stuck with nothing but iron at this point, leaving nothing for the stars to burn. So it seems we owe our sunlight, in some sense, to the fact that the universe cooled fairly quickly?

> At lower temperatures, it starts being better for them to be things like helium and carbon, and at lower temperatures still, iron.

It actually doesn’t work that way: we don’t expect that the whole universe will ever be all iron.

Not an expert - just guessing. If entropy is a measure of overall degrees of freedom, then even though gas collapsing into stars and planets might seem more “ordered” - the total amount of degrees of freedom is increased. Hence no laws of thermodynamics are violated. Planets and starts are more “complex” systems compared to gas - so there must be much more entropy aka degrees of freedom contained within them.

Something like this maybe?

"Ordered" actually means "has a wide variety", not "everything is the same." The self-attractive potential energy of a primordial gas cloud is highly directed (it has a center) and the range of distances from that center is great. When the cloud collapses the range collapses, there's less variation, and so entropy has increased.

Also, a gas cloud isn't a closed system. As it collapses it heats up and emits infrared photons that escape.

[edit: Actually, I think the important variation is "center" vs. "particle's position." A large sphere, with all particles at the same distance from the center, would still have a lot of potential energy, and a small sphere less so.]

I’ll venture a guess.

I suppose that if we encode the state of a cloud of gas, we need relatively fewer bits to do so than something that requires more bits like a star. A star is like white noise, to a cold cloud of gas.

At the limit, you have gas particles all with zero velocity, so you can encode the velocity of each particle with no bits (I.E. constant of zero). Compare that to a star with every particle flying around at high temperatures. You’d need a number of bits to encode the velocity (state) of each particle.

Black holes require even more bits to encode the state of massive amount of particles they contain.

Am I close?

Now that enough time passed, the link with enough technical details but written to be accessible. Take care to read it whole, as the details matter.

https://www.forbes.com/sites/startswithabang/2017/04/15/ask-...

It does not actually explain why black holes have significantly more entropy than gas they formed from. In fact, there is not much difference between the question why do stars form considering the second law vs why do black holes form.

The entropy in the various particles (photos, neutrinos, gravitons etc) the stars are emitting from being formed probably is more than the entropy lost from the gas clouds ordering themselves into stars would be my guess.

I find it interesting that he mentions gravitons as a definite piece of particle physics, on the same order as photons, etc.

I was under the impression that gravitons were simply one part of one possible conception of quantum gravity, and that we were definitely unsure about whether or not they exist.

Since this is coming from a physicist / mathematician, I immediately assumed it meant the placeholder for those — i.e. whatever conveys gravity, all of it must hold that many bits to describe the observable universe.

The same thing stuck out to me, and definitely seemed presumptuous. However, I'm not a physicist, so what do I know?

Maybe one way of looking at it is that despite the fact that gravitons are hypothetical, we can still do the math and calculate the amount of entropy or information that they would/could contain.

Gravitons are speculative in that they haven't been observed, but they're almost certainly the correct language for describing mild quantum corrections to general relativity. Simple scaling arguments show that _any_ theory of quantum gravity that reduces to general relativity at low energies passes through a regime where it can be described by gravitons. Whether gravitons are fundamental objects in quantum gravity is irrelevant for the kind of state counting Baez is talking about.

Thanks for the explanation!

That zoom-able image is amazing. Sometimes we forget how big the universe is. And that's just an image of the Milky Way.

Yea that also amazed me, aside from the science a great way to put things into perspective when you are struggling with something.

I honestly think this is what I love in sci-fi, some space opera most notably: whereas most stories are usually told in terms of space and time, sci-fi sometimes speaks in terms of scale, of orders of magnitudes — because space (I mean the cosmos, not the dimensions) is a natural playground for that.

Sometimes, when I ponder the discrepancy between quantum, or whatever's smallest, perhaps strings 'below' (in scale) or 'within' (physically), and the largest relativistic objects like "dark attractors" and meta-galactic structures (streams of 'dark matter', light nodes, etc)...

It almost seems like scale is but another kind of dimension in and of itself.

Like you've got spacetime at our human-ish scale (roughly 10 orders of magnitude smaller to larger around our size), and that spacetime behaves in a certain way, that probably Einstein describes accurately. Then you've got spacetime at smaller scales, below "quantum uncertainty" if we must place a limit, and there spacetime behaves in a much different way, the picture is very, very different.

And so there could be yet another spacetime at a higher threshold of scale, and intuition tells me it may be at the galacatic level already, because dark matter, rotation discrepancy, and those weird supermassive blackholes; the flow of galaxies, the meta-structure we see. Dark energy, if it's real, also begins at that scale (doesn't break galaxies apart, but makes holes bigger between them, the non-correlated enough sets).

Going from this idea to postulating disjointed (1+3) spacetimes forming, in effect in this example, 3x4=12 dimensions to explore mathematically, let alone physically, is a stretch which I'm certainly not willing to make (although I think that's what some string theories must reduce to, somehow, maybe reduced to a single unified dimension of time maybe?); no really this doesn't make sense in my head. It's pretty but nonsensical, like art I guess. Worth the sci-fi, not the studies.

But the feeling, the intuition really is that, and conveyed by these extreme zooming animations: a different scale means totally different phenomena, and no macro-system can be reduced to its parts in that view, nor can it be deduced from its parts (I mean, we can't even solve the 3-body problem, that's harsh on "scaling continuity" or smoothness, on the "linearity" of scales so to speak). A "basic" intuition I guess is that the 4 fundamental interactions have ranges, and kick in or out depending on a particular scale of the system.

Scales are obsessing me.

What struck me was the link to Hoag's Object which came up a couple days ago. And that even though it's a bazillion miles away there's another much more distant ring galaxy that you can make out way in the distance beyond it in the picture.

https://en.wikipedia.org/wiki/Hoag%27s_Object

The total entropy of all the stars, 10^82, is 2^270

I love this, because the useful 256 bit string (2^256) is only 43 characters long in base64, and to get to 2^270, it is only 45 base64 characters.

I was just playing with this on Thursday, January 9th 2020 because I wanted to predict how many bits I needed to address every atom in the observable universe, which also just so happens to be 10^82. I use this example to depict the power of cryptography.

I thought you 0 kelvin was the lowest possible value on the kelvin scale. How come this post mention negative kelvin values? Is it not temperature?

There are no negative kelvin values in the article. 10^-30 is a positive number.

If you're referring to "there will be a cosmological event horizon surrounding each observer, which will radiate Hawking radiation at a temperature of roughly 10-30 kelvin" this is a very tiny fraction: https://www.wolframalpha.com/input/?i=10%5E-30

I was referring to that, my mistake, and thank you very much!

I don't see negative Kelvin in the article. I do see Kelvins with a negative exponent, which is just a positive value that asymptotically approaches zero.

My mistake, I thought that notation implied negative numbers. Thank you.

I was saying that the temperature of the Universe in the far future will be about 0.000000000000000000000000000001 kelvin, if our current theories are correct. This is 10^(-30) kelvin.

Interestingly, there actually is such a concept as negative temperature. It applies to systems that have a maximum allowable energy as well as a minimum one. Such systems must be isolated from the outside world, and set up carefully, but it is possible to actually get negative temperatures. https://en.wikipedia.org/wiki/Negative_temperature

I remember that from a class on lasers I took when I was an undergrad. All the physicists thought it was really cool and significant and argued about who was the first to see it coming. The engineers were totally unfazed. Whatever it takes to get the project working. And I was a would-be mathematician, so I asked whether temperature could be imaginary.

Did you get a no or something more interesting?

(I am asking for elaboration on what physicists and engineers have to say about imaginary temperatures if they didn't flatly dismiss the concept. I recognize it may read as snarky, but I really was just aiming for clever.)

No problem.

Basically the physicists thought I was winding them up and the engineers, notably including the prof, thought my priorities were all screwed up. I really did want to know the answer, but in retrospect they were both a little bit right.

The article got me thinking about the universe's mass distribution.

Is the number of galaxies at any edge of our observable universe similar to the number of galaxies in our observable universe?

tl;dr does the (entire) universe have a uniform distribution of mass?

Now, I'm aware that the universe has always been expanding from every point in all directions since the singularity before the big bang. This means that effectively 1.) every point in the universe can make the claim of being "at the center" 2.) none of them can 3.) the question is meaningless

Which of those answers you find satisfying is more of a philosophical question which I'm not concerned with for this question.

My question is this - if we were to somehow reach the edge of our universe instantaneously, simply to observe what things looked like from that point of view, would we find that there's roughly the same distribution of stars/galaxies/matter in all directions?

I'm aware that questions with the premise that FTL/instantaneous travel is possible tend to be "unknowable" to a degree.

Perhaps another way of wording it would be - do we expect the amount of matter to be of a uniform distribution in every spot in the universe? Is it impossible to know? Or can we use models of the big bang theory to predict that the distribution is roughly equal everywhere?

Some philosophical implications that are curious but are probably scientifically meaningless:

If there's reason to believe that there is a uniform distribution of mass in all observable universe, and the universe is infinite, does that imply there are an infinite number of galaxies, all of which are inaccessible if they're outside our observable universe?

If so, is out ever appropriate to consider these infinite observable universes as essentially a multi-verse, containing many combinations of observable universes, one of which is statistically likely to be similar to our own observable universe?

As always with questions regarding the size of the universe, it's center, etc. I suspect the answer will be "it's a meaningless question because we can never know", but I'm hoping that research into the big bang itself could provide some evidence.

Please let me know if I haven't used proper terms or am making a critical mistake in my current understanding of astronomy.

The way I understand it is that there is not really an edge. Spacetime itself is expanding, the best analogy I've heard is consider a spherical balloon that is being inflated. We exist as two dimensional beings on that balloon. No matter where you put two dots on that balloon, they will move further apart.

You could also draw a circle on the surface of that balloon which represents the furthest points that we can reach if we traveled at the speed of light. Nothing outside that circle would be observable to us, so it could be argued that it doesn't exist.

The image is awesome!

Reply on news.ycombinator.com