Battling Entropy | Hacker News Reader

Battling Entropy

fs.blog

64 pointsgalfarragem60 comments

This article begins with an analogy to the thermodynamic concept of entropy, and then attempts to relate it to other kinds of "entropy" by making the simplification that entropy is equivalent to disorder. That's all fine and well, but then I really thinks it goes too far: you really can't try to "reduce entropy" in your business by looking at a physical process, nor is coughing "the transfer of energy as heat". It's getting dangerously close to the kind of pseudoscientific platitudes that ended up making something like this marketing description of the Pepsi logo, which "uses" general relativity in its description: https://www.goldennumber.net/wp-content/uploads/pepsi-arnell...

    The Pepsi Orbits
    Dimensionalize 
    exponentially.

The most profound thing you’ll read today! (The last diagram is a bit disappointing, though.)

I'm not sure Peter Arnell (the guy who authored the Pepsi rebrand) was ever setting out to be profound. I think he was just a brilliant opportunist who recognized that Fortune 500 marketing managers would pay hand-over-fist for his utter bullshit.

Oh my god that pepsi document.

It's real? Was someone actually paid to produce that document?

As I scrolled downwards I just kept getting more and more convinced that it's some kind of ridiculous surrealistic joke thing.

Seconded. It must be some kind of ersatz April Fool.

(although I just know I'm going to show it to my marketing chum later and they'll go "Yep, that's the kind of thing clients want" and I will be sad.)

I know, right??

Some real gems:

pg. 21: Emotive forces shape the gestalt of the brand identity.

Actually no, the whole thing is patently absurd. Wow. What the fuck?

Apparently this presentation earned a million dollar fee:

https://www.cbsnews.com/news/pepsis-nonsensical-logo-redesig...

In fairness it is marked as WIP.. I suppose it could have been completely scrapped and redone before presenting to the client :P

Edit:

AdWeek was less convinced at the time:

https://www.adweek.com/digital/the-pepsi-logo-design-pdf-emb...

Can't find anything definitive on what this actually was, seems like a lot of "hoax or not?" type pieces with no conclusion that I can see.

>pg. 21: Emotive forces shape the gestalt of the brand identity.

Which is trivially true.

Forces of emotion (how consumers feel about the brand) is what shapes brand identity.

Gestalt here means something greater than the sum of its parts, which brand identity is -- what we perceive as a "brand" is not a single thing but tons of them (the logo, the product line, their retail shops, the ads, our past experiences with their products, the cultural significance of the brand, and so on).

The Pepsi doc is real https://www.cbsnews.com/news/pepsis-nonsensical-logo-redesig...

Interestingly, biological beings seems to defy the second law by working to reduce disorder, or surprises, in the sensory system. This is the meat of Karl Friston's Bayesian hypothesis of the brain.

> biological beings seems to defy the second law

Well, maybe they seem but they certainly don't. Biological beings are not closed systems.

The second law only applies to closed systems, which living things are not. While they may internally reduce entropy, when you consider them along with the environment they live in, they end up producing heat and other disorder that is far in excess to the small amount of order they have created.

You forgot to include the sun in your analysis.

And the rest of the sky. ;-)

The energy radiated from the earth is in a higher entropy state than the energy it absorbs. If the theory that biological beings defied the second law were true, you'd expect the opposite to be the case. Life is a kind of machine that uses useful low entropy energy to create organisms, emitting less useful high entropy energy as a byproduct. Not all that different than any other kind of energy-using machine.

> Let’s imagine that we start a company by sticking 20 people in an office with an ill-defined but ambitious goal and no further leadership. We tell them we’ll pay them as long as they’re there, working. We come back two months later to find that five of them have quit, five are sleeping with each other, and the other ten have no idea how to solve the litany of problems that have arisen. The employees are certainly not much closer to the goal laid out for them. The whole enterprise just sort of falls apart.

Is that true? I mean, it sounds intuitively appealing (especially if you fancy yourself a boss-type) but has anyone actually done this experiment?

Maybe some of those 20 people are ambitious and take it upon themselves to lead the project, maybe they all form a self-organizing collective and make sensible decisions by consensus, maybe they hold a vote to elect a de facto CEO...

There was a german entrepreneur who did something like that. He picked a bunch of young software developers and gave them an office and budget to do what they like with it. It didn't end well, they spent the budget on gaming chairs and spent most of their time playing video games

Sonnet against Entropy

  The worm drives helically through the wood
  And does not know the dust left in the bore
  Once made the table integral and good;
  And suddenly the crystal hits the floor.
  Electrons find their paths in subtle ways,
  A massless eddy in a trail of smoke;
  The names of lovers, light of other days
  Perhaps you will not miss them. That's the joke.
  The universe winds down. That's how it's made.
  But memory is everything to lose;
  Although some of the colors have to fade,
  Do not believe you'll get the chance to choose.
  Regret, by definition, comes too late;
  Say what you mean. Bear witness. Iterate.

John M Ford (originally here: http://nielsenhayden.com/electrolite/archives/003789.html)

People think that entropy increasing is basically atoms going from a state of organization to a state of random distribution.

All you need to do is look at the universe to see how off this definition is.

The universe started as a big bang: a soup of randomly distributed particles.

Then the atoms proceeded to self organize into perfect spheres called planets and stars which in turn organized themselves into flat ordered spiral structures called galaxies.

If entropy is increasing always how does this happen? It happens because your understanding of entropy is off.

I've always considered the "increasing disorder" definition of entropy to be a simple analogy that most people can relate to in their everyday lives, not necessarily a formal definition. Indeed I would be interested to know if you can even formally define order vs. disorder since it seems to require a subjective observer. For example the universe doesn't consider a glass sitting on the table to be "more ordered" than one that has shattered on the floor. It's just like, our opinion man.

But even if you use the order vs. disorder analogy, it's arguable that the universe was close to perfectly ordered right after the big bang. After all it was a pretty smooth and uniform distribution of matter/energy at some point, and that seems more ordered to me than what we have now with all these random clumps of star junk everywhere :)

Anyway I like to consider entropy as the amount of usable energy in a system. A whole glass has more energy to release (e.g. by shattering) than a shattered one does. Maybe this is the formal definition of order vs. disorder?

> Anyway I like to consider entropy as the amount of usable energy in a system. A whole glass has more energy to release (e.g. by shattering) than a shattered one does. Maybe this is the formal definition of order vs. disorder?

I think this is correct. Like the sand castle example: there are very few ways the grains can be arranged in order to be a sand castle, but orders of magnitude more ways of being re-arranged into a pile of sand. It could then be said that the sand castle has less entropy, and left to nature, will slowly but surely move towards the predictable high entropy state of being a pile of sand. The reverse is not true. Without work the low entropy state will struggle to reach the orderly state of the sand castle.

Someone else mentioned the big bang, which I guess would be the work that created the orderly universe as we see it. But that work was expressed a long time ago and without more work matter will just move to an un-orderly state again.

"Useable energy" is vague word and thus not a good description.

There is a quantitative definition of entropy but it is based in a way off of "opinion". Given a system and its laws look at all possible final states. Then group the states according to an arbitrary "rule"

For example: the state of white marbles and black marbles in a jar.

What is the number of possible arrangements of all black marbles to be arranged on the left side of the jar and all white marbles to be arranged on the right? This is an arbitrary "rule." It's a big number but that number is much lower than every other possible arrangement.

From these quantities you can derive an entropy value.

However you will note that it depends on that "rule" you define. I could point to the state of marbles in the jar after I shake it and call that my "rule" and it makes that arbitrary state one of low entropy. I could point to any arbitrary state and do this and thus any specific state is one of low entropy. It's a complex definition and it encompasses "opinion" and "choice." The definition of entropy allows for an numerical value to be derived based off of your "opinion."

In Boltzmann's definition, entropy is a measure of the number of possible microscopic states (or microstates) of a system in thermodynamic equilibrium, consistent with its macroscopic thermodynamic properties (or macrostate).

The macrostate is basically what I defined above as "rules" and what you defined as "opinion."

There are two things wrong with your analysis: one, the universe will not remain in the state of perfect spheres of stars and planets, it will eventually end up in a radiation and fundamental particle soup as all complex particles decay and the universe’s expansion overrides gravity. Plus, you’re not looking at the whole picture: a random gas cloud has potential energy, while the spherical object it collapses into has much less-the increase in order means that heat and other disorder is produced in the process, which leaves the system.

>one, the universe will not remain in the state of perfect spheres of stars and planets, it will eventually end up in a radiation and fundamental particle soup as all complex particles decay and the universe’s expansion overrides gravity.

Right, that's a technicality. Before that happens my statements about entropy apply.

>Plus, you’re not looking at the whole picture: a random gas cloud has potential energy, while the spherical object it collapses into has much less-the increase in order means that heat and other disorder is produced in the process, which leaves the system.

So the universe automatically divides itself into solid spheres and excess heat radiation? Sounds like a type of organization to me. Keep in mind, according to the definition of Entropy, as the universe transforms itself into this state, it is STILL transforming into a state of higher entropy. I am not talking about energy exiting or entering the system.

I am saying in a closed system, particles condensing into spherical planets is a transformation from a low entropy state to a high entropy state. Spherical planets are a high entropy form of disorder according to the definition of entropy. You'd know this if you truly understood entropy.

This is very different from something like life on a closed system like earth, where energy is continuously fed into the system then dissipated... overall lowering the entropy of earth but increasing entropy of the universe overall...

>The second law of thermodynamics states that “as one goes forward in time, the net entropy (degree of disorder) of any isolated or closed system will always increase (or at least stay the same).”[1] That is a long way of saying that all things tend towards disorder. This is one of the basic laws of the universe and is something we can observe in our lives. Entropy is simply a measure of disorder. You can think of it as nature’s tax[2].

I don't understand this. Please note that I've never had any physics training so I'd love for someone with a more formal physics education to help me out here. Why is this a measure of "disorder"? In my eyes, a perfectly uniformly distributed of zero temperature mass (or maybe absolute zero, not sure) of grey energy is the perfect order?

The fact that stars are forming together and creating heat -- that -- is the disorder in my eyes. We are all disturbances in the energy of the universe. I would call a perfectly uniformly smoothed out heat-death of the universe perfect order (i.e. the opposite of chaos).

Or am I completely wrong here?

You're making the (very common) mistake of taking a poetic English-language interpretation of a physical law -- which is necessarily lossy/imprecise, since the actual law is mathematical -- and over-interpreting the words without referring back to the actual law.

You can see the law (in the context of thermal & statistical physics) at https://en.wikipedia.org/wiki/Entropy_(statistical_thermodyn.... You can of course argue that 'disorder' isn't a good way of translating the maths into English, but shrug, it's the one people seem to have settled on. And of course, that's orthogonal to whether the actual law is true or not.

'Information' is another way to translate. See https://en.wikipedia.org/wiki/Entropy_(information_theory)

It always confuses me as well (and I have at least some Physics background, having done a year as an undergrad). I think this is because I confuse it with the statistical measure of the entropy of a distribution, where a uniform distribution will maximise entropy.

I'm trying to learn all this, I have the same issue. So far my best interpretation is that order means energy-transfering organization, a solid is ordered, a gaz is not. It's not to be seen as complex vs simple, a machine is more complex than a bunch of cold molecules, but it's would be less chaotic in that sense.

Another definition is the number of possible microstates that can be reached. A cold universe of evenly distributed particles would be the upper limit of these microstates.

ps: coursera just started this https://www.coursera.org/learn/statistical-thermodynamics

Personally, I don't like the description of entropy as disorder.

Mathematically, it's a measure of spread of a probability distribution (the more spiked a distribution, the lower the entropy, which is an 'ordered' state insofar that things are arranged in specific places instead of randomly thrown all over the place).

Depending on your point of view, I would describe entropy in physics as microscopic indeterminacy (essentially the number of microscopic states that are part of your statistical ensemble defined in terms of macroscopic constraints), or as microscopic freedom (essentially the volume of microscopic phase space accessible to time evolution of a system in thermodynamic equilibrium).

It might be easier to view entropy of a state as sort of the probability that that state would occur after randomly arranging all of its parts (usually particles). In fact this is pretty close to the actual definition.

Here's an example: Suppose you had a container and you put some marbles at the bottom such that there are reasonably large gaps between the marbles. Now if you close and then shake the container hard (this is the 'random arrangement' process), then it is with high probability when you open the container that the marbles will be arranged reasonably spread out across the bottom. In this sense, when the marbles are spread kind of far apart in the container, it is in a state of high entropy (probability).

There is also the chance that when you open the container the marbles form a perfect straight line at the bottom. However, this is much more unlikely, so that you would consider this a low entropy state.

Now, consider the universe in place of the container, and subatomic particles as the marbles. Randomly configuring all of these particles, it would be really hard to end up with a stars, galaxies, humans. These are like straight lines in the marble example. It is much more likely you just get a soup of somewhat uniformally distributed particles (as in the marbles just being sort of randomly spread out from each other). This is what you describe as "a perfectly uniformly distributed of zero temperature mass (or maybe absoute zero, not sure) of grey energy is the perfect order". It wouldn't really be 'perfect' in some senses of the word though, which may be your cause of confusion. It would be more like if you took a snapshot of the static on an old TV.

The common depiction is of a bunch of particles of two kinds, sorted into separate compartments by kind, which will mix if a divider is removed. In this picture, every part of one kind receiving a partner of the other kind, that can be called a state of order. If we take heat instead, every particle swinging differently is more chaotic--at least in my understanding--than all synching up so that their speeds relative to each other are zero ... 0 degree is just not higher order, because "high" is associated with high frequency (or energy or order).

In your shaky example, you transmit energy to the system by shaking, so if shaking a certain way, you'd well expect standing waves, if shaking a bit more you'd pulverize the marble and ultimately a hot plasma with density gradients. I wonder how hard you'd have to shake and swirl to eventually get a black hole, for which the notion of entropy doesn't even really make sense, if you aren't inside.

I think the response you got pointing out that "disorder" is a poor approximation of the actual physical law is the best explanation but it's probably dissatisfying.

I like this analogy from Sean Carroll, explaining the difference between disorder and complexity: the moment you first pour cream into your coffee, the coffee-cream system is low entropy. All the cream is at the top, all the coffee is at the bottom, none is mixed together. It's very easy to describe the system. If it was stored as data, you could compress the data very easily. After you stir the coffee for a while and it's all mixed up, it is in its highest entropy state. It's all smooth and mixed together. This is also very easy to describe: everything is mixed up. It's in the middle where you get all the neat swirrles and interesting patterns. We're living in the middle state, between the low entropy moment at the big bang and high entropy heat death of the universe. The galaxies and clusters and life are the swirles.

Entropy is more accurately defined as hidden information, or the number of possible microstates that a system in a certain macrostate could have. Here's a great lecture by Leonard Susskind on the topic: https://www.youtube.com/watch?v=n7eW-xPEvoQ

What distinguished macrostates from each other? What distinguishes a macrostate from a microstate?

https://en.m.wikipedia.org/wiki/Microstate_(statistical_mech...

>> a perfectly uniformly distributed of zero temperature mass ... is the perfect order

At max entropy, things would not be uniformly distributed: equal distance between each atom. Rather they would be randomly distributed. There'd be some clumping but without much pattern. If such a system were of near infinte size, the number of variations of its alignments would far exceed the much smaller near-infinity of slightly or somewhat more ordered possible systems in the same universe. Especially in the case of your uniformly ordered one, which is almost unique in that universe.

This is entirely incorrect. Our solar system started out as a gas cloud of randomly distributed atoms which then proceeded to become atoms organized into spheres called planets. This happened spontaneously and is not the result of energy entering or exiting the system.

Almost all atoms started out in a state of random distribution before transforming themselves into almost perfect spheres orbiting in an almost perfectly circular orbit around a giant glowing sphere called a star. These stars then organize themselves into beautiful ordered spirals called galaxies. Entropy always increases and most people think high entropy is a state of random distribution, but the organization of atoms in the universe is definitely not heading in that direction.

Thanks to gravity the universe will not become a place where atoms are perfectly randomly distributed. This does not make sense from an intuitive perspective and observational as well.

Strange right? Entropy has not reversed. What's going on most people don't have an intuitive grasp of what the true definition of entropy is.

Allow me to elucidate.

The "disorder" that entropy refers to can actually take on many forms. In the case of our solar system it is perfect spheres that orbit another glowing perfect sphere. In the case of food coloring dispersing in a glass it is atoms randomly distributed.

Entropy simply put should not be called a law it is really a consequence of probability. If the organization of atoms is in a high probability state than it has high entropy if it is in a low probability state than it has low entropy. Put it this way: atoms and gravity by probability are more likely clump into perfect spheres rather than stay in a perfectly randomly distributed gaseous state. Spherical planets are then according to this definition entropy, at a higher state of entropy than a gas dust cloud.

Seem crazy right? It seems like entropy is reversing... randomly distributed atoms organizing themselves into clumps of perfect spheres... The reality is nothing crazy is going on. Intuitively we know gas dust clouds organize them selves into spherical planets and solar systems. It's your definition of entropy that is off. Are intuitive notions of disorder and chaos are clashing with what entropy defines as 'disorder' and 'chaos'

Wait, what happens if we look into the collapsing of the galaxy from an informational-theoretic view of entropy?

As far as I know, thermodynamic and informational-theoretic entropy are the same except for the Boltzmann constant. It certainly seems to me that the informational entropy of the galaxy is lower than the original gas cloud.

That is, if I wanted to describe the state of a single atom in the gas cloud (lets presume the number of atoms is constant) using a perfectly tuned compression algorithm, I'd need more bits to do it than if I wanted to do the same for an atom in the current galaxy (again using a compression algorithm perfectly tuned to that situation).

Perhaps, because the gas cloud was cold, but the galaxy is 'hot' we actually need more bits because the individual particles can go faster, and thus have a larger range of possible speeds? Is this indeed enough to compensate for the much easier to describe location? Do I need to take more than just location and speed into account?

I'm less familiar with the information perspective of this topic.

Why would you need more bits to describe a single atom in a galaxy vs a single atom in a gas cloud?

If a set amount of bits are required to describe an atom then that amount will not change whether or not the atom is in a galaxy or a gas cloud.

If we have a lot of similar locations, we can use compression to reduce the amount of bits required.

A basic example of this compression would be to take the current location and represent it with two parts. The first gives the location accurate to the nearest light-second, and the second gives the location within that cubic light-second.

Now, we can do huffman-coding [1] on the first part. In a gas cloud, the atoms are distributed over all possible cubes of light seconds. In a planetary system, they are concentrated in a much smaller set of planetary light seconds. This would make the huffman-coding a lot more efficient.

There are almost certainly smarter compression methods than what I just described, but the above serves as an illustration. The information-theoretic entropy of a distribution can be seen as a lower bound on the average amount of bits needed when applying compression to the distribution. I believe (am not sure) that this is a tight lower bound (i.e. there exists a compression method that gets as close to the lower limit as you want)

> This happened spontaneously and is not the result of energy entering or exiting the system.

This is not true: gravitational collapse usually produces energy in the form of heat that radiates away, making the surrounding universe more entropic.

When you look at the universe as a whole as a closed system this is happening to every particle in general. The system is organizing atoms into perfect spheres while making heat radiation randomly distributed. There is still a division here, still a form of order and organization happening spontaneously.

However this system is still ascending into higher entropy. What is wrong here is your notion of what entropy is. You think that high entropy as something similar to very random but uniformly distributed noise. This is not what entropy is. The definition is more complex and can encompass very organized structures as well.

>Why is this a measure of "disorder"? In my eyes, a perfectly uniformly distributed of zero temperature mass (or maybe absolute zero, not sure) of grey energy is the perfect order?

Order here means something specific.

A sand castle has more order, in that sense, than sand thrown around (e.g. the default state of sand in a beach) -- the sand in the castle is ordered in specific ways to represent something.

>I would call a perfectly uniformly smoothed out heat-death of the universe perfect order (i.e. the opposite of chaos).

Again, chaos in this sense means (to put it in layman terms) that nothing is put in some particular order. The "perfectly uniformly smoothed out heat-death of the universe" means nothing is explicitly ordered in some way -- any chunk of that universe that you take will have an equal distribution.

Here's another example:

A layered cake has an order (e.g. chocolate, cream, biscuit, marmelade, biscuit).

A layered cake mixed up on the blender on high, has no such order, it's all a chaos (even though it's smoother).

" Too little autonomy for employees results in disinterest, while too much leads to poor decisions."

The author appears to confuse autonomy with a lack of feedback mechanisms. Economies are based on a vast number of autonomous agents (independent organizations) who are selected for survival via market feedback (prices) and the long term trend in these has been towards ever greater agent granularity (vertical disintegration).

The definition of the second law of thermodynamics at the top is not right. A closed system can have energy put in from the surroundings to decrease entropy. Only isolated systems that cannot exchange energy with the surroundings follow the second law. Only the universe is truly isolated (see the common definition of the second law),

You decrease the entropy (disorder if you like, or not) of things by putting energy in.

> A closed system can have energy put in from the surroundings to decrease entropy.

Then it wouldn't be a closed system anymore.

What you call an isolated system is the same as a closed system. At least, in the common parlance of thermodynamics.

Closed means mass cannot go in or out. Heat and work can still be exchanged.

See https://en.m.wikipedia.org/wiki/Isolated_system

Entropy has been increasing since the big bang, when it was at minimum value. While the total energy didn't change since then, there has been a vast increase in the potential locations of that energy and a vast increase in the number of different interactions possible within that energy. This increases entropy because the possibility space has increased.

Boltzman entropy is defined as the number of potential microscopic possibilities that could equivalently produce the measurable macrostate which is observed. (No wonder people just call entropy disorder -- it's complicated). So, the entropy of a glass of water consists of all the possible combinations of position and momentum that all the water molecules could be in. When the water is frozen, the possibility space is smaller than when the water is hot.

https://en.m.wikipedia.org/wiki/Entropy

How does this relate to order? I think of this in terms of the likelihood of a random transformation of the system to affect it's functional interaction with other systems. So, let's say you randomly transform the position or momentum of a part of a house, it is likely change functional interactions of people in that house. However, when the same material in the house is just sitting in a pile of rubble, it has more entropy because a random transformation of the rubble won't really affect the function of the rubble -- it is still, for all intents and purposes, the same pile of rubble (even though it would take just as many bits of information to describe with precision).

I'm not a physicist, but a humanist who thinks it is important to understand physics. So open to correction here.

Thermodynamical entropy is not a physical property of a system. It’s a property of our description of the system as a macroscopic state.

Quantum (von Neumann) entropy is a related but different concept. It’s worth noting that it is constant for a closed system.

Cosmological entropy can be defined in different ways. In summary, entropy means many things and not all “entropies” behave in the same way.

Interesting enough, even though entropy is in some sense a property of a description of a system, there are things like 'entropic forces' and they play a big role in physics.

https://en.wikipedia.org/wiki/Entropic_force

Entropic forces are not more real than centrifugal forces or the Coriolis force. They appear when we use a thermodynamic description of a system.

That wikipedia page doesn’t make much sense, but note that the “mathematical formulation” is about macrostates and canonical ensembles.

Yes, no more and no less.

Thermodynamical entropy is not a physical property of a system.

From a thermodynamic point of view, it is: As ΔS = ΔU/T and assuming one holds temperature and inner energy to be physical properties, then so is entropy.

The thermodynamic point of view is a description of the system as a macroscopic state.

And properties of macroscopic states are 'unphysical'?

Macroscopic states are not a complete description of physical systems and the same microstate could be part of different macroscopic states.

The thermodynamic variables are not properties of the microstate (the "physical reality"). They are properties of the macrostate, which is a model of the unknown "physical reality".

If you don't like the original phrasing, let's say that "Thermodynamical entropy is not a fundamental property of a system. It’s a property of our description of the system as a macroscopic state."

How could the same microstate be part of different macrostate? Like, if I described the microstate of a drop of water, wouldn't it have, as a whole, one temperature? I thought that a macrostate could have many different microstates, but not vice versa

Consider for example, a system in thermal equilibrium with a heat bath, with a fixed number of particles and a fixed volume. The macrostate is defined by the variables T, V and N.

The energy of the microstates corresponding to the macrostate is not fixed. The probability of each microstate is proportional to the exp(-E/kT). https://en.wikipedia.org/wiki/Boltzmann_distribution

The microstates don't have a temperature: they are more or less likely to happen depending on the temperature.

And that's assuming a particular thermodynamical model. The same microstate could be looked at in the context of a different thermodynamical model as a different (or differents) macrostate(s) characterized by another set of thermodynamical variables.

What are example physical properties of a system that can be separated from the properties of "our description of the system as a macroscopic state"?

This was good read

Reply on news.ycombinator.com