Hacker News Reader

Comment by captaincole | Hacker News Reader

I really had fun reading this and thinking deeply about the blurry Ill defined line between things that are alive (an individual) and things that are not.

To take this experiment to a conclusion which the author has “left to the reader” as all good texts do, let’s think about a virus.

A virus has information, in the form of dna or rna (I believe this is what a coronavirus uses), that it injects into a cell to then cause the cell to create more virus. The goal of this is to pass “information” to successive versions of the virus which are then programmed to do the same thing. Since they are trying to keep the same information, then they are in fact attempting to reduce entropy across time therefore they are as the author defines, an “individual”.

Would love for someone to test that logic train that I just rode

View on HN

The blog entry is quite good, but it seems to suggest that the breakthrough consists in analyzing living organisms through the lenses of information theory. This is not a mistake in itself, but a shallow perspective nonetheless: information theory is, supposedly, a mathematical representation of the ways our brains make sense of the world around. The blog entry accidentally gives leeway to readers think that there is some intrinsic quality to living beings that transforms dull matter into information propagation channels.

As expected, the article does a better job in setting aside this line of reasoning[1]. What you are really measuring is how information is persisted over time, and over the "background noise" of environment. They specifically talk about viruses at a point, and the usual odd distinction of them being not quite living creatures. When they are found sitting outside their hosts, it is very hard to point out if they actually carry information over time.

[1] Disclaimer: I have just skim-read it.

If I could edit the parent comment, I would replace “our brains” by “an observer”, which is common physics parlance.

As far as I know, all uses of information-theorical approaches outside of its original scope also involve a scenario where an observer is originally unable to formulate a model that allow them to draw conclusions from alleged discrepancies. This paper is just one more example of that.

This somehow reminded me of the following quote by Bertrand Russell. “Physics is mathematical not because we know so much about the physical world, but because we know so little; it is only its mathematical properties that we can discover.”

Maybe I misunderstood you here but I don’t think information theory and in particular entropy has anything to do with “our brain”. It’s simply a statement of the number of micro states in a system. There is no requirement for reading them out or having a brain look at it. This value exists without observer. In that sense there is indeed an intrinsic quality to living beings that leads them to have a reduced number of micro states. They spend energy to keep it that way over time.

I know this is a pretty bold claim, but it is hard to overlook the idea that information theory as proposed by Claude Shannon works as a model of information as related to human cognition, and that it does not imply that information is a fundamental property of the physical universe.

There are several interviews in which he mentioned that the understanding of how the human brain works was one of his main inquiries. His work on AI and the Theseus. And the seminal paper on information theory contains a section in which a fidelity evaluation function is defined with relation to the human ear and brain.

Shannon wasn’t the first to think about entropy. It’s a general concept that is hugely important in thermodynamics and statistical physics. Information theory itself becomes increasingly more important in fundamental physics, see the firewall problem of black holes. It’s even harder to overlook these ideas and claim the human brain and our ears are somehow relevant to the definition of information and entropy.

Indeed, he was not the first at all. The concept of thermodynamic entropy predates, with a huge margin, its information theoretical counterpart.

You make it look like I am saying there is a mystical property of the human brain that backs the validity of information theory, an assertion that has no resemblance at all which what I am attempting to express, and sits in the same category the ones I am trying to debate against do.

What I am trying to say is that there is no meaning in a teleological interpretation of information theory. Things like “the purpose of living organisms is to propagate information” contradict information theory because there is no such a concept as absolute information, you always define it on mutual terms.

> there is no such a concept as absolute information, you always define it on mutual terms.

This is true of classical information theory, but not (as I understand it) algorithmic information theory. I think the jury is still out on whether it makes philosophical sense to generalize over universal machines like AIT does, and the practical applications compared to classical information theory seem minimal, from a layman's view it seems like it's been mathematically very fruitful.

My understanding is that, in order to recognize information, you should at least compare it to the outputs of a source of entropy.

It is a common mistake to conflate information and entropy. A bad analogy to mechanics is that the outputs of an entropy source are the frame of reference, the signal is a body and information might be whatever property you wish to analyze, such as velocity or acceleration.

> in order to recognize information, you should at least compare it to the outputs of a source of entropy.

Again, this is correct for classical information theory which requires some frame of reference for "likelihood". But AIT claims a "global frame" over the minimal representation in all universal machines, the particular choice of machine being at worst constant overhead.

You can argue, I think somewhat plausibly, that this frame is still an (inter)subjective frame rather than an objective, absolute one. But if we assume C-T (and we virtually always do), that argument is pretty weak - any other definable frame becomes formally "worse" in that it becomes "merely" a specific case of the universal one.

I am delving into speculation here, since I am not familiar with AIT and I don’t know any other definition of algorithmic information besides mutual information as defined by Kolmogorov which, remember, relies on Kolmogorov complexity but is a separate concept.

My point is that if I ask you, “given a bit sequence A, is it an optimal program?”, your answer would probably be: “I cannot even say if this represents a computable function and, also, is it an optimal program compared to what?”. You must establish a frame of reference such as the Kolmogorov complexity of a given program.

> “given a bit sequence A, is it an optimal program?”

> Kolmogorov complexity of a given program.

You seem fundamentally confused about the objects of study of information theory. They're not programs, they're e.g. strings of symbols. We measure by the information content of those strings based on likelihood / programs. Information theory asks "given some bit sequence A, how much information is in it?" not "is it an optimal program?" - instead we measure the information in it by constructing or otherwise proving facts about programs that generate or predict it. We talk about Kolmogorov complexity of strings (/ signals / states / whatever) as measured by programs, not Kolmogorov complexity of programs themselves.

Obviously programs are also themselves representable strings of symbols, and this is why we find the usual suspects of self-reference paradoxes in IT. But that doesn't mean the measure does not exist, or that it's not possible to find in lots of interesting, easily-computable cases. It's a bit like handing me a ruler and asking me how long it is - sure, if I don't trust any ruler I'll have a hard time measuring it. But I don't have to trust that specific ruler to do so, and the fact it's a device used for measuring itself is completely incidental to my measuring of it.

> You seem fundamentally confused about the objects of study of information theory.

It is hard to argue against your slightly condescending remark if my comment is not accurate, which is still up to debate. I am sure I could not observe all due formalities even if I tried. But please understand that my comment was written taking into consideration your previous comment, by which I mean:

- You mentioned that the overall approach in Algorithmic Information Theory is to assume Church-Turing thesis as valid. My understanding is that having a standard representation of data is one among the various accidental benefits of that---raw data could pretty well be represented by a Turing machine itself, as well as any other program representation that could generate it as long as it is a computable function. Notice that, in this scenario, talking about the Kolmogorov complexity of a program is valid, as strings of raw data are also represented as programs.

- The "is it an optimal program?" question was a rhetorical device which apparently did not work well, even due to the fact that I did not define what "optimal" meant in this context---I thought it was given. But I can't understand how you came to the conclusion that I was defining the subject of study of Algorithmic Information Theory there.

So if I’m understanding this correctly you have a relativistic understanding of information in which the zero state is observer dependent? Just for my understanding, consider for example the spin of an electron. It can be in one of two states, up or down. In which scenario am I unclear about the absolute information content of knowing the spin state?

It is really hard for me to reply to that comment.

I would say, first, that I need a definition for absolute information, as I have been insisting that information is defined on mutual terms. If we move past that, though, information about the spin state is unclear before measurement.

viruses have no agency so they don't attempt to reduce entropy across time. viruses are a reduction of entropy across time, just by being an organized structure (of rna and protein strands folded together) requiring energy to create and maintain. information is encoded in that structure--both in the sequence and in the shape--and that's transmitted through time.

I think it is a mistake to say that agency is a property of a system, rather than a lens through which we can analyze a system. It truly is useful to describe a thermostat and heater as trying to keep a room within a certain temperature range, even though there is nothing actually striving there.

Much like anything under evolutionary pressure, the viruses that propagate are the ones that do manage (in their normal environment) to keep entropy across time from exploding. Are they "alive"? Meh. In combination with their environment, they act like it though.

What is the least complex entity that you ascribe agency to?

Not OP, but this is something I spend a lot of time with. I have a pet framework called Zodeaism which means "Living Ideas". In my theory, the real "life forms" are ideas which possess the capabilities of information storage, adaptation, self-repair, and transmission. My own consciousness is mediated by thousands of such ideas, some competing and some working in harmony.

One such idea -- Do Good Onto Others As Others Do Unto You -- is an example of an extremely powerful and resilient idea which lives and operates in the brains of billions of individuals. It is powerful enough to ward off weaker ideas and has lived a long time without much modification to its original essence.

With that out of the way... I felt it was necessary to reduce agency to ability to use internal energy in order to put oneself in a higher energy state in the external world. This can be observed externally. I am standing next to a rock. I can jump up, spending some of my energy, and fighting against the potential energy well of gravity. I've increased my external energy state at the expense of some of my internal energy. Thus, a living being needs a way to store and use energy. You can observe this externally and conclude that I am alive, while the rock is either dead or inactive.

I consider such an act of "living" motion which can take another path than that of least resistance to be a "kin". In other words, any motion which is the result of a physical calculation (Zodeaism is compatible with determinism) and leads to an increase in external energy state. A kin is any such motion, large or small.

So now the problem becomes, what is the smallest kin we've observed in nature? Single-celled bacteria can expend energy in order to move through their environment against forces like friction and gravity, but a virus "rides the waves" if you will, never expending energy for things like respiration or locomotion. Any energy which is spent internally is potential energy like chemical or gravitational, released through a physical process without need for computation. I am unaware of anything smaller than a single-celled organism which produces such kins, but that doesn't mean they aren't out there. Even ethereal life forms such as ideas can produce these kins within the bodies of countless individuals across the planet, so physically local computational circuitry isn't a hard requirement.

So, according to this framework viruses aren't alive, however we can make the case that some machines are, except the experience is incomparable because of the advanced circuitry we contain which mediates our experience through things like emotion.

Are you familiar with Friston/free energy/markov blankets?

Yes to all three, when I was exposed to Friston's work I found many parallels in my own research and I would like to reach out to him when I've reached a more complete formalization of my ideas.

What do you think of his free energy principle and related concepts?

i haven't thought about it enough to even speculate, but one criteria would be the ability to make decisions, however simple.

viruses are entirely passive in their action. they're exquisitely complex structures (for what they are) existing entirely by chance that happen to have the property of self-replicating in the presence of the right cellular machinery. they don't decide to do anything, they just are, and therefore don't have agency.

Unfortunately, by your definition, computer programs are closer to being an individual than viruses. I am assuming that is not intended, so probably the idea needs more refinement.

I also think this is a weakness of the original article: a lot of things we probably shouldn't consider individual life forms would probably fit the definition (nations, for example).

yes, good point. some autonomy and skin-in-the-game is probably required too.

the article seems to have sparked some good conversation, so it's been useful, if not as strong as it might be.

Viruses increase entropy like all life. Their reproductive process destroys a much larger and more complex cell.

Plants convert light into chemical energy at low efficiency, the rest goes to waste heat. Animals do the same thing with chemical energy.

yes, as @selestify also noted, that seems contradictory on the surface but actually isn't. the virus itself is a reduction of entropy while it's net existential entropy contribution is positive.

where you draw the system boundary matters for defining entropy changes, but long-term we're all bound to increase entropy, otherwise life (and our semi-living cousins) would be violating the laws of thermodynamics writ large.

When you simply include the raw materials the virus is made from before and after it forms you have a net loss of entropy. So, it’s not clear what you mean by boundaries in that case.

I disagree with life increasing entropy; it does do that but it expels the entropy.

Living things are low-entropy internally, but produce higher entropy externally than would have been without life.

Hmm, I don’t follow the comparison. How could, for instance, the sun use “life” to generate more entropy than it already does?

To some degree depending on how productive they are externally, eg bee vs fox.

I don't know if the goal of the virus is to reduce entropy. Viruses mutate just like some clever computer viruses. Allowing them to trick immune systems and anti viruses and render antivirals useless. Bacterias are doing that too.

I think we can equate too much entropy with danger since cancer cells also mean mutations.

The viruses themselves do not mutate, it is the genetics of the infected cell that cause mutation.

How do you define the information-theoretic entropy of a physical object, like a virus?

Reply on news.ycombinator.com