There are several interviews in which he mentioned that the understanding of how the human brain works was one of his main inquiries. His work on AI and the Theseus. And the seminal paper on information theory contains a section in which a fidelity evaluation function is defined with relation to the human ear and brain.
You make it look like I am saying there is a mystical property of the human brain that backs the validity of information theory, an assertion that has no resemblance at all which what I am attempting to express, and sits in the same category the ones I am trying to debate against do.
What I am trying to say is that there is no meaning in a teleological interpretation of information theory. Things like “the purpose of living organisms is to propagate information” contradict information theory because there is no such a concept as absolute information, you always define it on mutual terms.
This is true of classical information theory, but not (as I understand it) algorithmic information theory. I think the jury is still out on whether it makes philosophical sense to generalize over universal machines like AIT does, and the practical applications compared to classical information theory seem minimal, from a layman's view it seems like it's been mathematically very fruitful.
It is a common mistake to conflate information and entropy. A bad analogy to mechanics is that the outputs of an entropy source are the frame of reference, the signal is a body and information might be whatever property you wish to analyze, such as velocity or acceleration.
I would say, first, that I need a definition for absolute information, as I have been insisting that information is defined on mutual terms. If we move past that, though, information about the spin state is unclear before measurement.
Again, this is correct for classical information theory which requires some frame of reference for "likelihood". But AIT claims a "global frame" over the minimal representation in all universal machines, the particular choice of machine being at worst constant overhead.
You can argue, I think somewhat plausibly, that this frame is still an (inter)subjective frame rather than an objective, absolute one. But if we assume C-T (and we virtually always do), that argument is pretty weak - any other definable frame becomes formally "worse" in that it becomes "merely" a specific case of the universal one.
My point is that if I ask you, “given a bit sequence A, is it an optimal program?”, your answer would probably be: “I cannot even say if this represents a computable function and, also, is it an optimal program compared to what?”. You must establish a frame of reference such as the Kolmogorov complexity of a given program.
> Kolmogorov complexity of a given program.
You seem fundamentally confused about the objects of study of information theory. They're not programs, they're e.g. strings of symbols. We measure by the information content of those strings based on likelihood / programs. Information theory asks "given some bit sequence A, how much information is in it?" not "is it an optimal program?" - instead we measure the information in it by constructing or otherwise proving facts about programs that generate or predict it. We talk about Kolmogorov complexity of strings (/ signals / states / whatever) as measured by programs, not Kolmogorov complexity of programs themselves.
Obviously programs are also themselves representable strings of symbols, and this is why we find the usual suspects of self-reference paradoxes in IT. But that doesn't mean the measure does not exist, or that it's not possible to find in lots of interesting, easily-computable cases. It's a bit like handing me a ruler and asking me how long it is - sure, if I don't trust any ruler I'll have a hard time measuring it. But I don't have to trust that specific ruler to do so, and the fact it's a device used for measuring itself is completely incidental to my measuring of it.
It is hard to argue against your slightly condescending remark if my comment is not accurate, which is still up to debate. I am sure I could not observe all due formalities even if I tried. But please understand that my comment was written taking into consideration your previous comment, by which I mean:
- You mentioned that the overall approach in Algorithmic Information Theory is to assume Church-Turing thesis as valid. My understanding is that having a standard representation of data is one among the various accidental benefits of that---raw data could pretty well be represented by a Turing machine itself, as well as any other program representation that could generate it as long as it is a computable function. Notice that, in this scenario, talking about the Kolmogorov complexity of a program is valid, as strings of raw data are also represented as programs.
- The "is it an optimal program?" question was a rhetorical device which apparently did not work well, even due to the fact that I did not define what "optimal" meant in this context---I thought it was given. But I can't understand how you came to the conclusion that I was defining the subject of study of Algorithmic Information Theory there.