> If you have 1000 bits of semantic information you want to transfer, you can't give 300 bits of semantic information to an LLM and have it fill in the remaining 700, because it doesn't know what those 700 bits are.

LLMs do inference or computation among other things, so the remaining 700 bits can be something like that. The hidden implication in your claim is that computation adds no information content, which leads to an interesting philosophical discussion.

So for example, if I ask an LLM to give a proof or derive a new theorem from a set of axioms, according to your assumption, if it answers correctly, then I haven't learned anything new.

I am not really sure how to resolve this paradox in information theory.