Hacker News Reader

Comment by stratos123 | Hacker News Reader

> you can't give 300 bits of semantic information to an LLM and have it fill in the remaining 700, because it doesn't know what those 700 bits are. If it's able to guess those 700 bits correctly, then they aren't true semantic information, and you really only have 300 bits you want to transfer.

It doesn't actually follow, because maybe the LLM is smarter than the original writer (at least in the domain the writing is about) and hence really is able to complete the ideas in a way the writer can't. As an existing example, consider formulating a conjecture and having an LLM prove it. But I agree; if I wanted to read an LLM's output I'd simply ask it myself rather than read someone's supposedly-human writing.

View on HN

Lets do a thought exercise:

You are stranded in a desert island. You start writing a message "Help, I am..." and pass at that point.

Somebody finds the message. They can no doubt come up with plausible continuations like "Help, I am Robinson Crusoe" or "Help, I am hungry" but they cannot create information. No matter how smart and how long you stare at the message, that is not going to tell you what the original person would have written.

Isn't it from Claude Shannon that information lowers uncertainty? Infinite regurgitation or massaging of data does not create new information. You will get the information form the LLM, not from that original person.

What do you think about the prompt “The first 6 digits of pi are 3.1415”?

I think the prompt shows that you can't count to 6. :)

Also, "pi" is a unique constant name. You've conveyed far more than 10^[5|6] information in the subject phrase.

Once you've determined what the information-content of a message is, then you can apply information theory to it. But different receivers can derive a different amount of information from the same message.

Consider, for example, that if somebody doesn't know English at all, then before receiving the message, their best guess at what it is is some probability distribution over all English characters (or sounds, depending on what we assume them to know), and after knowing the first part is "Help, I am" that distribution might not change much at all. Therefore, they derived very little information from this message.

Going in the opposite direction: keeping fixed the knowledge someone starts with, there is an upper limit to how sure they could be (even if they are logically omniscient) in completing the message (that is, a lower limit on the entropy of their probability distribution) - this is what you're talking about in your example. But this limit only becomes important under these constraints - for example, knowing more about the person who wrote the message can let you predict it better, and if predictor A isn't logically omniscient, predictor B can do better than it with the same prior knowledge, just by being smarter than A.

>It doesn't actually follow, because maybe the LLM is smarter than the original writer (at least in the domain the writing is about) and hence really is able to complete the ideas in a way the writer can't.

Then what's the point of the original writer?

I think you missed the import bit: "transfer of information from your brain to my brain." Including some third-party necessarily adds more that was not in your brain. So by using an LLM, you are transferring what is in your brain + some sludge that may or may not be useful for the other person's brain.

Reply on news.ycombinator.com