Hacker News Reader

Comment by kryogen1c | Hacker News Reader

> You can encode anything into the "language

Im just a layman here, but i don't think this is true. Language is an abstraction, an interpreative mechanism of reality. A reproduction of reality, like a picture, by definition holds more information than it's abstraction does.

View on HN

I think his point is that LLMs are pre-trained transformers. And pre-trained transformers are general sequence predictors. Those sequences started out as text or language only but by no means is the architecture constrained to text or language alone. You can train a transformer that embeds and predicts sound and images as well as text.

A picture is also an abstraction. If you take a picture of a tree, you have more details than the word "tree". What i think the parent is saying, is that all the information in a picture of a tree can be encoded in language, for example a description of a tree, using words. Both are abstractions but if you describe the tree well enough with text(and comprehend the description) it might have the same "value" as a picture(not for a human, but for a machine). Also, the size of the text describing the tree might be smaller than the picture.

> all the information in a picture of a tree can be encoded in language

What words would you write that would as uniquely identify this tree from any other tree in the world, like a picture would?

Now repeat for everything in the picture, like the time of day, weather, dirt on the ground, etc.

Reply on news.ycombinator.com