Hacker News Reader

Which LLMs perform better or worse will be determined entirely by the scoring formula used and how it penalizes errors. It is not in the nature of an LLM to be capable of lossless compression.

We already do this.

Prediction is formally equivalent to compression, so loss is just a measure of how well you can compress the training dataset.

In a sense, yes. But that's compression of an input/output map, not arbitrary information. Also, it is not lossless.

Lossily or losslessly?

Losslessly, because that's easier to test.

When you're recalling a memory, do you remember the position of every blade of grass, or the exact angle of the Sun?

Humans, the only extant example of a general intelligence, don't do lossless compression at all.

I don't think you get to AGI by trying to compress noise.

Lossless compression requires the recognition of patterns in the data. And a smart way to use those patterns to perform the actual compression.