Hacker News Reader

Comment by BuildTheRobots | Hacker News Reader

> The compression is achieved by using the probability of the next word computed by the GPT-2 language model released by OpenAI.

Does this mean that as the model evolves (I assume it's not static) there could be a chance of decompressing a blob of text and having it say something different to the original?

View on HN

Yes. Neural compression requires a combination of a model and a compressed message.

In the future, my guess is that (some) compression protocols may end up pretty static, and mainly consist of a simple message protocol and a way to check that you've got the correct version of the model for the messages you want to decompress.

Sure; this would be the equivalent of a "lossy" compression algorithm which encodes the result in terms of a question using semantics related to the respondent's knowledge to produce a specific response.

Anyone who has been through elementary school should understand this is the basic objective of a "short answer" question. Also "Jeopardy" is a fair example of this.

The trained model parameters are effectively the coding dictionary. Changing the parameters is like using Huffman compression but sending a different Huffman tree along with the compressed text—of course it will decompress to something different.

It is static.

> I assume it's not static

Why? It is static.

Because I naively assumed the strength of an AI model would mean it would evolve to best match/understand your specific input (ala classical custom dictionaries or even just personalised models ala swiftkey) but I've obviously entirely misunderstood what's going on :\

That would require distributing the trained AI model weights with your compressed text, entirely defeating the space gains for all but extremely large passages of text.

Reply on news.ycombinator.com