Before I read the article, I was about to comment something like "generation, prediction and compression are broadly isomorphic to one another, so I don't think it should be "surprisingly" great."
But this is about audio and image compression using text models, which is quite surprising!
This is interesting, but is it that surprising? Many of these models are fairly large (usually measured in GB, as the article rightly points out). But how would this use of space fair against a dictionary of the N most common sequences seen in a space of choice, using the same amount of space? I would be surprised if someone hasn't already tried this (or perhaps just a larger scaled up cousin of Huffman coding).