A zip file is not a good analogy to a language model as the zip can reproduce the original exactly with 100% recall, language models have very few episodes of verbatim reproduction and coverage is spotty. A zip file can only reproduce the original, but a language model can be asked to write custom, situated code. The language model still works if you block all the training data from appearing in its output.