I couldn't resist asking it about the word entropy of its training corpus and quite predictably it wasn't able to understand my question and instead answered similar questions. However, it stuck quite firmly to a figure of 10--12 bits per word! I'm fairly sure that refers to the vocabulary entropy and not word entropy of English.

Yes, the GPT's are literally trained to minimize (cross-)entropy of text. You can see the bits-per-byte of text for lots of models here: https://crfm.stanford.edu/helm/v0.2.0/?group=targeted_evalua...

I was wondering the same thing. Even the prediction as I type this in is pretty good. It seems like LLM's could be even better.