The encoding used by GPT-2 and GPT-3 greatly obscures many of the textual properties of words. This at least partly accounts for why it has so much trouble with meter, rhyme, syllables, and some math.

More info: https://www.gwern.net/GPT-3#bpes