Evidence that transformers cannot generalize beyond their training data

This may be a ML version of "you can't imagine a new color".