Also when you use a dataset containing watermarked stock photos and the watermark is treated as a signal instead of noise :) https://i.imgur.com/O1ZRwSF.jpg
But now, in 2019, I've fully acclimated such nightmare-fuel concoctions: "Oh look, another GAN cat!"
There are three of these generators that have shown up on HN in the last few days—people, cats, and anime faces—and what the other two (more successful) ones have in common is that the things they're trying to generate all have the same basic shape and structure: that of a face.
The cat images in the training data are clearly just cats from every angle. There's much less of a clear structure for the neural net to recognize and reproduce.
That it's doing as well as it is is actually kind of remarkable, given that issue.