Language models like ChatGPT seem to "remember" paragraphs of text in a highly compressed form which is not always fully accurate when reproduced - so a mistake does not tell us whether the text is from memory.

Lossy reconstruction of training data is still memorisation.