I would imagine that if you are training on the text of a novel, then anything that happened earlier in the text may be relevant for predicting the next events. Especially if it's something like a detective novel that has clues about the criminal's identity scattered across the story.

Also if you are training on a database of code.