In chats that run long enough on ChatGPT, you'll see it begin to confuse prompts and responses, and eventually even confuse both for its system prompt. I suspect this sort of problem exists widely in AI.
Delete the bad response, ask it for a summary or to update [context].md, then start a new instance.
The same issues are still happening in frontier models. Especially in long contexts or in the edges of the models training data.
It seems to degenerate into the same patterns. It’s like context blurs and it begins to value training data more than context.
Things get really wacky as it approaches decoherence.
I’ve also had it fail to respond in long chats but I thought it was a network error despite having no error messages.