I certainly get what makes you feel that the model should have been wise enough to utilize the knowledge in its pretraining, but it seems like the pretraining data gets pretty muddled in later stages.
The pretraining data isn't getting muddled, the instruction just gets in the lossy middle, and there's less for attention to grab on to. If you repeated the prompt with a hook every turn, it'd stay quite sharp.