You’ll probably get better results by putting the examples only in the completion part of the training examples.

GPT-3.5 learns how to generalize better when it’s just in the completion.

This is the same problem that vexed the researchers who did the paper on the alleged reversal curse.

(https://andrewmayne.com/2023/11/14/is-the-reversal-curse-rea...)