The paper is not talking about information provided in the context, only training and fine-tuning.

> In fairness, it’s also worth pointing out here that they’re making the claim that the reversal curse only applies to training and fine-tuning and not in-context – i.e., putting all your information inside a prompt. They point out in a footnote that you can put A to B data in a prompt and GPT-4 will make B to A connections just fine. Unfortunately, this was lost on many of the people covering the pre-print.