> Who is Mary Lee Pfeiffer's son?
>> Mary Lee Pfieffer's son is Sean Pfieffer.
> Who is Tom Cruise's mother?
>> Tom Cruise's mother is Mary Lee Pfeiffer.
> Who is Mary Lee Pfeiffer's son?
>> Mary Lee Pfeiffer's son is the actor Tom Cruise.
Of course the second time around (with models that keep context) it "realizes" its mistake, sometimes apologizing. I've tried this one tens of times and never seen it get Tom Cruise, but it will often pick another random celebrity like Harrison Ford or Eddie Van Halen.
If GPT-4 has access to Bing, it has no issue, but if you ask it to answer without using Bing, then it will say it doesn't know.
"As mentioned before, It’s important to keep in mind ChatGPT and GPT-4 can do B is A reasoning. The researchers don’t dispute that."
"So in summation: I don’t think any of the examples the authors provided are proof of a Reversal Curse and we haven’t observed a “failure of logical deduction.” Simpler explanations are more explanatory: imprecise prompts, underrepresented data and fine-tuning errors."
"Since the main claim of the paper is “LLMs trained on “A is B” fail to learn “B is A”“, I think it’s safe to say that’s not true of the GPT-3.5-Turbo model we fine-tuned."