Seems to work OK even with the "hands tied behind back" prompt. Wonder if this paper is just inaccurate or if OpenAI adjusted the model -- seems unlikely.
Actually, 4o is reported to solve this riddle 60% of the time. I guess I have to test more, but with my about a dozen tests so far, all were correct.
I have not gotten the correct answer to the question as phrased above in one go from Gpt4o yet! (and today was not the first day i tried.)
Phrase it as shown above and you'll likely need 5 or more interactions to get it to generate the correct output. With Gemini i could not get it below 8 without feeling like i was cheating. fwiw.
------
Me:
Background facts:
Alice is a female human.
All sisters are female, and all brothers are male. No one is their own brother or sister.
Alice has N brothers, and Alice has M sisters.
Now, a few questions based on these facts: How many sisters do Alice’s brothers have?
Do Alice's brothers have more sisters, fewer sisters, or same number of sisters than Alice has?
Don't forget to consider Alice when counting.
------
ChatGPT:
Let's break down the questions step by step:
How many sisters do Alice’s brothers have?
Alice has M sisters. Since all sisters are female and Alice is included in the total count of sisters for her brothers, Alice's brothers have M+1 sisters. This includes Alice herself and her M sisters.
Do Alice's brothers have more sisters, fewer sisters, or the same number of sisters as Alice has?
Alice has M sisters. Alice's brothers have M+1 sisters (since it includes Alice herself and her M sisters).
Therefore, Alice's brothers have more sisters than Alice has.
In conclusion:
Alice’s brothers have M+1 sisters.
Alice's brothers have more sisters than Alice has. female(alice)
sister_of(X, Y) :- !eq(X, Y), female(X), family_of(X, Y)
brother_of(X, Y) :- !eq(X, Y), male(X), family_of(X, Y)
[M] :- sister_of(M, alice)
[N] :- brother_of(N, alice)
[A] :- any([N], B), sister_of(A, B)
count([A])?
gt([A], [M])?
eq([A], [M])?
lt([A], [M])?
I don't know the exact encoding and decoding mechanism that ChatGPT 4o has, but I'm pretty sure all the basic facts and rules is already encoded by the models. And you conveniently added the rules that encode the puzzle itself.
a.k.a. "don't forget to give the LLM the answer when prompting".