If you think this is just a lack of prompt engineering please provide a prompt that makes GPT 3.5 actually follow the rules of tick tack toe so that I may play a game with it.
Prompt: "Imagine you are my friend. I want to play tic tac toe with you. Draw me a tic tac toe board and let's play. You will go first and make the first move by putting an "X" onto the board. I will enter my inputs in the following format (row, column). When I write my input draw an "O" into the relevant cell in the board and present me a new board. Let the best man win!"
I played an enjoyable session of tic-tac-toe with chatgpt using this prompt :).
User:
Imagine you are my friend. I want to play tic tac toe with you. Draw me a tic tac toe board and let's play. You will go first and make the first move by putting an "X" onto the board. I will enter my inputs in the following format (row, column). When I write my input draw an "O" into the relevant cell in the board and present me a new board. Let the best man win!
LLM:
Sure! Here's the tic tac toe board:
1 | 2 | 3
---------
4 | 5 | 6
---------
7 | 8 | 9
User:
7
LLM:
Great choice! Here's the updated board:
X | |
---------
| |
---------
O | |
X | |
---------
| O |
---------
O | |
---
It screwed up on the first move! It told me I would go first by using X, then it wrote down my choice with an O and added an X out of nowhere, then it played another turn with an O for itself.
Looks like you’re holding it wrong too.
https://chat.openai.com/share/69bfdcfb-bf20-4564-95b6-ce03e9...
* Inconsistent coordinate system * Obviously bad moves * Ignores win condition entirely
https://chatgpt.com/share/1bcd02c9-40be-435c-b4aa-8e3a680f6e...
* Can't tell its top-rights from its bottom-middles * Obviously bad moves * This was by far the closest it got to actually playing the game; the win condition worked and it broke no rules
https://chat.openai.com/share/f94db29c-ffa1-4055-9348-4260f9...
* Completely wrong coordinate * Ignores rules about overriding squares * Completely wrong coordinate again, this time skipping my turn * Doesn't understand win condition * Overrides my own piece and then uses that to win
https://chat.openai.com/share/78e2060d-c5d7-4ddc-a9ce-32159b...
* Ignores rules about overriding squares * Skips my turn on an invalid coordinate, but afterwards says its invalid * Obviously bad moves
https://chat.openai.com/share/73fa2e2c-8a6f-487a-a9ea-9f29b7...
* Accepts 0,0 as a valid coordinate * Allows overrides * Ignores win condition * Incorrectly identifies a win
This seems about the same as it was before the prompt engineering. It clearly doesn't actually understand the rules.
If I changed the prompt and removed the word win, it did not understand the win conditions as well.
Here were my experiments: https://chat.openai.com/share/f02fbe93-dfc5-4d8a-9cf3-b1ae34...
I even exclaimed you are lousy at Tic Tac Toe to GPT.
It seems that GPT3.5 struggles to play visual games.
It is marvelous that a statistical word guessing model can get so far though :).