Hacker News Reader

Comment by themaximalist | Hacker News Reader

sorry about that, the primary issue is the context length gets too long and ChatGPT starts throwing errors

it'd be interesting to create embeddings for the game and try to create longer games that way, though it might hallucinate more frequently.

gpt-4-32k should also be a big help to creating longer games

View on HN

Yeah! I'm going to ramble for a bit because I'm about to leave the house and just want to print all my thoughts on the matter. I actually think there will be a new genre of 'non-fiction' games once hallucinations are minimized. For example I experimented with this animal game for Wikipedia, where you could play about any animal and learn about its unique lifecycle and so on. You would go through a few years of its life, die, and be reborn as a new animal. Was really fun especially if you start at the beginning of life on Earth, GPT-4 will already let you evolve pretty reasonable traits. It can simulate living through historical events etc. The problem though is costs... if we want the game to be accurate biologically, eg. for plants or bacteria the decision are boiling down to much more simple chemical responses, than more abstract human concepts --- then my prompts are way too big. GPT4 is too averagey it get it perfect without defining a ton of end cases. And then getting it to not hallucinate, just makes it not feasible. Scraping Wikipedia for random animals, loading in that data, and making sure that animal, its environment, the types of actions it can take, and so on is just too expensive to prompt engineer. I tried to simulate all the scenarios out and print it into mermaid.js and it is a cambrian explosion. At the point I had an interesting game with realistic characteristics I was spending $10 every few hours just by myself to test it. So IMO there will be a fine tuned model for this task... Chat GPT 4+ will probably be used for the highest level function for deciding if a goal was met, understanding its own limits, etc. and more finely tuned models will live down in the stem for example to check if an action an animal can take is realistic (eg. I am a bear, so I have a paw, so I can do this)... that kind of stuff needs to get hardcoded into the model at the bottom layers.

Reply on news.ycombinator.com