Hacker News Reader

Comment by daxfohl | Hacker News Reader

Has anyone tried having them DM text games? Seems like they could create a dungeon and DM a game pretty well. It should be easier than playing, I'd think. Though I'd be curious how good they are at making fun games or whether they struggle with that.

View on HN

Yes, many people have tried this. The inherent deference of the language models makes them ineffective DMs - they're unable to effectively enforce rules or create meaningful challenges for the player. Even if that were somehow avoided, though, they're still inherently unable to do any serious planning or worldbuilding, or to conceal information from the player - at best, you might get the experience of playing with a frazzled, unprepared DM who's making everything up on the fly.

Makes sense. In my experience, LLMs are not very good at maintaining deterministic state machines internally either. Our team did an experiment trying use an LLM to guide users through a four step signup flow, and gave it very explicit instructions what order to collect information, rules for validating it, under what conditions a step could be skipped, how many times each step could be retried. The rules were highly simplified, just as a proof of concept. Still, it would get lost half the time. So yeah, I have a hard time imagining it would do any better managing the full state of some RPG it made up

I'm reminded of the infamous vending machine benchmark:

https://arxiv.org/abs/2502.15840

in which some LLMs simply failed to manage the basic task of keeping a vending machine stocked with products to sell, and some went completely off the rails, e.g. by threatening their (simulated) supplier with "ABSOLUTE FINAL ULTIMATE TOTAL NUCLEAR LEGAL INTERVENTION".

Had a friend try to build an AI-driven text RPG. Told me it was rather bland and unimaginative, and gets boring fast.

I've actually have a pretty good experience with gemini-2.5-pro doing this, but I was focused primarily on role-play and collaborative story telling. Definitely some issues, but I enjoyed it.

Reply on news.ycombinator.com