Comment by xahrepap | Hacker News Reader

Comment by xahrepap | Hacker News Reader

Parent Full thread

I've had a few moments with ChatGPT that are great anecdotes similar to your own:

- Asked it to generate a MadLib for me to play that was no more than a paragraph long. It produced something that was several paragraphs wrong. I told it "no. That's X paragraphs. I asked for one that is only 1 paragraph long" and it would respond "I'm sorry for the misunderstanding. Let me try again" and then would make the same mistake. It never got it right

- Asked it, "Can you DM a game of Dungeons and Dragons?" and it said something like, "Yes! I'd love to DM a game of Dungeons and Dragons for you". Dumped some text to the screen about how we'd have to adapt it some. I asked it to begin, and it asked a few questions about the character I would want to play. I answered the few questions it asked. Then it finally dumped a page of text to the screen as "background" to my character and the quest I was going to embark on. Then it said something like, "You win. Good job! Hope you enjoyed your quest!"

I showed these to my family and they were all a little deflated about AI. Like they realized how willing it was to pretend like what you wanted and just make up its own answers.

There's definitely a potential for a D&D DM with an LLM, but you'd need a lot of careful prompting and processing to handle the token limits today's models have. Simply put: a d&d game has more story and state than the 30,000-ish words an LLM can think about at once.

I think there's a lot of interesting opportunities there.

Aetolia tried this in a 24 hr experiment: https://nwn.blogs.com/nwn/2023/04/aetolia-mud-virtual-world-...

Sounds like it was a success! I suppose it comes down to cost - I think it'd be fun to try a single player game authored like this and would be willing to use my own API token to try it out.

I've also heard (here) that after you get 20-ish questions into an instance you start getting the really weird output. Some of the conjecture was because that's about how deep they trained.

In any case, if that's true, that's a very short role playing session, unless there's a good way to retain info but reset the state that accrues and causes problems (if indeed that happens).

Yeah, but you could ask the LLM to do a few different things at each step.

You could provide it with the background, the story, the secrets, and the summary of everything that has happened so far- as well as what new things have taken place. Then ask it to re-write the summary of the story so far.

Separately, you could give it all that context and what the players have asked of it, and as what response to give.

As well, you could be recording all the events that have happened in a vector store, and do a search on it when players ask questions, and use those as context to the LLM when asking it what to reply.

There's lots of neat tricks we can use to help an LLM overcome it's limitations.

That’s the whole point of these "agents", and things like LangChain or LlamaIndex.

Haven’t gotten around to that part yet, it seems it could help.

'AI Dungeon' has been a real product since GPT-2. You can buy it on Steam.

https://en.wikipedia.org/wiki/AI_Dungeon

I find those example quite encouraging, actually. In that it shows that the current SOA is still pretty far off from creating a planet-killing and/or species-ending AI.

Did you really expect ChatGPT to be a full blown dungeon master?

With all the hype about it, the average person does.

Eventually, sure. But the point is that ChatGPT claims to be. But then does nothing more tell a story and then say “You win!”

Reply on news.ycombinator.com