Very exciting times ahead in game design!
So your subscription pays for generations, but also fills out the persistent narrative and lore.
I actually like the 3 choices dialogue system and don't want to come up with my own NPC small talk on my couch, but the idea that any NPC that's been talked to by any player gradually has a whole catalogue of interactions to pull from means there would be compounding depth to the world as more players interact.
Like Minecraft but where it's collective world building by exploration and interaction rather than actual building it yourself.
Humans tend to be pretty predictable, so I feel like the universe would quickly feel like it's insanely deep after just a short while with a somewhat decent player base.
Also I have a feeling that the NPC narratives when enriched by all player interactions would tend to go towards some average, i.e. become more and more generic and dull. The truly rich experiences have always been on the fringes in all art for me. I'm interested to see a counter example to this though.
a) LLMs are excellent at keeping a "linear enough" storyline without being linear. They'll let you do outlandish things, but given the assignment of "tell a cohesive story" they manage to corral the story back to something sensible unless the player intentionally keeps pushing at the boundary (in which case they probably do want things to go off the rails)
b) LLMs can do delightfully colorful dialogue, they just need to be grounded in a character. Everyone thinks of factual grounding, but given enough description of speech patterns, character motivations, etc. they're capable of dialogue that's lively and completely rid of "GPT-isms", which are what tend to break immersion
I actually trained an open model [2] on the task of grounding LLMs in characters and actions as opposed to factual things like RAG, and eventually I want to build a game demo out of it
[1] https://www.tryspellbound.com [2] https://huggingface.co/hf-100/mistral-spellbound-research
Didn't go through the registering phase cause I couldn't find any info on what info you store and on the pricing. Could you provide that?
The pricing page is also visible when signed in and in a story (no dedicated pricing page yet but it's in the backlog)
Since existing worlds are (currently) where most of the stories are set, it's worth it to use a closed source models and wrangle their issues with dialogue.
To it's credit though, Llama 3 is the first OSS model trained on enough tokens to not feel lost for most worlds, so I've started routing some traffic to it for free users
The output format the site uses is also really really hard for most models to follow without fine-tuning, but fine-tuning then causes them to pick up the vocabulary of whichever model they were fine tuned on, which is a bit unfortunate
Obviously we don't want experience averaging NPCs.
> Mad Hatter’s first goal is to ask the player to humor him with a joke.
This is ok for simple stylized, small world, single storyline games, but not for open worlds.
For that, NPCs need their own motivations, so they essentially play the game too. With needs like maintain their smithy, so they can build quality armor, so they can make customers happy, so they can make money, so they can feed their family. I.e. not just being props.
And flexible in how they adjust subgoals to meet their core goals, relative to player interaction: such as being convinced to go on a trek as an armorer, or on a search to find and extract rare materials for a magical shield. Willing to fight in revenge for their home town's sacking.
Westworld got NPC dynamics right.
As c048 said best (https://news.ycombinator.com/item?id=40316308), good art needs direction and focus, and AI NPCs being able to do anything they want in an open world doesn't make for a good story. It could make for a good sandbox with emergent gameplay, sure. But my point was that there are people who yearn for a good story, and a good story doesn't have many options in it, otherwise the story's beats would become meaningless.
Hmm. Maybe with this the problem Rockstar has with their linear missions mixed with open world could be fixed a bit - i.e. the open world NPCs could react in story missions to what you've done in free roam. This would lessen the cognitive dissonance the NPCs seem to have when you can now blow a town into pieces and then start the story mission and no one cares.
Subgoals that are resource, constraint defined, could make very flexible solutions. For instance, if you need help with some subgoal, it is going to play out very different depending on what NPCs you have established credibility with before, and their skills and dynamics.
If you need money, then how you get that money is also going to depend on your history, knowledge of a city, previous connections, etc.
That could provide an overall story arc, but with a very open world and fully functional NPC experience.
So: organized series and trees of goals. Open ended solutions.
So you want a system like rimworld, and a LLM doesn't really help on that front. It can power the dialogues, but that's it, all the logic you describe would need to be encoded somewhere else.
In Skyrim it took me multiple years to finally complete the main story line. I would always look for books, store potion ingredients, walk in the forest...
There's a big role play component in this. The game could learn from the players RP and improve.
1) you need to be able to ensure that the characters don’t hallucinate new details. No big deal.
2) you need to be able to ensure that the characters don’t suggest they can or will do anything that they can’t do. For NPCs, this is typically literally nothing. They can’t do anything. It’s actually way worse for realism and immersion if every character has a unique reason why they can’t do anything than every single guard citing their arrow to the knee and leaving it at that.
3) NPCs for story reasons often should know something but don’t tell you it. It’s very difficult to execute any sort of story flow if your characters are suddenly free agents. At best they merely lack self consistency. Stories are not going to become better if you can meta game them as genre savvy speculators.
4) NPCs with irrelevant backstories are very dull. They are absolutely annoying to talk to. We have these in DND. And you may say “hey, I loved my DND NPC”, but that character probably started to become very important as consistent because of the DM’s attention. Which is cool for DND, but it’s not cool for a structured game.
Like, would Harry Potter be a better experience if you could personally grill dumbledore for information in the middle of the first book? I think 100% no.
Dumb NPCs are an aspect of the suspension of disbelief of game writing. And it largely works. Asking for LLM agents is like asking to remove the need for the suspension of disbelief. But most narratives are too fragile for this to happen.
Narratives generally follow a flow that ultimately concludes after a climax. If you give players more agency the most likely outcome is that they follow a path that does not hit the climax.
But then you are adding a component of "how can I interrogate the NPC?" to the game. Anything you add to a game makes it a different game, and not necessarily a better one.
A good example is dwarf fortress where the narratives are created very dynamically and the player is free to interact with them as they want. LLMs could add immensely to these types of games.
Dwarf fortress creates great stories because YOU are filling in the blanks. It would be MUCH WORSE if you could no longer fill in the blanks because some ai keeps breaking your head canon.
This can be said about all NPCs, yet we bother to script them, and I would hope nobody would suggest we just remove NPC dialogue.
* NPCs that don’t want to speak and usually offer a dismissive comment or a single repeating phrase. Fine candidates to LLM so long as this pattern doesn’t change. But also, who really cares?
* important NPCs with specific written dialogue. Don’t even THINK about gambling with your story’s self consistency by giving these guys agency
* the limited townsfolk type that has a small bit of something say. Like a typical Baldurs gate NPC. These people are fun to talk to because they “might” have something relevant or fun to say. Usually you don’t have to say anything. They just want to share things with you. It would be very tedious to chat with these people if they were infinite wells of lore and nothingness to be probed with written language.
Game devs will still provide structure, "this big baddie is trying to do X, there's this mysterious treasure over here". But the guard rails will be gone, and there will be a real world to explore, not just cutscenes to sit through.
You can’t unlock the guard rails without making the general story completely arbitrary. It’s just not possible.
That's the exact promise. BotW has a main story, but it's not where the real fun is. The game devs will still write some overarching plot for people that want that, but you can just ignore that and live your own story.
There will also be a lot of fun in seeing how far you can stretch the original story to its breaking point, like how people enjoy breaking Skyrim with too many mods. Testing the limits is the fun. "What if I defeated Ganon by taming an army of crabs?"
BotW’s appeal is not the story. It’s definitely not the dialogue. It’s about exploring a carefully planned world full of goodies. Not a shitty LLM procedurally generated one that goes forever. And DEFINITELY not a run time generated one.
This is such a stupidly thought out vision. BotW is a beautiful game that emphasizes minimalism and simplicity and you’re wanting to cram shit like this in.
You can’t do those things because that’s not what the game is about. And if you could do those things, it would be a different fucking game, and the magicalness of the world and the ingenuity of its game mechanics would become nothing compared to your “limitless player narrative agency”
I think your takes suck and have no basis in the reality of plausible to implement technology but this one really crosses the line of just being irritating to imagine asking for. This is the polar opposite of BotW’s design thesis and I hate you for tainting it.
LOL, calm down mate. Fwiw, I've read your back and forth with the GP and I'm much more excited with their vision than yours.
Anyone with any imagination can see that LLMs are the future of story telling.
You can rage all you want, but progress will march on without you.
Giving a chatbot brain to an NPC who would otherwise only get a single generic line to say creates an infinite amount of negative possibility space. A player can waste a literally infinite amount of time probing a chatbot NPC for narratively interesting details, and all they'll get is chatbot slop. Now it doesn't matter if you put something interesting at the end of the hallway: You've wasted so much of the player's time now that they no longer trust you to. They assume you're just wasting their time. You teach your players to be cynical and incurious and to engage with your work as little as possible, because in their experience it's mostly meaningless.
For this reason, I consider the integration of chatbots into video games to fundamentally anti-artistic. It actively pushes players away from the work such that they get less out of it. Nobody wants this, actually. It sucks.
I would phrase that as "A player can have an infinite amount of fun creating an immersive world with the LLM". Any details you want will be narratively interesting, because the LLM will work them into the narrative as necessary.
There's no end of the hallway, the hallway leads to everything. The ultimate sandbox game.
It’s main draw is also typically sharing YOUR creations with other people.
There will also be extensive structure and curation to these games. It will just be elements that you can play with as you see fit, much like Minecraft.
Here's why Minecraft works: It's an ecosystem full of carefully-designed areas and resources (biomes fine-tuned to generate specific types of landscape, enemies with focus-tested behaviours, items with particular abilities) which combine to form a sandbox where players can exercise ingenuity and build projects that satisfy their creative vision.
Where does generative AI fit into this? Should it be generating an infinite variety of dull and unoriginal items that don't do anything cool? Should it be building derivative-looking houses for players to ignore or burn down so that they can build their own houses in their place?
LLM-generated slop has no place in games. It is an obstacle to having a good time.
LLMs won't generate items or houses (or at least be the primary source). They'll give the world agency, in a way that's straight up impossible now.
Like, maybe you screw someone over and they dedicate their lives to destroying you. Maybe you're just running around doing your thing and discover that the world has become embroiled in war all on their own. You can choose to join and fight, or play both sides, or just ignore it and keep adventuring. Maybe you're tired of adventuring and open up a flower shop, working your way up to creating a ruthless floral version of Standard Oil. Maybe you decide to kill the king and take his place, playing a game of cut-throat politics where people are constantly trying to backstab you. Then you decide, "Hey, I want to play as this other character and backstab my old self" and the LLM takes over your old character as you gleefully kill them off.
If none of that sounds interesting to you, then there's lots of other games to play and I'm sure you'll have fun. Those are all things that I can't wait to see happen.
People absolutely were complaining about this, with basically the same argument ("It's too much freedom! Players will get lost! They'll never get to the "goal" the artist sets!" etc)
Of course, that turned out to be silly
Before Minecraft came out, I played Nethack. No, I was never principally opposed to procgen games. However, there is such a thing as procedurally-generated slop. Good procgen games use a very fine-tuned algorithm to create engaging environments. Minecraft caves aren't fun to explore by accident; the algorithm was designed and tuned to consistently produce novel and fun layouts. Items and mobs were designed to specifically interact with those environments to lead to fun gameplay.
You're missing the insight that games featuring procedural generation are still designed. Bad design will lead to a bad game where the algorithm produces unfun gameplay. LLMs are not independently capable of good design. Games where LLMs are given agency over the player's experience won't be fun.
> maybe you screw someone over and they dedicate their lives to destroying you
And maybe it's really annoying and unpleasant, because the game is doing a bad job of telling that story. You're assuming it'll be good, but LLMs aren't smart enough to tell stories well or to understand what kind of gameplay will be enjoyable.
Best example of this is Spelunky, and this book goes into good detail: https://bossfightbooks.com/products/spelunky-by-derek-yu
Gist is that Derek designed these playable mini rooms which were ultimately randomly stitched together.
Anything interesting will have to be provided by the player (as in a sandbox game) or by the designer (as in a conventional game), and at that point you've just ruined a perfectly good game by introducing an LLM to slather boring slop all over the interesting stuff the player or the designer has tried to do.
> ai keeps breaking your head canon.
We must be fundamentally different types of people. I can't imagine even having this thought. If I'm playing a game that has characters, I'm not going to make up some facts about them and be upset when I talk to them and those made up facts turn out to be wrong. This is the only interpretation of your words that I can think of. Feel free to confirm it or deny.
I don’t know what you’re hoping to talk to these dwarves about.
I’m not saying I would be “upset” if the head canon wasn’t actualized. I’m saying I would be underwhelmed because the generic LLM content is nearly guaranteed by this other thing that I stated as a premise I found interesting or amusing.
The LLM's role will very much a "yes, and" improv role that will let you guide your own personal story. Or have it push back on you if you want. "What? You thought we were friends? No, I was just doing my duties".
What I want to talk about with the dwarves is everything, and LLMs will deliver in spades. I want to ask the lowly NPC what life is like working hard in the mines. Then I want to decide to recruit him on my quest because I like his backstory and personality. Then I'll watch as he levels up with me, in a way that could never be scripted by the game creators. That's just one small way that LLMs will provide a far superior gameplay experience.
That’s the opposite of what I’m saying. Asking the LLM to justify something is possible but, imo, very boring.
What I’m complaining about is asking about their friend and they don’t believe they have a friend. They just fail to capture the apparent narrative that you’ve enjoyed because it’s not really close to whatever game logic they’re scripted to utilize
I'm not sure why you think they'd say they have no friends. You'd say "Tell me about your friend", and the NPC would respond with something like "I have many friends, which one do you mean?". Then you could say "The one I saw you walking into town with yesterday", and the NPC might say, "Oh Rolf, him and I go way back!".
Maybe he says that because the LLM has instructions that him and Rolf are friends explicitly, put there by the game devs. Maybe it's not there explicitly, but the LLM sees what you mean and writes the backstory dynamically. Maybe a little bit of both.
LLMs are great at picking up narratives like that. It's kind of their whole thing.
If the dwarf bot does not know that the character he represents spent the last 24 hours in combat alongside another specific dwarf without you communicating this in text then it might as well not be attached to any character. Liked what would the point be??? That’s just an out of game bot.
Your example conversation of leading the bot to say specific things is… awful in my opinion. Who would want to talk to an NPC like this?
I don't understand your beef. My awkward dialogue was just to demonstrate that the LLM can pick up on all of these things you claim it can't. The LLM wouldn't "believe it didn't have a friend", why would you think that?
It's also fine to just enjoy on-rails games with a prewritten story, those will still exist. You might even enjoy the epic stories that people create in these new sandboxes and share with the world.
Your crappy awkward dialogue is probably a very realistic example of a stupid bot that doesn’t even know what actor it represents in game.
You’re asking why if would think that? You tell me how it would come to the conclusion correctly by using the game logic. And don’t give an answer that requires a hard coded dev interface between specific game elements and the LLM because that is on its own a more complex task than the LLM itself. It’s incredibly difficult. Shitty home assistants that can’t actually do anything being the prime example. The language processing is not the hard part. It’s the mapping of language to data and capabilities that’s extremely hard.
Synthesizing an agent that can form an identity based on its experiences and leverage game mechanics to pursue goals and convey all of this logic in flexible natural language is basically asking for a general intelligence. It’s an incredibly different thing from the shitty dialogue example you wrote
I also think that you're intentionally missing the point so that you can be insulting.
May you find games to play you enjoy.
Players don't talk to NPCs simply because they want to see text on the screen. They're hoping for well-written and interesting dialogue. If the devs couldn't be bothered to write lines for this guy, why would I want to talk to him? He clearly has nothing to say worth listening to.
Have you actually tried asking ChatGPT to tell you a story? It's boring. It's not a good writer. I don't know why you think players would want to spend time reading that stuff.
I think you just don't get it, and maybe this genre wouldn't be for you, which is fine. It's not about the devs bothering to write lines, it's about the devs enabling you to create your own world. I don't want some shitty NPCs slapped in there by a dev trying to hit a deadline. I want to experience my own story, freed from the shackles of the on-rails experience that game devs must currently provide.
> Have you actually tried asking ChatGPT to tell you a story?
What you're not getting is that the storytelling will be cooperative. You'll still be able to consume content to your heart's content. Some of that content will soon be people creating compelling stories in cooperation with LLMs. LLMs will also be great at storytelling too, but the best stories are ones that you can help write yourself.
Then why not write your own story? An LLM can't write your story for you. They're very bad writers. It can't even help much—its contributions are going to be dull and uninspired.
> the best stories are ones that you can help write yourself
If you're trying to co-create a story with someone in a play environment, the best analogy would be D&D or another role-playing game. And playing D&D with ChatGPT would be boring and unfun. It's just not a good storyteller. It's going to give you shit material. You'd be better-off cutting it out of the equation and just writing a story alone.
> LLMs will also be great at storytelling too
Will they? They're godawful at it right now, and they're not showing much sign of improving. Let's discuss the tech as it currently exists, rather than some speculative future version of it with imaginary capabilities.
> What you're not getting is that the storytelling will be cooperative.
No, I get that. But why would I want to cooperatively tell a story with a bad storyteller? I'd rather tell a story alone.
Games don’t need to be near movies but they should have an either a cohesive plot or no plot. A shitty made up on the fly plot is just not going to work
The truth is that most people are used to games as these formulaic simulations with rigid rules. That's partly making a virtue out of necessity (very hard to play a game without rules; and some of us just like rules), and partly habit. Maybe LLMs can act as something new in that sense.
Bang bang.
NPCs will be limited by the game requiring the LLM to generate an action ("Move 5 units forward"), and then the game checks that it's a valid action, and rejects invalid ones.
If you don't like metagaming LLMs, don't do that, and enjoy a regular roleplaying experience
NPCs won't have irrelevant backstories. The backstories generated on the fly for them by an LLM will be incorporated into the overall story.
The overall story climax will happen based on player actions, indicating they're ready for the climax, based on the world created by the player's actions and specific instructions.
I think you're trying to cram the square peg of limitless, custom narration into the round hole of a premade story. You can experience a decent roleplaying session already with an LLM you can run on your computer. Compute is only going to get cheaper, and LLM capabilities will only increase.
Which means it’ll be a shit story because the world will not be robust enough to capture the infinite edge cases you’re pursuing.
Current LLM DMing is rancid garbage. I’m assuming it gets several iterations better than the current state for us to even begin having this conversation
> I’m assuming it gets several iterations better than the current state
Yeah, that's the thing. LLMs are only going to improve at this. If you hate all LLM roleplaying everywhere, just wait a few years. Other people will have their own fun with them in the meantime
But, if recent attitudes to AI are anything to go by, this will be immediately seen as boring and derivative at best, or at worst people will accuse the back stories as having been "stolen" from other works of fiction.
While my expectations for the capabilities of AI have never been higher, my expectations for how the public will perceive this amazing technology has never been lower. It's all very disappointing.
And that's without specifically placing blame by the way, perhaps it's big techs fault for going about this the wrong way, I don't know, but the public blowback against AI is just making working in this field that I used to love super depressing.
- Amazing art is rare. Finding this amazing art nowadays is hard, with AI generated mumbojumbo increasing the noise it will be harder. It will be easier to just skip all AI generated stuff to find the real good stuff.
- Art has human touch or a message almost always in it. Purely AI generated stuff lacks meaning, because there's no creator.
- Socially sharing experiences that are unique only to you will be hard. I like to talk with my friends about the newest Nolan film or about Elden Ring's best bosses. If the whole game experience would be unique to me I wouldn't be able to relate with my friends at all and it would be the same as playing a different game.
- Valuing human work. I idolize people who have the skills to make great art. AI takes away from it.
But to these points, I can also see the future where
- AI could generate amazing art a lot, and would make me reconsider my categorical exclusion of AI art.
- AI could make it seem there's meaning behind the art, or an artist would use the AI as a tool to convey his meaning. - Unique game experiences would be so wild that you would be sharing them and relating via that way.
- AI becomes so common that valuing human skills isn't important anymore
Last point, I think novelty is something I value a lot and that's why in my opinion amazing art is rare. If AI would start creating a lot of novel art, novelty and rarity would just become common, and common is boring.
I'm not sure why this is shoved in as an aside near the end of the comment. This is how it's used. This is how it'll be used to create games.
Does anyone think that, what, ChatGPT is going to poop out an entire finished game and we'll release that? Or would artists use AI as just another tool to create their art?
Well if this trajectory continues upwards, it almost certainly will happen in the next decade.
Right now feels like how the 70's-80's felt with the rise of the microprocessor. Tons of promise and implementation ideas, but the tech just wasn't there yet. But sure as hell it came around.
No one who is good at what they do wants to do this. Everyone who is bad at what they do is doing this. I don't know why tech people think they understand the art industry at all.
Writing[2], Print, Photography, Movies, Electronic music, Synths, Sampling, Trackers, Digital Photography, Photoshop, Ray Tracing, Interactive Art (eg computer games) etc etc ...
Every time "it's not really art" , "there's no human touch".
Every time, after a while artists learned to use it.
And they made amazing art!
[1] https://www.youtube.com/watch?v=yzLT6_TQmq8 Propellerheads feat: Miss Shirley Bassey - History Repeating
[2] Surely I'm kidding about the writing, right? https://fs.blog/an-old-argument-against-writing/
The current expectation is that these models allow the creation of equivalent artwork to what artists are currently producing, but at larger scale / lower cost / shorter training. All of these boil down to less effort. And perceived effort is a large part of what brings value to art.
The player can interact freely with the NPC rather than through predetermined conversation decision trees, potentially driving a lot of unique emergent gameplay.
Downsides are that it is a lot quicker just to skip through conversations with the decision tree format and having to think about and type in questions could get tiresome. Another downside is players jailbreaking the LLM...
They could be offloaded to the cloud instead but that brings its own set of issues (ongoing upkeep costs, finite capacity, servers will inevitably shut down...)
For now. LLMs are still relatively new. The technology will continue to be optimized, and we'll be shipping tiny LLMs along with games soon enough.
(Aren't companies already running LLMs on phones?)
It might not even be necessary though, similar to how physixs (sp?) cards died out quick in the early 00's, the models might become slim enough and the GPU's fast enough that the extra load is easily manageable.