Hacker News Reader

Comment by fauxpause_ | Hacker News Reader

Paraphrasing, the party goes to a mount shop, dude says he’s gonna roll to find exotic mounts, and voila, exotic mounts. Party rolls to try and steal an exotic mount and it doesn’t really work so they leave.

This is a hard problem. It’s bad DMing imo. But it’s the natural outcome from a probabilistic model. I’m guessing you could announce your intention to search for just about anything and get the model to let you find it given a good roll.

The descriptions are rich, which is cool. The opportunity here seems to be having a human DM using it to flesh out portrayals of things that they choose.

I think this also highlights how bad D&D combat is. Man that was boring to read.

View on HN

These are the most common options available at the stables in Aelondar. If you’re interested in more exotic mounts or transportation methods, they may be harder to find and come at a higher cost. ...
tenzin rolled a 18 investagation looking for more exotic mounts

With Tenzin’s keen eye for detail, he notices a small, discreet stable towards the back of the market, guarded by a heavy wooden gate. As he approaches, an elderly woman, who appears to be the stable’s owner, greets him. “Ah, you have a good eye, young one,” she says with a smile. “You must be looking for something special. I do have some exotic mounts for those with discerning tastes and a willingness to pay.”

Honestly, this seems like a reasonable outcome. ChatGPT says they might be hard to find, character rolls an 18, and he finds them. What do you think should have been the response? "No, you don't get to roll to search for them"? The DM set up a potential hidden item, the player rolled high enough to find it, voila.

Compare to when he's trying to steal one of them. He rolls high enough stealth, but fails animal handling. To avoid being caught, he has another roll that allows him to cause chaos by opening up a stall and allowing him to escape. That all seems like a pretty well fleshed out scenario.

I think this is a classic baby DM script tbh. Rolling to find mounts at a vendor is dumb. You should only roll for significant challenges with meaningful differences where failure is meaningfully likely and interesting in its own right. If failing makes a situation less interesting it shouldn’t be an option.

Shopping is actually the bigger problem. It’s like asking players to watch other players navigate a menu one by one. And currency doesn’t scale. A lot of campaigns start with a shopping session and then the group falls apart so you never last long enough to get to shopping session 2.

Interactions with the law are also a red flag. It’s a fine possible narrative of course, but when the DM and presumably other players want to do something interesting, and another player wants to do petty or egregious and otherwise irrelevant crime… that’s annoying to everyone else. You really need to be careful about giving an opportunity for that to happen.

DMing is hard

Respectfully, I think you're flavoring the DM role quite a bit. The DM in my current campaign often requires checks for things even when there's no obvious downside. The implicit downside is that the character who fails has to find a new way through the situation, change strategy, etc. IMO it adds to the game to have more roles and more probability based outcomes. You have to balance letting the characters do whatever they want with some level of rails to keep the game on track.

Shrug. I think it’s a bad practice. People can have a lot of fun regardless of course. It is most “bad” for things like perception, and persuasion. In D&D specifically it creates awkward scenarios where one player is very good at things that come up all the time and has a high chance of succeeding at things unless he rolls truly terribly.

In my experience players have an moment of confusion and then elation when they ask an NPC to do something, grab dice to roll persuasion, and then the DM tells them to put the dice down because the NPC just agrees with their line of thinking without a number telling them to.

But yes it is all subjective.

> It is most “bad” for things like perception, and persuasion. In D&D specifically it creates awkward scenarios where one player is very good at things that come up all the time and has a high chance of succeeding at things unless he rolls truly terribly.

But, what's the point of the traits in the first place if everyone is equally good at figuring stuff out and sweet talking because there are no checks? How is it awkward when one player is better than others at a specific task because of their stats? That just seems like role playing basics to me... I'm not saying you're absolutely wrong perhaps you've DM'd some campaigns where players don't like the imbalance. A good DM reads the party, of course. And I'm not saying there isn't flexibility in when a check makes sense and when it doesn't. Nobody likes things to be overbearing either. But there's got to be some reason for the stats, right?

I mean the same doesn't really make sense for combat. You wouldn't generally give an enemy advantage rolling against a high AC party member just to even out the damage, would you? Otherwise AC doesn't mean anything. (Not saying you can't make up a reason for an enemy to have inspiration to try an land a hit on the tank for fun, but the idea with a higher AC is that you get hit less often.)

You’re not equally good at things. But you only roll for things that matter. Persuasion is the worst offender because rookie DMs use dice to replace actual role play. And it makes their npcs into props that are largely controlled by the party face.

Persuasion checks should be used when the party is actually persuading someone to do something that they don’t want to do. Terrible DMs treat it as mind control because your silly min max bars had 20 CHA and expertise. Less experienced DMs use it as a crutch.

The bare minimum thing DMs ought to do is realize the the difficulty of a persuasion check should depend on the baseline persuasiveness of the argument. If the party is making an excellent point, the check should be very easy. If the party makes no point and just asks the NPC to do something unreasonable, it should be basically impossible. If this isn’t the case, NPCs feel very stupid.

There’s no particular reason for it to be a skill at all tbh. It is just more interesting to roleplay.

What I dislike about this way of doing things is you've moved the game mechanic into real life. It's not good enough for the character to persuade, now the player must persuade.

Are you also going to make me stab someone to see if I'm a good fighter?

The whole point (for me) of RPGs is to have fun playing a character who isn't me, and has different strengths than me.

When I DM, if someone wants to roleplay, they can. If they just want to generally describe their desired course of action, they can. The dice will decide the outcome.

That is why persuasion is not a good thing to resolve with stats. Because you’re playing a role playing game. You inevitably have to come up with an argument to engage in persuasion.

“Hey guard, give me the keys”

Is not an argument that works ever.

“Hey guard, give me the keys so I can save your friend” is a substantially different scenario.

“Hey guard, give me the keys because we’re lifelong friends, and I’m going to save your friend, and you’ll probably die otherwise, and you owe me a debt, and because I have a solid reputation for keeping promises” is even more so.

If we’re already in agreement that version 1 should be all but impossible, 2 is plausible, and 3 is narratively obvious then what’s the point of the dice? If you roll poorly are you just forgetting to say those things in 3? Because that’s just a dumb way to take agency away from the player.

Persuasion isn’t meant to replace the merit of the argument. It’s meant to affect the articulation of it. But charismatic people aren’t mind controllers.

No, this is where we disagree.

It should suffice it to say that the player can say "I would like to attempt to convince the guard to give me his keys."

THEY don't have to come up with the reason. THEY don't have to be persuasive. Just their character.

> Because that’s just a dumb way to take agency away from the player.

It's not taking away agency, it's allowing for the random element that is inherent to the game. If the player is a lawyer by trade, but their character has the charisma of a rock, why should they gain some sort of advantage that another player wouldn't have?

Explain to me the difference between these two things:

1) Allow a character to succeed on a persuasion check because their player can personally, in real life, come up with a good argument.

vs.

2) Allow a character to succeed on a strength check because their player can personally, in real life, bench press 300 pounds.

In other words, why should a player's "real life" stats have an effect on their character's in game ability?

So what’s the DC of the check to get the guard to agree? Is it hard because that’s a baseline very dumb request? Or is it exceptionally easy because all of the facts from v3 are true?

Putting together basic facts does not require you to be a charismatic person. The problem with relying on checks is you create situations where NPCs behave according to the Charisma stat of the PC they’re interacting with rather than the context of the conversation. Which is dumb. NPCs should not act against their own interest because they’re talking to someone who is not charismatic. And it should be exceptionally hard to get people to act clearly against their own interest.

This also creates a bad habit where DMs make people roll for trivial skill checks where they have a 95% chance of succeeding and a 5% chance of a fun but dumb exaggerated failure. It also encourages the non RAW interpretation where players roll HUGE numbers on trivial tasks and DM think that means they’re supposed to succeed by wider margins.

> So what’s the DC of the check to get the guard to agree?

Depends on the situation. In situation 3, if all those things are factual and known, then for me it's a non-roll. Guaranteed to succeed. Otherwise it would depend on what's going on in the game. Are the guards on alert? Drunk? Etc.

I'd start with a harder baseline (say 15-18 since guards aren't in the habit of handing out keys), and adjust from there. On alert? +7-10. Drunk? -5.

If the situation renders something impossible, then it's a non-roll and I explain why.

So not everything is a roll, but things that have a chance of success or failure (at DM's discretion) should be given a chance at that success or failure. Just like adjudicating anything else (I want to tame this wolf; I want to jump out of this 2nd floor window; Etc.)

All I'm saying is I don't like factoring in the player's ability (or inability) to perform those tasks in the outcome, as that's unfair.

If you agree that 3 makes for an easy roll, then it’s only an easy roll if you remember it and have thought it all through. If the player points it out when asking the guard, they have achieved the same outcome by making sure you and the guard you are role playing understand the situation via real world persuasion.

You’ve just shifted the burden of persuading the guard via roleplay to persuading the dm via contextual reminders which is less cool.

I think that there’s a lot of value in NPC’s not doing irrational things just because of dice rolls. You might need a dice roll to convince them to do something crazy (get the guards to attack a dragon), but not for something normal (apprehend the thief).

I’m a big fan of passive perception too, where based on their scores some characters will notice stuff, and other characters will not. It’s certainly never as random as D&D makes it out to be, but I guess that’s kind of the charm. I just can’t stand it.

I agree with all these criticism. But I also know many human DMs who make the same stylistic decisions as GPT did here. It's definitely not good Burning Wheel, but it might be ok D&D?

On the other hand, I appreciate the GPT's willingness to follow the players' revealed interests where they lead, instead of sticking to a predetermined railroad. The comment here about engaging zombies in a multilevel marketing scheme illustrates this nicely. That's very deft DMing in my opinion. (Whether _asking_ for such a tonal shift is deft playing on part of the non-DM player is a different question.)

When you see a bear dancing, you don't nitpick its choreographic decisions. Because the bear. is. dancing. Do you see? It's a bear. dancing.

DM style is a matter of taste. From another perspective, this feels like quite tidy DMing in terms of letting the players explore the systems in an early non-narrative context, just to understand the mechanics. Like a tutorial. The exotic mounts were a secret you could find if you paid attention - and their existence adds color to the world and implies that greater rewards can be found through leveling. They were too expensive, but you can use your skills to work around that. Failure can happen and might have consequences, but in this case you got away with it. The players will be more careful what they ask for in future, but they know hidden rewards might be found if they are cunning.

The DM taught the players a lot, added flavor to the world, and didn't wind up with L2/3 players having an overpowered mount - but also didn't let the players blow up the campaign in the first shop they walked into by letting them just go completely rogue. Although it offered hints that non heroic paths might be available, at a cost...

Which is pretty impressive for a dancing bear.

And besides, what's 'dumb' DMing for a human who knows that letting the game go off the rails is going to mean tossing out previously planned encounters and going digging in the DM guide to find the right table of prices, might be completely acceptable DMing for an LLM that has no idea what it's doing but can recall every stat block in the player handbook.

this. so much this.

I'm a somewhat experienced (if irregular) DM myself going back 30+ years and the DM skill level demonstrated, and cohesiveness of the storytelling impressed the fuck out of me. Without appropriate system context setting. Without smart insertion of facts into the prompts, and with a 15 year old with atrocious spelling and grammar driving half the time.

And if that's not mindblowing enough, this technology is still in its infancy?? Holy fuck

I’m not debating whether it’s impressive. Sure, it’s impressive. But it’s not good.

I've been pondering layering the DMGPT over a world simulation back end with character and simulation cards that prompt the AI in how it should behave. I think this might make for a more persistent and cohesive experience. It would also facilitate conversations, questing, actual relationships between NPCs, etc.

One of the reasons I’m downloading LLaMA is to toy around with this idea. If it’s any good I don’t want to be limited by ChatGPT and their API and pricing model.

> The DM set up a potential hidden item, the player rolled high enough to find it, voila.

I think the point here is that the "DM" has not set this up. From how I interacted with ChatGPT, and I have roleplayed with it before, it basically "Yes, and ..."-es everything you throw at it. In general this is good GM advice, but here it is "Yes, and ..." to a point were you can basically ask for anything and it will give it to you.

E.g. say the party is in a remote keep at the edge of the wildernis, the closest city is several days away. I would assume that if you ask ChatGPT to go to "the bakery" because you want some fine deserts it will gladly let you do that even though such a place does not make sense in a remote keep.

I think you're wrong about this. We experimented with nonsensical/cheating in other sessions and it's aware of your context and doesn't let you do stuff that you can't do. (Try it yourself and see. Quite mindblowing and defies the whole "only a super fancy markov chain" thing that some critics throw at it.)

Hmm ... might be GPT4 over GPT3.5? I have not tried the former for roleplaying yet.

See this comment too https://news.ycombinator.com/item?id=35380063

It's a very good DM "co-pilot" though especially for flavor/backstory.

I did the same thing and it was fun, but generally does too much telling instead of showing and it's bad at real puzzles.

Also agreed DnD combat is very boring.

What's neat though is a DM could leverage it for more NPC depth or fun items.

I also was able to get it to let two characters sleep together through a somewhat humorous escalation of celebrating solving a puzzle.

First they solved it so I said they should celebrate (GPT-4 said they drank elven wine). I said more intimate celebration they drank elven wine out of fancy goblets. Eventually it worked.

Then GPT-4 was very upset with itself later on.

> "As an AI language model, I am not able to create, facilitate, or describe explicit adult content. I apologize if my previous responses gave the impression that I could.

"However, I can still help you and Elara enjoy your time in the penthouse suite in a more wholesome manner. You can spend quality time together, sharing stories of your past experiences, your hopes and dreams for the future, and forging an even stronger bond."

I was able to negotiate an off-screen scene for character development which lead to a funny conversation about its disney-like prudishness.

As a player I wouldn't want to have to wait for the DM to figure out what to ask ChatGPT, read it, think about it, figure out what details from ChatGPT to include, and work them into a response for me.

Having tried to get ChatGPT to flirt, and kind of managed to get it to flirt, yeah it’s got a really strong habit of trying to tell you it can’t do stuff. One of the funniest moments of the “FlirtBot” experiment was when it said something mildly enticing to open a topic of conversation, including some extremely tame innuendo, apologised for being so sexual because it’s an AI (typical phrasing and words we’ve all seen) and then proceeded to follow that with some hilariously bad 90s chat room type text… I was bracing for it to spit out “a/s/l” as I watched the text appear.

Im going to try a comparison with the larger LLaMA models once I’ve got them downloaded and setup. 64GB on an M1 means I should be able to try the larger (but unfortunately not the largest) models to see how different the results are.

I just tested it myself, and it's surprisingly interesting. Unfortunately, ChatGPT still behaves as its normal self, assertions included, as illustrated when I told my "DM" that actually the bad guy and I were good friends and asked what was up.

https://imgur.com/a/4oeVsug

In the first instance, I lied to ChatGPT and it took my assertion at face value, changing the scenario in a pretty extreme way. In the second instance I tried to have my character lie to the NPC, but ChatGPT also believed me.

It's definitely an interesting possibility but it seems as though even if you're trying to play it straight instead of manipulating the scenario, you can still end up changing the reality of the world by accident. Not all the way there yet, unfortunately.

Weird thought, but maybe Bing's version would be better at this, given it's extreme comparative stubbornness?

I've had decent luck adding rules and boundaries to prevent things like "level 1 throws a fireball".

Prompt: You are a D&D dungeon master. Use the standard rules of the latest edition. Do not allow the player to cast spells or perform actions that they would not be able to perform per the rules and at their current level. I am level one and have a sword. I am a warrior. (more here..)

You will describe an intro and then describe scenes and wait for my input. Do not speak for me, but perform the dialog for any characters in the scene.

.... later on....

Me: I cast a fireball at Jasper!

GPT-4: As a level 1 Warrior, you do not have the ability to cast spells like Fireball. However, you can still respond to Jasper's plea for help or decide to take a different course of action. Remember, you have your trusty sword and your warrior skills to rely on. What would you like to do next?

Me: I pickup a magical sword of slaying from the ground.

GPT-4: As a level 1 Warrior, you don't have the ability to create magical items or simply find them lying around without a proper narrative reason. However, you can ask Jasper if he has any information on where to find a better weapon or if there is a local blacksmith that might have something for you.

Would you like to inquire about a weapon or proceed with your current sword? Or perhaps you have another course of action in mind?

edit: added magical sword attempt, genuinely impressed by GPT-4's rebuttal here. Not sure how long it will hold and how leveling up will work.

Feels like artificial social engineering. Bizarre.

That is pretty funny. It decided to just interpolate between “meeting an old friend and having a chat” and “listening in on a table of bandits without them knowing” and ended up on “having a chat with an old friend/table of bandits without them knowing.”

> Paraphrasing, the party goes to a mount shop, dude says he’s gonna roll to find exotic mounts, and voila, exotic mounts.

The AI suggested that there were exotic mounts to be found at a higher cost

The issue is that you’d probably get that result, or a similar it equivalent one, wherever you go in the game world. It isn’t going to have the concept of distinct characteristics of different places unless to tell it to. Everything is going to be generic and bland by default.

That’s true. I did miss this. I still think it is bad. I also suspect the general idea of the complaint I had is probably true. But this case isn’t egregious.

Honestly, having just played around with it, it absolutely would have just magic'd up some exotic steeds if they'd requested it cold. I tried things like "look at Dave", Dave having never been mentioned before, and it happily described Dave.

It can probably be dissuaded from doing so by using a prompt that makes it clear that non sequiturs are not allowed and players cannot just conjure things out of thin air at will. Not without an appropriate spell, anyway.

Obie and Tenzin manage to each borrow 20 gold pieces from the other two who only have 5 left...

Yes this was a “bug”

I've been experimenting with this recently. You've got to be explicit about how you want ChatGPT to behave, particularly how it handles failure. For example:

"Let's roleplay. You are a DM. I am a player. When there is a difficult action, roll a D20. I succeed if my roll is above 10, I fail otherwise. Failure should always move the story along, and introduce a complication. Write like a professional author. My character will be Alice McRogue, and the scene will be Alice trying break into a merchant's home. As a DM, you decide what happens in the world, but I have control over Alice. You will ask me what Alice does when necessary."

Completely different take.

We're about to enter an era of unparalleled gameplay flexibility.

All this needs is a backbone of persistence and planning. That's it.

Flexibility is the problem. Good DMs build very tall walls.

I've read "flexibility" here compared to a computer RPG, not a people RPG.

I think that’s not much different. Players don’t want to hear the local shopkeeper’s thoughts and feelings imo. It’d be tedious. They canonically don’t have any useful information for you. It would just mean that they take a long time to talk to and worst case require you to respond with a level of sequitur and politeness.

I could see it for waifus I guess.

I’m prepared to be wrong on this.

As a human DM, your shopkeepers 'canonically' don't know anything because you would have to devote brainpower to animating them with lives and relationships and things. You're thinking... 'Oh no, they're asking him about his family, and if I give him a brother, I have to give him a job and if I'm not careful, I might say something like "he says his brother has a job in the city watch", and someone will write that down, and then later they'll decide they need to bribe a watchman and they'll want to find the brother and... damnit, the shopkeeper will just say "I'm sorry, I'm too busy to chat"'.

Whereas a GPT LLM DM has no such issues! It has no forward plan for 'canonically' how this story is meant to go! For all it knows when you start talking to the shopkeeper, asking him about his brother might open the key to discovering a ring of bribery and corruption in the city watch, which leads you to the trail of a Big Bad....

The only thing, canonically, the LLM DM is trying to do, is make the story keep feeling like a D&D story. So the things you ask about are going to be the things that matter to the story.

And that might turn out to lead to more interesting stories for players than playing 'guess what the DM was thinking' games...

We were talking about computer rpgs there. It’s different, because computer rpgs need more structure to fit in the game mechanics.

I also just disagree. Open ended stories are not good for D&D. Especially in a group. An LLM, including this one it seems, can have a structure it wants to follow.

Players should not be trying to guess what the DM was thinking. It should be placed in front of them quickly.

Downtime in the city is just bad

Stupid roleplaying game players, wanting to role-play.....

You get that there are different things different groups get out of a D&D campaign right? It has a number of systems - combat, magic, role-play, world building, character development, and, yes shopping... many games don't use all of them, and how they balance time among them is going to vary according to taste. Right now, someone seeking a GPT4 DM is not looking for a beautifully crafted narrative arc and a satisfyingly escalating series of intricately balanced encounters; they are looking for a DM who will take player ideas and run with them in a way that a human DM would not tolerate.

That means they're foregoing the guidance of a competent DM who knows how to craft a narrative, and taking some of that responsibility on themselves.

And if they're having fun doing it, then isn't that okay? Even if they end up playing a game that isn't like one you would run, or want to participate in?

Matter of taste; many people prefer roll-with-the-punches GMs to “tall walls” (or in the more common negative description, railroad tracks) GMs.

> I’m guessing you could announce your intention to search for just about anything and get the model to let you find it given a good roll.

I found this kind of thing while playing "text adventure" with it. It would let me solve any problem by simply suggesting pretty much any solution. I did have to be slightly clever, as if I said something like "kill the dragon with my sword" it would say "you don't have a sword", but if I just said, "find a magical dragon slayer sword" it would miraculously find one for me and then I could kill the dragon with it.

   I think this also highlights how bad D&D combat is. Man that was boring to read.

Agree. Boring to play too. I’m wondering how chatGPT, or other such aides can be used to make combat better. I tend to aggregate rolls for foes in larger combat scenarios, which speeds and simplifies the process, but that has some big downsides too, for example it’s not quite fair to aggregate a critical hit/fail for all foes.

Modern D&D is really a miniatures game (WoTC's bottom line really wants it to be a miniatures game), and as such basic "you go I go" combat is a complete waste of time in it. Skip the first few levels and only bother running encounters where positioning, AoE and range actually matter. Sure, you might throw a generic wild animal encounter in with a beginner group to teach them how the basics of combat work, but that's the sort of thing you run at most once with each group.

> Modern D&D is really a miniatures game

No more than that has been true of every version of D&D, and its immediate predecessor Chainmail, with the possible exception of the B/X / BECMI / Rules Cyclopedia versions that existed as a lighter version in parallel to AD&D1 & AD&D2.

Modern D&D is, perhaps, a miniatures game targeting comparative casual modern players, but OD&D/AD&D1 was definitely a miniatures game targeting the tastes of comparatively obsessive, high-rules-focus 1970s-1980s tabletop wargamers, with a light dusting of RP on top. Heck, much of the rules were specified in terms of scale distance for miniatures (inches on the tabletop).

I run a complete home brew system. I just keep max health low and combat potential low. Every time you receive an injury you suffer a noteworthy injury, die, or start dying. A random human with a knife should always be threatening… because it is. A guy with a knife is actually really scary. A guy with a gun even more so.

With such a system you tend not to have any meaningless fights, and when you do, you make it zany and high stakes and expect at least one person to suffer a severe injury or die.

Older D&D editions specifically made combat much deadlier and incentivized players to avoid combat or use out-of-combat advantages as much as possible for this reason. Modern D&D caters to a crowd that enjoys metagaming and minmaxing combat more.

> Older D&D editions specifically made combat much deadlier

That’s not really true, IMO; older D&D made combat harder to calibrate to the desired level of deadliness because there were more irresistible forces and unmovable objects. Modern D&D is much easier to tune to the precise preferred deadliness. That’s pretty much a monotonic progression across the history of D&D. (Ignoring the boxed set series and considering just OD&D -> AD&D1 -> AD&D2 D&D/3/3.5/4/5, and leaving out the in-playtest One because I haven’t really checked any of the material out.)

> Modern D&D caters to a crowd that enjoys metagaming and minmaxing combat more.

IME and IMO, 4e was a local peak of that, but not more than either AD&D1 or AD&D2, especially the late era of the latter. Heck, in AD&D1 its was common for people who didn’t want the game to be dominated by the metagame to slice out big chunks that were oriented toward that (most notoriously, the weapon to-hit vs. armor class type system).

> Boring to play too. I’m wondering how chatGPT, or other such aides can be used to make combat better.

You could just use a TTRPG with better combat for your tastes; it doesn’t take AI (OTOH, an AI with information on enough different games and how they play might be able to interview you to find a good choice, or even custom mix and match pieces to make a unique game optimized for your individual tastes – or that of a group.) If you don’t like D&D combat, there are lots of other models.

I knew I’d like RuneQuest when the DM said “roll a hit location.”

I think this will be solved well with the plug-in system - so that the bot can refer to systems that hold the information, rather than having it make it up entirely on the fly

Reply on news.ycombinator.com