Imitating Human Behaviour with Diffusion Models | Hacker News Reader

Imitating Human Behaviour with Diffusion Models

arxiv.org

107 pointsbelter33 comments

Open world RPGs are going to get really weird. I'm looking forward to the version of Grand Theft Auto where you can abandon the main quest chain, meet a nice quirky NPC with a really great sense of humor, and raise a family together. Maybe get involved in local government or do some volunteer work.

Sounds like modded skyrim

where the mods plug advanced (post-chatGPT) AIs into this.

it happened already, but it's still "military grade and security cleared" away from plebs but I am a crazy person so feel free to disregard what I say if it makes you queasy. (q easy, cue ez)

also, it's online.

I'll bite, what are you referring to?

This sounds very similar to the game that was described in Ender's Game, with the giant and the two shot glasses. That's always been one of my wishlist games, excited to think that it is starting to become a possibility.

Imagine characters as well written as those in Mass Effect 2, but who actually connect with you, who remember your adventures together and your conversations, who listen and respond more thoughtfully than many real world friends.

To the human brain, they will be real, as will the suffering from losing them, or the passion of falling in love with them.

Thankfully, resurrections will be an option for a fee, and virtual marriages will be sought but I think the mental health and legal issues will put the brakes on some of that.

Sex workers with VR headsets will allow for hook-ups with your AI lover.

> Who remember your adventures together

Hah.

(GPT and I have a long history of me pointing out that it can’t remember a darn thing, and it never remembering anything. https://news.ycombinator.com/item?id=23346972)

You can prompt it with context, but that’s still only a few pages, not chapters. It’ll be hard to turn in game actions into a coherent prompt, then parse the output to take in-game actions. Both directions are difficult, but I suspect it’ll be much harder to get it to do things in the game in a unique way — i.e. you’ll run into the classic problem of all NPC’s feeling “kinda same-y” without being able to put your finger on why they don’t feel like unique human characters.

On the other hand, it does seem like I’ll be proven wrong within a decade, and I really look forward to it. :)

I wonder how GPT would do with a high but clearly achievable goal — “as much context as is stored in a Dwarf Fortress Character.”

Will this mean better AI in computer games? "Better" here in the sense of "more realistic", not necessarily "higher scoring"? Because I'd love that.

Certainly is a possible outcome although there are a few problems with the current paper as I see it:

* Quite slow to execute (somewhat inherit to diffusion models)

* Requires a lot of human data which increases dev time, since it needs to be done near the end of gamedev to get a consistent enviroment

* The current paper doesn't consider previous observations (I can't find the reference so I could be wrong)

I believe issue 1 & 3 can be overcome quite easily with changes in model architecture, and 2 can probably be overcome with RLHF to pretrain on self-play and fine-tune on human input.

We're definitely like 5 years from being able to have really realistic NPCs in games. But then probably another 10 until they're actually widespread (RL AI have been able to be incorperated into games for years now but these things are surprisingly slow to be adapted).

If the game uses voiced npcs that might be trickier. People expect to hear a voice actor not Microsoft Sam.

Maybe, but most games use simpler tools like behaviour trees or utility systems not just over machine learning based AI, but also over fancier game AI techniques like GOAP, MCTS or even HTN’s simply because:

1) they are easier for the designer to understand and easy to understand why an NPC made the decision it did

2) they are easier for the designer to tweak and update to get the behaviour they want

3) they’re computationally expensive. Most games only give a few milliseconds at most per frame to AI. The model would have to run very quickly. GOAP and MCTS are known to work well but the search space can explode too large, eg the total war games use MCTS and can only look a few turns into the future because the search space is too large.

The problem with machine learning based game AI is that it is difficult to understand why the NPC is doing what it’s doing and if you need to tweak or update it, you have to retrain the model which could be costly or time consuming. It may also be difficult to train in the first place because you need to create example data that has the desired behaviour.

A big part of game AI is designers authoring an experience that they wish to convey on the player and machine learning takes a lot of control away and puts it in an opaque black box.

GOAP = goal oriented action planning, a graph-search based planner

MCTS = Monte Carlo Tree Search, a heuristic search based decision making algorithm. Some of the Total War games use it.

HTN = hierarchical task network, a hierarchical planning system that solves some of the issues GOAP has: it can prune the search space more aggressively and it gives designers more control over the resulting behaviours. It’s lesser known and has less talks/articles/sample code compared to GOAP

Interesting. It's honestly quite disappointing that progress has been so slow in this space, but in this time dependent context it makes sense.

I remember AI NPC behaviour being a discussed and promoted all the way back to the late 90's, but it's like there's been a standstill and characters still run around i zig zaggy pathway patterns, switch between pre-made states and get stuck and never really surprise you with anything really "outside the box".

So much that it's not even really a topic anymore as far as i am concerned? Maybe because of multiplayer.

Which is sad, because even multiplayer would be very interesting with Alien style NPCs who were actually clever and cooperated and had memories stretching over the timespan of the games existance. (Years.)

I think there's more to it than time constraints. Of course the efficiency of the method is a big concern, but we also have to remember that games are primarily experiences. In a lot of the games I play, stuff like stealth games, rougue-lites, and DotA, the simple and predictable behavior the current models is required for the game to work. The models HAVE to be stupid to make the game work.

Another thing worth noting is that "smart" game AI is as much about presentation as it is about actual intelligence. That is, if your NPC's are super smart, but doesn't communicate their smart decisions to the player, they will often still be perceived as dumb. That's why "barks" are such a big thing in games: NPC's tell each other what they saw or what they're doing to communicate to the player that they made a potentially smart decision. This communication may not be so simple with a machine learning based AI where the intent of the NPC isn't known.

Similarly, often super intelligent AI doesn't look very intelligent, doesn't feel very intelligent or is just not very fun to play against.

It also depends on the type of game. If the player spends a lot of time observing NPC's (eg in a stealth game), then intelligent AI is more important than if the expected lifetime of the NPC before the player slaughters them is only a few seconds (eg in a shooter).

I don't think its necessarily time constraints. Its also understanding the AI: why did the NPC make the decision it did? Exlpainable machine learning is still very much a research topic. Its about giving the designer the ability to control and author the interactions that the player will get. Its about controlling what situations are actually desired (there was a story about the AI in The Elder Scrolls: Oblivion's "Radient AI" system where they had to tone the NPC's autonomy down because they found that NPC's would tend to do unfun things like murder everyone in town). In terms of training machine learning models, for many games you may simply have no way of gathering sufficient quantities of data to train it to do the behaviours you want for the scenarios where you want them. Time may not necessarily solve these, or at least you'd need a decade to do everything.

Personally, I like sandbox RPG's, so I personally want to see better NPC AI and deeper NPC simulations. I want to see NPC's go about their lives independently of the players interactions and have their own agency in the world. So I'm very much interested in seeing better decision making, planning and simulation of characters in games. But machine learning based techniques aren't necessarily the answer, if you want the game to still be a game and be fun for the player.

In fact, in my own opinion, if NPC's have true autonomy and agency, then you likely also need a storyteller/director system that manages the NPC's so that they 1) don't go too far off script, 2) don't go too crazy in undesirable ways (like the Oblivion murder spree), and 3) that their interesting interactions tend to occur when the player might actually notice them (otherwise you may simply "miss" all the intelligent behaviour because it always happens when you're not around or in places you don't visit in time to notice the consequences)

I've been waiting for civilization franchize to do this.

by this point anything less than free form chat with the other 'leaders' won't cut it

Is this realistically useful outside of games?

The moment we need to express all this AI in the real world all we have is some clunky pieces of metal joined together with electric motors like a 50s Sci-Fi movie that costs multiple thousands of dollars.

Right now the "brain" looks like the Star-Trek age but the body is stuck in the 19th century.

Having said that, I would probably freak out if I saw some bot casually strolling down the street.

The "brain" is good enough for a Star-Trek sound stage but not enough for a Star-Trek real life. It's a big phrase book.

Simulating human behavior is useful in a ton of areas, think of economic and sociology research as a first step before real world studies

> Is this realistically useful outside of games?

Really, nowhere.

In most cases, human behavior very simple, just question of taste, nothing rational.

Rational behavior appear, when high stakes on board. For most humans, borderline , where become rational, somewhere between one and 10 his salaries.

Have not read this text, but seen hundreds before.

This is dead end. At least it consumes too much energy and computational resources to give very simple answers.

But really important, humans are just very simple in most cases, don't need to make things so smart.

https://arxiv.org/pdf/2301.10677.pdf

This is bad. This feels like something that could be used as a pumped up Cambridge analytics scheme.

It is for robotics, no relation to social media.

> On a complex control task, diffusion models achieved a task completion rate of 89%, exceeding recent state-of-the-art of 44%.

Romance scams are definitely going to benefit from this.

I have a serious if not possibly ignorant question here, but shouldn't experimenting on human behavior without consent be just as illegal as any other human experimentation and shouldn't that be extended to imitating art or language?

Certainly ignorant, since you mentioned it. Read the synopsis. Then, read the paper. Then evaluate based on merits, not some (frankly unrelated) prior agenda about copyright infringement.

The title doesn't even really imply what you're suggesting. Quite a reach. At risk of breaking the rules, RTFM.

Curious, based on the research referenced, where do you are non-consensual methods of research utilized?

Why should it be illegal?

'without consent'

Imitating someone is not experimenting on them.

Reply on news.ycombinator.com