Making a text adventure game with GPT2 | Hacker News Reader

Making a text adventure game with GPT2

quicktotheratcave.tumblr.com

116 pointsrwnspace21 comments

There's definitely not enough training data being used here. In particular, the GPT2 output:

> YOU ARE IN THE PIT, SO TO SPEAK, BELOW THE RAMPARTS. THERE IS A NARROW PASSAGE FROM EAST TO WEST AND A STONE STAIRWAY LEADING DOWN. THE ROOM IS DEAFENINGLY LOUD WITH AN UNDETERMINED RUSHING SOUND. THE SOUND SEEMS TO REVERBERATE FROM ALL OF THE WALLS, MAKING IT DIFFICULT EVEN TO THINK.

is largely copied from the Loud Room in Zork 1:

> This is a large room with a ceiling which cannot be detected from the ground. There is a narrow passage from east to west and a stone stairway leading upward. The room is deafeningly loud with an undetermined rushing sound. The sound seems to reverberate from all of the walls, making it difficult even to think.

Similarly, the response to "kill the spirits":

> THE SPIRITS, SENSING A GREATER POWER, FLEE THROUGH THE WALLS

is a direct quote from a similar (and much more dramatic) scene in Zork 1:

> Each word of the prayer reverberates through the hall in a deafening confusion. As the last word fades, a voice, loud and commanding, speaks: "Begone, fiends!" A heart-stopping scream fills the cavern, and the spirits, sensing a greater power, flee through the walls.

And a lot of the more original output from GPT2 (like "ROAD NEAR BRIDGE" and "THE SHAPEK FAN") is simply incoherent. So this isn't really a very good showing, overall.

GPT2 is fascinating. I like the idea of achieving a kind of immortality by training GPT2 on the sum total of your life's data: all your comments, all your tweets, all your text messages, everything. Presto, now you have an unkillable version of yourself that can answer questions about your life. Your loved ones might like it when you're gone, at least.

Maybe in the future we'll have "harry potter photos" that use GPT2 + speech synthesis to pretend to be you.

Sans the magic of GPT2, somebody made a chatbot based on their father: https://www.wired.com/story/a-sons-race-to-give-his-dying-fa...

Another thought: you could use this to have an AI make a best attempt at answering a text/email for you and then ask for human acceptance/rejection/editing. I know some messaging apps have suggestions based on the last message sent to you, but I'm not aware of any that suggest responses based on your specific history of language use.

It's the verbal likeness of having a bust of them carved out though. From some literary angle there is certainly a similarity, but the AI in this case doesn't know that it functions or why. It's as if you've trained a baby to say all the things that someone used to say, and nothing more.

Then you may be interested in Rudy Rucker's lifebox concept.

I’ve thought about this! A product I’m working on would have all your info locally for something like this. But GPT2 is big. A lot to download, so I discounted doing it. I suppose it could download after the fact, however. At user request, like downloading a new voice for Siri. 2020...

Yeah, GPT-2 117M is only 500MB. Nowadays apps are larger than that.

"Goodbye for Now" by Laurie Frankel has a similar plot line where an engineer builds a "simulation" for a grandmother of his girlfriend based on her emails, texts, facebook messages, and video chats. I found the story quite interesting and engaging.

"What’s that, Dix?"

"This scam of yours, when it’s over, you erase this goddam thing."

-- Dixie Flatline construct, Neuromancer

There's a fictional podcast which has this as its premise: a company trains an AI on your loved one's social media corpus and then sells you the ability to continue to interact with them after they're gone. A fun listen:

https://www.stitcher.com/podcast/panoply/the-message

I lost a loved in the last year and I find that idea extremely odd, if not just plain wrong.

I'll listen to the podcast, but the concept of having something crawl through someones social media, namely someone you care, to emulate him/her (to become a "it") it's disturbing.

Sounds like the episode of Black Mirror "Be Right Back":

> The episode tells the story of Martha (Hayley Atwell), a young woman whose boyfriend Ash Starmer (Domhnall Gleeson) is killed in a car accident. As she mourns him, she discovers that technology now allows her to communicate with an artificial intelligence imitating Ash, and reluctantly decides to try it.

I don’t think gpt2 is useful in the way described, its just spitting out nonsense.

Text adventures are hard because you have structured data (eg. a scene with N objects in it), but you want to generate a text fragment dynamically as the state of those objects changes.

It meaningless to just randomly generate text fragments; you have to parse the input of free text (hard, eg. turn off the light; yes it is hard when its free text), update the state of the scene (easy, eg. set light level to low)...

... and then return a new text snippet that describes the scene (extremely hard, unless you constrain the state transitions to a finite hardcoded set).

so, like, tldr, you could do something really cool here, to use a language model to describe a scene over a continuous range of states using a finite set of samples and that would be amazing!

...but, I dont really see seeding a text fragment from a set of words being a particularly interesting addition to the field.

Here is a paper where they had users training a bot to follow request (given in natural language) of increasing complexity to accomplish task in a simulated environement :

https://research.fb.com/wp-content/uploads/2018/04/mastering...

Results apparently vary. I tried this and the result was, shall we say, less than interesting.

Slightly off-topic: are there text-adventure games that can be played over siri / alexa / etc.? it would be awesome while driving or walking somewhere.

Yes, there are some simple Cyoa on Alexa.

The Alexa port of Skyrim is essentially a text-based version of the game

Love this! My take on this same idea was to train GPT-2 on a bunch of D&D adventures and then pipe the output to speech synthesis. I ended up just publishing the output as a podcast. The app also creates its own static site. You can read about the AWS workflow I used to create it.

AIDM: Artificially Intelligent Dungeon Master http://iws.mx/aidm/

Some of it sounds surprisingly creepy:

  look around
  YOU CAN'T SEE ANYTHING
  IT'S TOO WIDE TO CROSS
  YOU CAN'T HEAR THE SONGBIRD NOW
  IT CAN'T BE FOLLOWED

Creepy indeed:

  climb down
  YOU CAN'T CLIMB THE BOTTOM OF THE CHASM
  ?THE BOTTOM OF THE CHASM IS THE ONLY EXIT
  THE ABOVE ROOM IS A GOOGLE+ GROUP OF YOU, ME, AND A BUNCH OF BUDDIES

Reply on news.ycombinator.com