Hacker News Reader

Comment by dzdt | Hacker News Reader

No human has ever beaten NetHack without being "spoiled". That is, in NetHack culture, reading how-to guides or the game's source code to understand what process to follow to identify unknown objects or to navigate tricky situations. It is unsurprising that with training rules that prohibit AI's from similarly being spoiled no AI has beaten the game.

View on HN

Actually this is false and probably have occurred more than what’s recorded. The guidebook is enough information (although if there’s an AI who can comprehend the nethack guidebook that’s far more impressive than being able to play nethack).

https://groups.google.com/g/rec.games.roguelike.nethack/c/wc...

A sufficiently large GPT model could probably comprehend the guidebook, in some sense, assuming they manage to increase the input size enough to ... I guess, let it condense the guidebook to some kind of internal language and then make use of that and now I'm just speculating wildly. ^^;

If I had the GPUs, it would be an interesting thing to experiment with.

However, having comprehended the guidebook, it then needs to also comprehend the nethack terminal client. The pile of obstacles does not cease to grow.

By "training rules" are you referring to the way machine learning systems are traiend? In that case, you're right.

The Challenge highlights an important limitation of modern machine learning systems, that they can only learn from their training data and have only a very, very limited ability to incorporate "background knowledge".

"Background knowledge" in this case is the "spoiling" you're refering to. There is virtually no way to teach a neural net system how to play Nethack, other than to let it try, and fail, and fail again, at ascending. You can't explain to it for example, how writing "ELBERETH" in the dirt with your finger will scare monsters away (which is important background knowledge about the game that is very hard to learn just by trial and error).

Nethack is full of such things, that would be meaningless in many other games, and AI developers would probably not look at in other games... eg. what your pet stops on (or doesn't step on/pick up) and its curse status. If you don't track the dog, remember the status, and then notice the un/cursed status in the end, you never find out stuff like that.

Yes. Some of those things can be discovered by trial-and-error (exploration in Reinforcement Learning, I guess?) but some others are much more of a tall order. One classic problem is that the things one does in early levels affect the progress of the game at higher (lower) levels but it's not clear how before getting there. So while I think that, ultimately, a big neural net should be able to learn all those tricks that human players have learned over the years, it would take an exhorbitant amount of data and training time, that perhaps few companies would want to spend on something obscure and esoteric like nethack.

I started playing when I was 12 and have gone relatively spoiler free.

I have ascended once. (I was 25 at the time)

I got stuck on the vibrating square many times, the castle many times, and have died on the elemental planes many times. Because I had no idea to do what was expected to me. The first I got to Medusa I died instantly.

Yes, I think some of the final puzzles are a bit too obscure... But I think you can beat it without spoilers.

But I would spend entire days to learn mechanics... IE: day of kicking sinks is a day I remember fondly in my childhood.

I am sure there is tons I don't know. But I have never used explore mode, and I have never looked up spoilers before ascending.

Note, I had some word of mouth friends, and it took me 3 years alone to figure out altars.

Where can you see "training rules that prohibit AI's from similarly being spoiled"? It says "We imposed no limits upon how teams would design or train (if relevant) their agents" so seems they do allow spoiled AI.

Even spoiled, when it's achieved, an ability to complete a task like ascending in NetHack seems more world changing to me than the LaMDA conversations.

I'd be completely flabbergasted if they trained an AI to parse source code while playing a game. I think it's the "lack of addition" rather than the "expressly disallowed" training rules that are being referred to.

The AI doesn't need to parse the source code. I haven't played Nethack, but based on the example given of writing ELBERETH in the dirt, the person training the model could add that to the potential actions being considered at each step and the model learns what it does by playing a million games.

They wouldn't necessarily need to parse the source code. E.g. initially training a neural network on replays of human runs (AlphaGo style, as opposed to AlphaZero) would probably be considered "spoiling" in the Nethack community, but it's a plausible approach assuming you can somehow obtain a training dataset.

I used to read the source code before I managed to find a guide and I never won the game. That could be Hack, not Nethack, too many years passed by.

It's worth noting that even people who have been spoiled still usually find it _extremely hard_ to beat NetHack.

Reply on news.ycombinator.com