Hacker News Reader

Comment by jka | Hacker News Reader

One interesting result from this challenge that could stoke debate is that symbolic (rules-based) agents are clearly achieving better results than neural (machine-learning-based) agents.

Maybe that's related to the nature of NetHack itself, compared to other games where ML has seen (often overwhelming) success?

Will rules-based systems continue to succeed for NetHack, or is this a temporary condition until the state-of-the-art in machine learning improves?

View on HN

Nethack is notorious for being utterly unwinnable without spoilers. There are a few claims of people who have done it, they don't hold up very plausibly: at the very least those people got indirect spoilers from talking with people who were spoiled.

So it's not really surprising that hard-coded rules based bots incorporating spoiler knowledge beat learning bots trying to learn the game directly from the interface.

DCSS might be a better target, in that it doesn’t reward hanging around on one level, and has far fewer things you need to be spoiled on (you need to know that picking up runes is good, and how to kill hydras, that’s about it) But it has a steady stream of qualitatively different challenges on different floors, forcing a lot of contextual behaviour. And there’s a little bit of long term strategy that can really mess up an AI.

DCSS has been beaten by a handcrafted Lua bot, written by the (at the time) indisputably best player in the game. Unlike the approach here, it got to benefit from human expertise, and also it can only win with a very narrow set of characters. Learndb entry:

qw: A fully automated lua bot written by elliptic, with some code borrowed from parabolic and xw. The first DCSS bot to ever achieve an uninterrupted and unassisted win (see '!lg qw won 2'). Now maintained at https://github.com/crawl/qw by the DCSS devteam. See qw[2] for a summary of results.

As of 0.29-a, qw has a 0.36% winrate with GrBe with 1 win in 276 attempts. See https://crawl.dcss.io/crawl/morgue/qwqw/morgue-qwqw-20220613... and games are sometimes played on cdi. Historically its best 3-rune winrate was 15% DDFi^Makhleb, and, for 15 runes, about 1% with GrFi^TSO before the 0.28 hell rework.

Branch order: D -> Lair:8 -> Orc:3 -> D:15 -> S:5 -> Vaults:4 -> Depths:5 -> S:5 -> Vaults:5 -> Zot

On the online servers, qw plays with an extra added delay so that it doesn't use too much server CPU. Playing locally without this delay, qw is much faster.

DCSS is just a much simpler game than Nethack, so a bot would likely do well.

It's questionable if it's simpler (notwithstanding all the things they've cut from dcss over the years). The main obstacle for bots is that to win nethack you need to know lots of things you're unlikely to find out by trial and error. Like how eating a lizard corpse can prevent stoning from a cockatrice's hissing, or hallucination can prevent a Lich's touch of death, how you can apply a towel to wipe your face if you've been blinded by a thrown cream pie etc. Know all the tricks, and the game becomes much easier.

You need a lot of knowledge to win DCSS, too, but it's more about wise use of limited resources, playing the hand you're dealt, and the dangers of various locations. It's much harder to codify as spoilers, but you will learn by trial and error.

But we have neural networks for NLP, who could read the spoilers.

I imagine you'd at least have to inject domain knowledge into the model, the ruleset isn't as simple as chess or go, and all the places where a bunch of actions have to be strung together just so will lead to a very spiky parameter space where gradient descent struggles.

I think neurosymbolic ai could do better here. It seems to be an interesting field and this might be a cool application.

Consider that the existing MVP for ascension is a neutral network.

I wasn't able to find out about that, from a quick scout around - could you share a link to it?

(I did find BotHack[1] and a video of its' first ascension[2], both of which are cool)

[1] - https://github.com/krajj7/BotHack

[2] - https://www.youtube.com/watch?v=unCQHAbGsAA

Human brain.

Hehe, ah, of course :)

Reply on news.ycombinator.com