Hacker News Reader

Comment by airza | Hacker News Reader

An astonishing insert:

>A good example of human exploitation of the engine's failings is GM Hikaru Nakamura's defeat of Rybka in the following three-minute blitz game from the Internet Chess Club. Nakamura cleverly locks the position so that progress is impossible for Rybka, then he offers two exchange sacrifices to the engine. With the position locked, the engine's rooks have no value, but the engine thinks it has a winning material advantage. With a draw by the 50-move rule approaching, the engine sacrifices a pawn to avoid a draw, but this proves a huge mistake as Nakamura is then able to win the game, an incredible achievement!

View on HN

There is a video of this on YouTube

https://youtu.be/DTMnrUdd8Tw

Thanks for the link. The commentary notes that this was possible because Rybka is an algorithmic engine with hard constraints like "must try to win in 50 moves to avoid tie", and this is what Nakamura exploited to win. DeepMind and other deep learning/neural network approaches don't have hard rules like this.

> because Rybka is an algorithmic engine with hard constraints like "must try to win in 50 moves to avoid tie"

That rule isn't the engine's constraint, that's a rule of chess. 50 moves without a pawn move or piece capture (basically "irrevocable progression of the game") is a draw.

The chess engine thought it was winning, so it wanted to avoid a draw, even if it meant putting itself in a still winning but otherwise worse situation. Unfortunately for the chess engine it was mistaken, in the original position it wasn't winning, but drawing, and in the new position it wasn't still winning, but losing.

Thank you.

> 50 moves without a pawn move or piece capture (basically "irrevocable progression of the game") is a draw.

A player may call a draw if they wish.

This game was also in 2008, I don't think cell phones could even beat all humans beings back then.

These days a human couldn't draw against a cellphone, contrary to the article.

And Rybka thought, materially, it was winning, so didn’t want to allow Nakamura to be able to call the game a draw.

So it lost a “small” amount of material which transposed the game from one that was an obvious draw to one that was obviously winning for Nakamura. That’s the “contempt” effect that Nakamura kept referring to: the computer would rather continue the game in a slightly-less-good position than allow a draw from a better position.

Nakamura exploited the fact that it was wrong about its assessment due to the horizon effect. A traditional chess engine evaluates a position by starting from the current position and evaluating promising moves downward along the tree. In a locked-up position like the one linked, the computer has to evaluate an enormous number of positions, none of which will materially affect the game due to the inability for either side to force meaningful progress.

A human can understand the themes in a position and understand where they want the pieces to go, then work backward from there to see a path to achieve it. This is particularly easy when the opponent has no achievable setup of their pieces where they could counter you once you’ve achieved that position. In such a situation, you can (almost) completely discount any moves they make and only consider the path for you to get to your ideal position. Even an beginner-intermediate chess player (1600+) could analyze this type of position easily.

This was such a situation. Rybka had no moves to progress things (other than the game-losing sacrifice Nakamura wanted to induce). So Nakamura didn’t have to evaluate any of Rybka’s moves, he just had to put his pieces in the right place to be ready for the inevitable blunder when Rybka decided to sacrifice a pawn to keep its “winning” position thanks to contempt.

> A human can understand the themes in a position and understand where they want the pieces to go

So can a computer though. This is an interesting edge case, but it seems to me that this particular bot mostly evaluated odds of winning by the pieces, not by the position. A more modern algorithm will actually use the positions on the board.

What would a human do if told "you're up by two rooks, do you want to allow a forced draw?" without being allowed to look at the board.

Sure a computer can, conceptually. But they don’t, at least not in the way we generally do.

Traditional engines have been designed under the premise that it’s better to have an extremely fast evaluation function you can apply deeply down the tree than it is to have a slower, more thorough evaluation function you can’t apply to as many nodes. Any extra processing budget is spent on more intelligent pruning of the tree of moves that don’t appear likely to yield anything promising.

So an engine will look at a position like this, see that it has a sizable material advantage, and doesn’t give as much weight to the obvious positional conclusion that it has no productive path forward. It also won’t understand that “just” sacrificing a pawn turns the position into one that’s completely lost due to the chain of passed pawns it eventually results in.

Yes, the engine could be improved to understand this type of position, but without extreme care that easily results in it taking longer to evaluate every other position. The net effect is an overall weaker engine, unless you find a way to improve the evaluation without slowing down the evaluation function.

> Traditional engines

That's the crux of it though isn't it? These are complaints about a specific approach, not generally of computers, and not even of the best demonstrated models.

I'm also not entirely sure this is the correct way to think about it. If the function to evaluate the accept-a-draw decision is separate from the next-move decision. Then if the accept-a-draw is refused, doing the poor pawn sacrifice is, without question, the best available move. My interpretation is that the accept-a-draw is made in a vacuum here.

Was it actually configured with "no draw" policy? The chess.com explains that Rybka thinks it's winning because of material advantage.

Looks to me that a modern AlphaZero based engine like Leela would understand this position easily. Even SF with its new neural net based eval function might understand it. My Leela reports a close to 0 evaluation after Nakamuras first sacrify.

Why did Rybka sacrifice the pawn to avoid the draw if the resulting position was bad? Was its contempt setting really high or something?

Edit: looks like in the video Nakamura says the contempt factor was too high, and later it was changed.

Correct. Rybka believes it has the piece superiority and doesn't consider sacrificing a pawn to be an issue but the only pawn that it can sacrifice opens up that side of the board for black. Nakamura explains a little before the sacrifice that if they exchanged queens it'd be almost certain that Rybka would have to sacrifice that pawn to avoid draw.

Might need a glitchless chess league!

Reply on news.ycombinator.com