Win by Segfault and Other Notes on Exploiting Chess Engines | Hacker News Reader

Win by Segfault and Other Notes on Exploiting Chess Engines

chess.resistant.tech

124 pointswglb15 comments

Heh, cool. There's a bunch of bugs here: the FEN parser accepts totally malformed stuff, which desynchronizes the internal state and leads to some odd behaviour (as far as I can tell, this is what causes it to erroneously output no moves for a reasonable position), and Stockfish will pretty happily analyze nonsensical positions and explode. I looked into the latter issue, which can be triggered even with "mostly-valid" FEN input, such as the "fixed" FEN from "The Endgame":

    position fen 4kb1r/B2pqBpp/3P1n2/Q7/PP2PPP1/1K4RP/8/8 w - - 0 1
    go searchmoves

With a debug build, Stockfish crashes immediately on the following line (Position::do_move, position.cpp:726):

    assert(type_of(captured) != KING);

As noted in the article, the position is invalid as the black king is checked on white's turn. During move evaluation, the king capture is considered as a possible move for white, which crashes in the assertion.

In a release build, without that assumption, Stockfish happily continues examining this line without a black king present on the board, and somewhere deep in the evaluation attempts to locate the black king (evaluate.cpp:540):

    const Square ksq = pos.square<KING>(Us);

This fails, producing a ksq value which is undefined: it uses tzcnt, which returns an "undefined" value if the register is zero (and it is, because there is no king so the bitboard is zero). The likely value is 0x40, which is out of bounds; any later operation that tries to index by this will explode in exciting ways. For example, in the same evaluation function, we have

    b2 = attacks_bb<BISHOP>(ksq, pos.pieces() ^ pos.pieces(Us, QUEEN));

which indexes an array called BishopMagics with ksq; this goes out of bounds and accesses a wild pointer.

I was unable to reproduce the second crash, "Win by Segfault Attack".

It’s not too surprising that SF can be exploited. The interface is designed to be operated from a (non-adversarial) GUI, and efficiency is paramount, so it understandably has little sanitization.

Also, this would be more interesting on previous versions of SF before the NN eval, as the handcrafted eval has been battle hardened for years.

That said, this is still a cool analysis to identify a concrete exploit!

I think it's a misconception that safe handling of input will always cost performance. For Stockfish, I imagine the performance critical part is analysis, where input should already have been parsed.

You can easily crash Stockfish from the xboard GUI by editing the position and playing with two kings on each side.

Illegal position, sure, but validating it first would not affect the fast core logic.

The lack of validation is surprising because the code looks extremely clean and well structured otherwise.

The British rules of Chess or some British federation) used to allow promotion to a second king. Oddly you could also promote to your opponents color if you wanted too. These rules were abandon in the 19th century though.

Yeah that would be Law XIII of the Code of Laws of the British Chess Association.

Law XIII said, "When a pawn has reached the eighth square, the player has the option of selecting a piece, whether such piece has previously been lost or not, whose names and powers it shall then assume, or of deciding that it shall remain a pawn."

No restriction on piece type or color. So under those laws, promoting to a king or a piece of the other color would have been legal.

From what I've read though, this sounds like it was sloppy wording, and the normal promotion rules we are familiar with today were actually intended, except for the option of remaining a pawn, which was obviously intentional since it is explicitly called out, and was actually the controversial part of that rule.

Interesting. I suppose that could let one draw an otherwise dominated game where an opponent is up by more than a queen. Under such circumstances, it seems like it would be quite unlikely to come up in practice. The promoting to an opponents color is interesting, though I don't know that I'd be inherently opposed to it.

Who gets to move that piece? It would be your opponent wouldn't it?

>Who gets to move that piece? It would be your opponent wouldn't it?

More an amusing curiosity than anything truly interesting. It would rarely be practical to give your opponent another piece. Puzzle creators have apparently come up with a few ways to make a scenario where that would be a move required to win, but my understanding is that none of them are terribly realistic.

Yup. I can't imagine it happening in reality but I have seen a puzzle that's mate in one by promoting to an enemy knight, causing a discovered check and the knight can't be captured because it is the opponent's piece.

That is like the requirement which was added in FIDE-rules 1972 stating that king and rook need to be on the same rank while castling.

Before that you could promote a pawn to a rook and then castle vertically across the board.

That would have been an interesting novelty, for sure.

I assume those rules also let you capture the king?

It would take a careful reading of the rules, but I think it is more likely that the wording would make it such that putting either king in checkmate would win the game.

So you could promote to king, but that would be a liability so it made no sense.

A better exploit would be to promote to your opponent’s king in a way that immediately puts it in checkmate.

I forget the nuance but yes, so long as there were 2 kings of a color they were treated like other pieces until 1 remained I think.

The promotion of an opponents piece could end in a game where you promote your pawn to their knight which becomes a checkmate as the pieces were organized in a way that blocked the king.

Restarting in the middle from FEN has another problem, the FEN doesn't encode past positions, so you could trick it into draw by repetition.

Reply on news.ycombinator.com