I've always been interested in understanding situations where this is the case (and the opposite, where the engine favours one side but it seems to require a long, hard-to-find sequence of moves.
Playing out the top lines helps if equality requires perfect play from one side.
But the position Ding was in was neither sharp nor complex. A good analog to the position there is the rook + bishop v rook endgame. With perfect play that is, in most cases, a draw - and there are even formalized drawing techniques in any endgame text. But in practice it's really quite difficult, to the point that even grandmasters regularly lose it.
In those positions, on most of every move - any move is a draw. But the side with the bishop does have ways to inch up the pressure, and so the difficulty is making sure you recognize when you finally enter one of those moves where you actually need to deal with a concrete threat. The position Ding forced was very similar.
Most of every move, on every move, led to a draw - until it didn't. Gukesh had all sorts of ways to try to prod at Ding's position and make progress - prodding Ding's bishop, penetrating with his king, maneuvering his bishop to a stronger diagonal, cutting off Ding's king, and of course eventually pushing one of the pawns. He was going to be able to play for hours just constantly prodding where Ding would have stay 100% alert to when a critical threat emerges.
And this is all why Ding lost. His final mistake looks (and was) elementary, and he noticed it immediately after moving - but the reason he made that mistake is that he was thinking about how to parry the other countless dangerous threats, and he simply missed one. This is why most of everybody was shocked about Ding going for this endgame. It's just so dangerous in practical play, even if the computer can easily show you a zillion ways to draw it.
Making one that realistically plays like a human is an unsolved problem.
Now that i think about it, i remember the people in the alphago documentary talking about the bot giving its moves percentage scores in both how high winning % the move had and how high % chance that a human would have made the same move that it just played. I wonder why they never showed what a full game of the most human-like moves from alphago would look like. Maybe it actually worked, by feeding it all the pro games in existence, and training it to play the high human % instead of the higest win probability moves like they did in the end.
What evidence lead you to think that, and how surprised would you be to be wrong?
Making a computer have a 50% score against a 1300-rated human is way easier than making it play like a 1300-rated human.
For the former, you can take a top-of-the-line program and have it flip a coin in every game whether to make a random move every move or not.
Despite decades of research, nobody has found a good way to make computers play like humans.
This means that the LLM does not actually have a lot of training data available to learn how a 1300 would play, and subsequently does a poor job at imitating it. There is a bunch of papers available online if you want more info.
I think the chess model would be successful at producing the desired outcome but it's not as interesting. There's something to be said for being able to write down in precise terms how to play imperfectly in a manner that feels like a single cohesive intelligence strategizing against you.
https://github.com/LeelaChessZero/lc0/pull/1791#issuecomment... https://lczero.org/blog/2023/07/the-lc0-v0.30.0-wdl-rescale/...
But even that isn't a good proxy.
Humans cannot out-FLOP a computer, so they need to use patterns (like an LLM). To get the human perspective, the engine would need to something similar.
Even Stockfish uses a neural network these days by default for its positional evaluation, but it's relatively simple/lightweight in comparison to these, and it gains its strength from being used as part of deep search, rather than using a powerful/heavy neural network in a shallow tree search.
I don't think the patterns are very human, but they are very cool.
Thank you.
To give a trivial example, if I take your queen, then recapturing my queen is almost always the single playable move. But it's also a line that you will easily find!
Conversely, in a complex tactical position, (even) multiple saving moves could all be very tricky for a human to calculate.
"a level" and "if given enough depth" are both underselling it. Stockfish running on a cheap phone with equal time for each side will beat Magnus 100 games in a row.
By this I mean, you can give me a winning position against Magnus and I’ll still lose. Give a winning position to Magnus vs Stockfish and he might draw or even win.
I wish top players like Magnus would do more exhibition games against top computers. They don’t have to all start with equal material or an equal position.
In the WCC match between Caruana and Carlsen, they were at one difficult endgame where Carlsen (the champion) moved and engines calculated it was a "blunder" because there was a theoretical checkmate in like 36(!) moves, but no commentator took it seriously as there was "no way" a human would be able to spot the chance and calculate it correctly under the clock.