Hacker News Reader

Comment by piker | Hacker News Reader

Question: does Stockfish beat more generic systems like AlphaZero?

My understanding is that AlphaZero only really existed for a year or two; there's no objective way to compare it at the moment.

Leela Zero tried to open-source that work, but Stockfish incorporated a number of improvements from AlphaZero, including a neural network and a different search method, and consistently beats Leela Zero.

I have a book, "Game Changer", in which a chess expert calls out several instances where AlphaZero made moves surprising at the time; situations where all chess engines rated things one way and AlphaZero rated them differently. When I enter them into Stockfish now, it usually rates things more similarly to the way AlphaZero did, and often chooses the move chosen by AlphaZero.

The only real test of course would be to dig up AlphaZero and run it again; but I think based on the evidence we have, Stockfish of 2026 would probably trounce AlphaZero of 2018 with equivalent compute available.

I guess someone can twit Hassabis and ask him :D He certainly has access.

who is Hassabis?

Demis Hassabis [1], co-founder of DeepMind

[1] https://en.wikipedia.org/wiki/Demis_Hassabis

Isn't one problem that it's hard to determine what equivalent compute is, for CPU search vs a neural net based engine like AZ or Leela?

One way would be to calculate a cost per game, factoring in both electricity and an amortized cost of the hardware, maybe having a penalty too for extra time run (e.g., if focusing only on hardware depreciation and electricity, 1 minute of TPU would translate to 2 weeks of CPU, that 2 weeks of waiting still costs you something). Obviously this isn't stable, as relative prices of GPUs and memory shift over time, and it's somewhat sensitive to setup; but done right it's probably more "what a user actually wants to know", in terms of what it would take to get equivalent performance.

It's more fundamental than that. AlphaZero is a shallower search with a heavier evaluation function. Stockfish is a deeper search with a lighter evaluation function.

In chess, depth usually wins because of how narrow the search tree is compared e.g. to Go.

Interesting, and if you don't mind, where do we put humans (and superhumans like Magnus Carlsen)? I think they have a heavy evaluation function and do shallower search.

Way way shallower search. A human doesn't consider more than a few candidate moves. But they (we?) build up much better intuitions and heuristics

The Stockfish NNUE is completely unrelated to AlphaZero.

It's a neural network rather than a bunch of hard-coded rules. That turns out to make a big difference.

Actually, there's this interesting snippet from the release page:

> These techniques have been applied to hundreds of billions of training positions, all of which have been consistently rescored using a strong Leela net.

So Stockfish's neural network evaluator is actually trained using Leela Zero.

"Completely unrelated" is not quite true. Stockfish current NNUE models are trained on LC0 training data. LC0 is pretty much an open-source community replication of the ideas from AlphaZero.

I stand corrected. However, fundamentally, the idea of "tiny CPU-only neural network" combined with traditional alpha-beta search is substantially different from "big GPU network" combined with Monte Carlo Tree Search. And historically the NNUE came from a 2018 idea for shogi engines rather than from AlphaZero.

Leela Chess Zero did surpass Stockfish for a while, until Stockfish switched its eval to NNUE. Modern Stockfish would annihilate AlphaZero.

> My understanding is that AlphaZero only really existed for a year or two;

It still exists, but it's private / internal, and sometimes used for a few different things.

It was used by Kramnik to test the hypothesis whether no-castling chess was viable (basically chess, but disallowing castling). That was a year after DeepMind published the match they ran of AlphaZero versus Stockfish.

It was still in use last year, I remember seeing some Grandmasters with interests in chess studies were invited by DeepMind to judge the beauty of chess problems composed by AlphaZero (or whatever form the thing that used to be AlphaZero is now).

So it's still kicking around in the background.

> It was used by Kramnik to test the hypothesis whether no-castling chess was viable (basically chess, but disallowing castling).

Viable in what way? That it's advantageous to never castle if an engine learns to play with that directive? Or that it still makes for a fun game with that new rule?

The hope was that it would be a slight rule tweak that would lead to fewer draws without favoring white too strongly.

I think so, see the tournament results: https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship#...

Technically nobody knows, because AlphaZero was never made public. In practice, we know Stockfish 19 would destroy it, because it destroys open-source reimplementations of AlphaZero.

On average, yes. Stockfish is the strongest engine and beats AZ-like implementations like Lc0 and the like. But on a game to game basis Lc0 can still win some games, depending on the starting position. It's rare that Lc0 can win both black and white starting from the same position, tho.

There are a few yt content creators that cover great engine games, if you're curious.

yes and it has for a very long time. Stockfish took the neural net approach after AlphaZero showed it was a good idea as all modern chess engines have

They basically replaced the static position evaluation function with a NN approach and kept the rest of the tree search mostly intact.

I ran fishtest on an old cluster for a while. Was nice to watch the stockfish team methodically improving things. Basically, every commit has to show it can beat the main branch before it gets merged.

AlphaZero does not exist, as far as modern computer chess is concerned

Leela proved that the ideas worked, regardless of the conditions that the AlphaZero vs Stockfish matches were run in.

What's funny is that if computer chess had paid more attention to computer shogi, Stockfish could have already been using NNUE by that time, and not lost to AlphaZero. But perhaps that would have stifled Leela's development, and would have led to worse training data for Stockfish in that parallel universe's present day, making it weaker overall.

> like AlphaZero?

Unknown, it was only around for a little while however there is Lila which is based on the same ideas as AlphaZero (and also a fantastic project generally) they generally trade blows with Stockfish coming out very slightly ahead.

That said Stockfish also uses an NN (NNUE) these days so the line is getting more blurry.

It's widely believed so because Stockfish finds all of the right (or better moves) in the games from the original paper.

Like a red headed stepchild.

Reply on news.ycombinator.com