Edit: Alphazero's record against Stock Fish is 28 wins, 72 ties and 0 losses.
In a 100 game matchup, AlphaZero had 28 wins 72 draws and 0 losses. I'd classify that as orders of magnitude better.
https://www.chess.com/news/view/google-s-alphazero-destroys-...
Check out the Leela project, which is trying to reproduce and improve the AlphaZero project to also beat Stockfish dev: http://lczero.org/ (at this point they are about equal with Stockfish 9)
28Win 72Draw and O Loss in my opinion reflects at least 300 ELO (8 times stronger).
I guess that the spectacular 28-0 was somehow confusing my ELO opinion engine by concealing the fact that 72 draws are a lot more difficult to achieve than 12 draws (28-12-0 would have been a 300 ELO difference).
https://www.chess.com/news/view/google-s-alphazero-destroys-...
But yeah, it would be interesting to see a side by side comparison of alphazero and stockfish's live analysis of these games.
> AlphaZero compensates for the lower number of evaluations by using its deep neural network to focus much more selectively on the most promising variation [1]
Even if you compare CPU to GPU by price, and not speed, it seems pretty even. It clearly has to sacrifice speed by making more intelligent pruning decisions than stockfish.
1: https://en.wikipedia.org/wiki/AlphaZero#AlphaZero_vs._Stockf...
2: https://sites.google.com/site/computerschess/stockfish9-benc...
3: https://www.reddit.com/r/hardware/comments/9jyts8/rtx_2080_t...
The computer itself was strong. But the weird timing setup, the disabled databases (ie: both opening book AND endgame book was disabled. Stockfish normally plays PERFECTLY when the board is reduced to 6 pieces or less, as well as perfectly knows the winner / loser in every 6-piece setup. But that was disabled for the AlphaGo games)
Without the ability to run AlphaGo on our own and recreate the test, we have no way in knowing how AlphaGo would work under "fair" conditions.
-------
And btw: 1GB of RAM is a lulzy setup. You put all the CPU time you want, but gimping the RAM down to 1GB is... weird. Its a VERY suspect "test" that none of us can replicate.
Remember that if stockfish uses tablebases and opening books it's really "cheating" by using human knowledge.
Although I am surprised that they reduced the hashtable so much and also used stockfish 8 when stockfish 9 was available. It was not RAM but just the hashtable size also their "test" has been replicated by plenty of people.
We have our own "replica" of the NN playing chess in lichess.
So if A0 wasn't built to access them, but stockfish did in the match, then that would be an unfair advantage.
Why is that an unfair advantage? AlphaZero developers wanted to prove that their AI was better than Stockfish.
They failed to do that. If AlphaZero wasn't built to access Tablebases, then they should have built it to access tablebases. Don't unfairly gimp Stockfish because you're lazy at programming.
One way is to do according to the most capable configurations of each program, so that means Stockfish with all of them. Other way is to specify the same limits of time, memory, etc of each one, although that is still not the excuse to avoid the opening books and tablebases and so on unless they use up more memory than AlphaZero does.
I thought the "point" of AlphaZero was that it was better than the sum of human knowledge.
By the way: tablebases are computer generated. There is no human alive who knows all combinations of 6-piece endgames. Tablebases are brute-forced endgames, created by computers to be used by computers.
Stockfish itself was only really designed to play the midgame. Its designed to be used with both tablebases and opening databases.
> It was not RAM but just the hashtable size also their "test" has been replicated by plenty of people.
Citations please. I'd love to see a "proper" test of AlphaZero vs Stockfish.
Your incorrect perception comes from the fact that, at very high levels (as shown by this match as well), draws become way more common than for lower levels. A 3400 vs 3300 Elo match might be 28-72-0, a 1400 vs 1300 match might be 60-8-32. At the lower level, you see one player winning twice as many games as the other, at the higher level you see one player losing all the time; that looks different to you, but as far as Elo difference is concerned the two results are exactly the same.
I dunno why AlphaGo devs would do that, aside from artificially gimping Stockfish on purpose.