But the game is not solved, which is where MCTS comes in. It picks a move, and plays through it, selecting moves that are good but varied, until it learns more about which side would likely win (typically by expanding a leaf of the search tree). Each playthough is a visit. Each visit allows updating its estimate of the win probability for that move, and thus, changes the policy probabilities.
More visits typically anchor the policy estimates into reality further, yielding better play. In fact, this is how the policy network learns.
On a good consumer GPU (RTX 3060), I get 1.6K visits per second, so 100K takes about a minute. That makes a game last three to five hours typically.
But the paper emphasizes that even with 10M visits per move (>1h per move), KataGo loses 72% of games against this exploit[1].
[0]: https://arxiv.org/pdf/1902.04522.pdf
[1]: https://goattack.far.ai/pdfs/go_attack_paper.pdf#figure.capt...
I assume this means they are supposed to be stronger than the best humans. If that is the case, this is a high level amateur defeating a stockfish-like engine.
Now the article mentions Leela Zero and that one is an open source version of Alpha Zero where they used a community effort to do the same as Alpha Go zero described in the paper.
From the status page here: https://zero.sjeng.org/
I see that this took multiple years but was declared finished in 2021. Status page says it played games in training are 21 million while Wikipedia says that for alpha go zero it was 5 million.
So you'd think it's better but actually it might just mean that they are not using they same techniques after all and so maybe worse.
So question for me still is whether this technique would actually be applicable to Alpha Go Zero.
But for me, this article is more a win for AI, because it means that they have found better techniques to find blind spots in models which can from now on be used in training to eventually get rid if those.
The fundamental issue is that they all do self-play, and those plays follow the same statistical distribution as a result. Weird games happen rarely, while the adversarial network forces a weird game every time.
On the other hand, I am not sure whether MuZero would be vulnerable. I would guess so, but its dynamics system may learn something extra about the state of the board.
Either way, it seems clear that future self-play training should include adversarial models, instead of pure self-play.
(They also all share another weakness, including MuZero, which is that their MCTS algorithm doesn’t allow the neural network to gain board information from their evaluation of possible futures; only win rate. Meanwhile a human sometimes realizes, while navigating a possible sequence of moves, that one particular early move could turn the tide.)
KataGo adds a few situations in training which didn't appear in self-play, "blind spots" that a shallow search depth fails to notice as good moves, so that it still tries to learn from positions it would not generate.
It is a little bit like learning to tie your shoelaces. Brains are not great at guessing the qualities of knots among all possible ties, but if you see it done once, you're set for life.