Hacker News Reader

Comment by Mtinie | Hacker News Reader

> According to DeepMind, AlphaZero uses a Monte Carlo tree search, and examines about 60,000 positions per second, compared to 60 million for Stockfish.

The previous statement was talking about how much faster and more efficient AlphaZero is, but the interpretation I pickup from that sentence is the opposite. Is this a “golf score” situation where lower is better?

View on HN

When Stockfish evaluates a position, it explores moves to a greater depth (a greater number of plays ahead), with its guesses and the value of the final board arrangements it can get to estimated using a relatively simple heuristic. AlphaZero evaluates the different potential moves using the neural network, which guides the search with a very complex heuristic that implicitly incorporates a tremendous amount of depth from prior games that have been incorporated into the model. Similar to the way an image recognition model takes in a whole image and says "this is an image of a goat," AlphaZero takes in a whole board and says "this is a winning board."

> Similar to the way an image recognition model takes in a whole image and says "this is an image of a goat," AlphaZero takes in a whole board and says "this is a winning board."

IIRC, AlphaZero has two outputs from the neural network. You described the first output.

The 2nd output was absolutely critical to it growing in strength. In effect, this 2nd output value is the difference from AlphaGo and AlphaZero. The 2nd output value guides the monte-carlo tree search.

Naive MCTS looks at board positions randomly. AlphaZero's MCTS looks at board positions the neural network deems "interesting". In effect, the neural network both guides the search (output #2), and evaluates the position (output #1).

MCTS chooses a position based off of the "interesting factor", as well as "how much that position has been evaluated". Ex: if "Knight to c3" has been evaluated 1-million times, MCTS will try to look at other positions. But if the neural network says that "Knight to c3 is really, really interesting", MCTS will still favor to look at that position, more so than other positions.

Etc. etc. down the hierarchy of moves.

AlphaZero does effectively a pattern matching of the board, whilst a traditional engine needs to calculate and rate every move into depth (<15). This search space is exponential, whilst AlphaZero's evaluation function is constant.

With deep search trees as in Chess or even more so in Go, a pretrained evaluator function easily beats any tree search. Applied AI.

AlphaZero: Few positions, very thorough evaluation. Stockfish: Lots of positions, more simplistic evaluation.

i don’t think either is better or worse, it’s just different strategies.

a probably somewhat wrong but evocative way to think about it is that what AlphaZero does is closer to “intuition”; it can look at a board and immediately tell very well how good it is. maybe you could argue this is more like how human grandmasters play.

Stockfish, on the other hand, has much simpler guesses for how to evaluate a board, but they can be computed very quickly.

Both of them employ a tree search: “if i do this move then they’ll have three countermoves and i can respond in 16 ways...”. (It’s Monte Carlo because they randomly sample, rather than exploring exhaustively.) But Stockfish is able to explore many more possible outcomes, whereas AlphaZero explores far fewer but is able to focus only on those that really are promising.

To all of the people responding, thank you for explaining how it works.

AlphaZero using GPU while Stockfish uses CPU. Stockfish performs (an almost brute) force calculation of positions while AZ has to "Guess" which moves are better based on it's neural network. In this case GPU is much slower than CPU. If AZ could calculate 60 million moves per second, stockfish would have no chance to win against it.

tldr: Higher is better.

Reply on news.ycombinator.com