> This is where Kellin Pelrine steps in. Pelrine is a good player, but an amateur. Specifically, he's one level below the top amateur ranking. He's also one of the study authors, so he was well aware of the vulnerabilities of KataGo, so he thought why not try his own hand?
> Apparently, it was surprisingly easy to find a way to defeat AI by exploiting its weakness. Pelrine managed to beat KataGo 14 out of 15 times. For comparison, KataGo beat AlphaGo 100 times out of 100, and AlphaGo beat mankind's best player 4-1.
One caveat is that this is vs AlphaGo which was trained on pro games, so has biases in what it's been trained on. I wonder if it would do as well against AlphaZero which is only from self-play. The article says KataGo is the strongest, so stronger than AlphaZero. Because of non-transitivity, it's also not known if the same weaknesses exist in AlphaZero. The warning for weaknesses in AI/ML is still of great importance when applying the tech.