A human just defeated an AI in Go. Here's why that matters | Hacker News Reader

A human just defeated an AI in Go. Here's why that matters

zmescience.com

66 pointsamadeuspagel22 comments

This reminds me of the anime Hikaru no Go. In one episode, prodigy named Touya Akira (the main rival of the main character) is being bullied by his schoolmates. They challenge him to a 3-board match (he has to play three games simultaneously, each against a different person) with his eyes closed. Because of his reputation, he accepts the challenge. He calls out his moves and the opposing player places the stone. The opposing players also call out their own moves so that Akira can know what's happening in each of the games. he has to construct and track what the three boards look like constantly using only his memory. The one game of the three where he starts losing is the one where the opposing player is the least experienced. It's not the skill that starts defeating Akira. Rather, it's the idiocy of the amateur who doesn't know what he's doing, making it more difficult for Akira to track what the game actually looks like, because it's harder to remember and track nonsensical moves.

In “Thought And Choice In Chess”, A.D. de Groot (https://en.wikipedia.org/wiki/Adriaan_de_Groot) noticed that for chess.

He asked good chess players (including Euwe and Alekhine) to take a short look at a board and then replicate it from memory.

They were better at doing that for real games than for artificial setups.

Also, their errors tended to be non-essential. When they misplaced pawns, for example, that wouldn’t affect the plan to pick.

He also asked amateur-level players. I don’t remember whether they made more errors, but they did make more essential errors, and for them, the difference between ‘logical’ and ‘illogical’ board setups was smaller.

It reminds me of one of Murphy's Law saying that an amateur is more dangerous than a professional since they are more unpredictable.

When I used to play a lot of CS:GO this effect was very obvious.

Ranking up was essentially learning to counter strategies at a certain skill level.

But sometimes when you played much weaker players your playstyle would be over optimized to predict/counter a different style of play. The solution was usually to stop overthinking it and just rely on your raw aim mechanics to beat them.

Exactly my experience as well. When I went back to playing and dropped to silver, you could frequently hear me go "what the fuck is he doing over there?!", Surprised that an enemy who lacks mechanics would not act like a reasonable opponent would, and surprise me.

Back in the days of Battle.Net and Warcraft III, I used to sit on my friend's computer playing 1:1 games. I played different races a lot and sort of got good, but I was bored.

So I decided that I would exclusively play Night Elf and build nothing but Protector Trees.

Now Protector Trees are Night Elf "buildings" which are, of course, rooted in the ground but they can throw boulders at attacking units, thus defending the home base. Protectors also have the unique ability to uproot themselves and move around the board, then take root somewhere else. I believe they are not capable of hurling boulders whilst uprooted.

So that's what I would do: I'd build a bunch of Protectors (and enough Wisps to repair them all) and then I'd uproot a "formidable" Protector army and march to the enemy base.

This was mostly ineffective and readily defeated by a capable adversary, but I really trololoved the reactions some of them had. They would be absolutely flabbergasted that I'd wasted my time on something so dumb. But it was undeniably hilarious from my perspective to just watch a bunch of trees waltz into the enemy's home base and start hurling boulders at everything in sight.

Quick info:

> This is where Kellin Pelrine steps in. Pelrine is a good player, but an amateur. Specifically, he's one level below the top amateur ranking. He's also one of the study authors, so he was well aware of the vulnerabilities of KataGo, so he thought why not try his own hand?

> Apparently, it was surprisingly easy to find a way to defeat AI by exploiting its weakness. Pelrine managed to beat KataGo 14 out of 15 times. For comparison, KataGo beat AlphaGo 100 times out of 100, and AlphaGo beat mankind's best player 4-1.

One caveat is that this is vs AlphaGo which was trained on pro games, so has biases in what it's been trained on. I wonder if it would do as well against AlphaZero which is only from self-play. The article says KataGo is the strongest, so stronger than AlphaZero. Because of non-transitivity, it's also not known if the same weaknesses exist in AlphaZero. The warning for weaknesses in AI/ML is still of great importance when applying the tech.

Analogous sui generis weaknesses in human cognition are the far greater threat.

From TFA: "... KataGo, a machine-learning algorithm that taught itself how to play, ...".

My question wasn't about KataGo. It was about it beating AlphaGo and could it also defeat AlphaZero 100 times out of 100. The fact that both KataGo and AlphaGo were self-taught doesn't answer if KataGo would defeat AlphaZero. I'm only taking TFA's assertion that it's the strongest with only a reference to AlphaGo.

Yes the trick used against KataGo would likely not have worked on AlphaGo.

But then again ... It took humans not AlphaGo to find the exploit.

Still a victory for the meatbags.

Still a victory for the meatbags HAHA

Another team beat AlphaZero with adversarial attacks: https://arxiv.org/abs/2004.09677

Good article. This is a great example of "AI"'s inability to extrapolate. Yes, it's able to find amazing shortcuts that no human could working in-distribution, but it has no understanding beyond that, and so can fail as outlined. I wish people would bear this in mind more as they pontificate about how we're near AGI and all that implies. It's all Eliza with just a really big set of phrases it can respond to. Doesn't mean it isn't useful but it's so much different than all the silly musings about sentience and whatnot.

Arguably "AI" is already plenty general enough, it's just not very intelligent.

Guess this is similar to ChatGPT and “ davidjl” - https://news.ycombinator.com/item?id=36242914 - and tricking image categorisers with specially designed noise.

Today’s AI seem to have all sort of unexpected failure modes which can be exploited by adversarial attacks.

Like humans?

The good thing is we have a lot of knowledge on human failure modes though and have planned around most of them.

AIs are currently an unknown quality when it comes to their failure modes* and we are often caught by surprise when they fail at something that we expect they should be able to do because we don't fail at it.

* Given how little we understand about how it all actually works in practice, I have a feeling we will never be able to fully grasp their failure modes unlike traditional computing.

I think it depends on what you mean by understanding. KataGo taught itself to play go by explicitly deprioritizing “losing” strategies. This means it didn’t play many amateur strategies because they were lost early in the training. This is hard for a human to understand because humans all generally share a learning curve going from beginning to amateur to expert. So all humans have more experience with “losing” techniques. Basically what I’m saying is, it might be that the training scheme of this AI explicitly prioritized having little understanding of these specific tactics, which is different than not having any understanding.

The Go book The Treasure Chest Enigma by Noriyuki Nakayama starts with an essay that gives the book its title. The Treasure Chest Enigma is a life-and-death problem too hard for professional Go players, but easy for amateurs. A real life problem that caused much consternation and merriment in the Nihon Ki-in in 1958.

It relies on the technique that is at the heart of this paper. There is a small dead group completely surrounded by a larger group. But the larger group runs out of liberties and the small dead group comes back to life when the big group is captured.

This kind of thing never comes up in professional play. (The game that it came from was part of the qualifying tournament for professional shodan; Mizuno Hiroshi and Noriyuki Nakayama were not yet professional players.) All the top players had drilled so hard on the life and death problems that actually arise in professional play that they had developed a peculiar blind spot around capturing races with one group completely surrounded by another.

My expectation is that the first fifty years of Artificial Intelligence disasters will be dominated by humans being over-impressed by the intelligence of AI systems and giving them responsibilities that they are not read for. This article https://www.gbnews.com/news/ai-latest-news-judge-legal-decis... already speculating that "AI machines 'could soon make legally binding decisions' in UK, says senior judge" is the kind of thing I fear.

I don't know what to make of The Treasure Chest Enigma.

Theory one: it tells us that we are already coping successfully with this problem in human intelligence. Yes, AI has weird mental blind spots, but people do too and there is no need to worry.

Theory two: Human mental blind spots are smaller and more obscure that the mental blind spots of current AI. There is plenty of quantitative difference so that we can legitimately worry that AI blind spots will cause unexpected disasters. We will be taken by surprise because we are used to human intelligence with its small, benign blind spots. The significance of The Treasure Chest Enigma is that it exists, which tells us that mental blind spots are inherent to intelligence. We will not solve the problem of AI having large, dangerous mental blind spots easily. We can only hope to shrink the problem down to an ineradicable core.

> ...there could be fringe situations where it behaves badly

I think the more important takeaway is that we humans don't know those fringe situations or corner cases beforehand. No way around finding the problem in prod.

I thought both alpha go and its zero ver. retired. Is that an issue only with KataGo?

Reply on news.ycombinator.com