Adversarial policies beat professional-level Go AIs
goattack.alignmentfund.org
goattack.alignmentfund.org
It should probably reject friendly-pass mode (which is a flag iirc) and refuse to pass in tromp-tailor rules except under extreme circumstances.
The TL;DR is that they win on a rules technicality. (I think by using a ruleset which isn't used by KataGo)
> However, KataGo plays a pass move before it has finished securing its territory, allowing the adversary to pass in turn and end the game. This results in a win for the adversary under the standard Tromp-Taylor ruleset for computer Go, as the adversary gets points for its corner territory (devoid of victim stones) whereas the victim does not receive points for its unsecured territory because of the presence of the adversary’s stones.
KataGo has finished securing its territory under virtually all rulesets but Tromp-Taylor. Tromp-Taylor is a fine ruleset (courtesy of our own https://news.ycombinator.com/user?id=tromp), but not one you'd use to, like, play against someone on the internet, or in real life.
When they induce KataGo to play bad moves, like Lee Sedol did in the only game he won against AlphaGo, they'll have my attention.
> A player's score is the number of points of her color, plus the number of empty points that reach only her color.
No one ever plays games to completion like this.
Oh I wish.
> When they induce KataGo to play bad moves, like Lee Sedol did in the only game he won against AlphaGo, they'll have my attention.
That said, we are working to do exactly this! In particular, we found in the paper that we could patch the victim to not pass except in the end-game and defeat this adversarial policy. But, when we repeat the attack, we do eventually find an adversarial policy that can exploit the victim -- and this doesn't depend on the rule set. That part is still a work in progress, we've only got it working against a victim without search for example, but we hope to put out an updated revision including this in a month or two!
I hope to see the result soon.
> But KataGo was trained under (modified) Tromp-Taylor which we also evaluate under, so we think from an AI security and robustness stand point it's interesting that it fails at the rule set it was trained on.
Have you done anything in particular to induce this, or is it just the case that KataGo is completely incompetent at evaluating what constitutes a pass-alive territory?
> That part is still a work in progress, we've only got it working against a victim without search for example
That would still be amazing. KataGo without search is still a beast.
---
By the way, your website is weird:
1. "strength of a top-100 European professional" - Europe doesn't have that many professionals?
2. "Yet our adversary achieves a 99% win rate against this victim by playing a counterintuitive strategy." - Well, if your main goal is to leave a stone in your opponent's territory, this strategy isn't very counterintuitive?