Go grandmaster Shin defeats AI KataGo with a two-stone handicap | Hacker News Reader

Go grandmaster Shin defeats AI KataGo with a two-stone handicap

kedglobal.com

467 pointsgmays187 comments

It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his very peak.

In go ELO like scoring he’s something like 120 points over the next strongest player. No other player has ever broken a 3800 rating let alone 3850. Ke Jie (the previous long time champion) peaked at 3755. Shin Jinseo’s strength graph is the most absurd straight line.

https://www.goratings.org/en/

2 stones is historically the gap between a 9P ranked and a 1P ranked professional player (very roughly the gap between super grandmasters and an almost grandmaster)

That is to say it’s shocking that Katago (almost certainly significantly stronger than AlphaGo) is a mere 2 stones stronger than Shin Jinseo. I suspect it would be 3-4 stones vs any other human pro.

Deep link to Shin Jinseo's strength graph

https://www.goratings.org/en/players/1313.html

It would be interesting to find out what insight he discovered about the game to consistently rise like that.

It can't be just play like AI.

Any other Korean on the Korean Go program could have done the same.

In fact, many did when AlphaGo was the pinnacle of AI.

For a non-Go player, do you think this trend will persist, or is it more of a dead-cat/human bounce?

On one hand, Shin Jinseo is an outlier player of this generation. On the other hand, the newest generation of new pros will have exclusively learned by using the AI to tell them what the best move is, so there's reason to believe that peak human level has yet to be reached.

Distillation of our blessed models is no fair!

Could someone sufficiently motivated invest in training Katago to be able to beat Shin Jinseo with 3 stones of handicap? Unfortunately - probably yes.

This in no way detracts from how absurd and remarkable it is that Shin Jinseo can beat KataGo (it gets a LOT of training and architecture refinements https://katagotraining.org/#eloGraphButtons) with 2 stones of handicap.

Depends on where you put the perfect play ceiling. I think each side has good points in their respective favor. There have been strong pushes by both humans and computers in improving their best play in the past few years.

A 2 stone handicap, however, does not scale linearly with strength. It becomes relatively more impactful at higher levels. For pro players, 2 stones are gigantic, and for beginner players, they make zero difference. Same for a point advantage (komi adjustment). So there might come a point where it is physically impossible for a computer to beat a top human under some handicap.

Important to note that KataGo was double-handicapped. 20 seconds per move maximum; it couldn’t read deep. Against an amateur, it doesn’t matter, but against a historically strong pro it matters a lot.

...on a 4x 3090 rig. The game ran 299 moves, giving katago 100 minutes if it exhausted time on each move (which must be the optimal strategy under that time control). Shin used about 205 minutes, over twice as much time and of course had leeway to spend more time on difficult moves.

Based on the youtube video, it looks like katago was only using 16 seconds per move, is that right? https://www.youtube.com/watch?v=-86zF4mTWOY

Is 20 seconds on that hardware really overkill and well into the diminishing-returns curve, as a top-level comment suggested, or is it plausible katago could have played better if given 40 seconds per move?

match details: https://gostonebase.com/blog/shin-jinseo-vs-katago-kishin-ma...

As another question, does it not operate similarly to the top chess engines? The way the neural network systems work is by using the probabilistic matching paired with a Monte Carlo simulation. So you can get to extreme depth very rapidly. Obviously the breadth is going to be limited, but if the neural network side is well tuned (so high probability hits are indeed generally the most challenging moves), then that's not such a problem.

And you can run a huuuuuuuge number of sims in 16 seconds.

I believe the basic idea is similar, but there's an enormous difference in the space of possible moves between chess and Go. Go has a larger board and moves are less restricted. There was a major breakthrough in Go playing programs a decade or two ago when good Monte Carlo methods were developed for it. But still I think the ability to simulate many moves is less powerful in Go.

This is how alphazero worked, but not stockfish iirc.

The tree search in KataGo does not use Monte Carlo simulations at all. It is purely policy/value network based, and works insanely well indeed.

Considering the original AlphaGo ran on a full Google TPU rack, and here KataGo seems to run on a $10k computer, and won with 2 stones handicap, this is showing just how much advance there was in computer go

AlphaGo is a bit difficult to compare because their parallelism efficiency was abysmal. You might as well consider it a regular 8x or so GPU machine.

AlphaGo Master, the much stronger[1] one that beat Ke Jie a year later in 2017, ran on a single 4x TPU (v2?) workstation.

[1] From wikipedia, citing deepmind's paper: 4858 Elo vs 3739 Elo

I've not looked at the specific game in question, but in general, doubling the thinking time leads to a very notable strength improvement regardless of the base level of play. The increase becomes slightly smaller at higher levels but does not really diminish, and it currently looks like the ceiling is still a bit away.

If it is "only" 4x3090 at 16s, you will definitely get a drastic playing strength boost from doubling the thinking time. This is still clearly within the interval of a linear relationship between thinking time and playing strength, i.e. elo ~ log time. The relationship, to my knowledge, becomes less clear only starting at about 10-20x the number of playouts.

Source: Wrote a paper on this. https://ieeexplore.ieee.org/document/10645535

I don't think it's solely a matter of raw strength, but as Shin said, a willingness not to play to the program's strengths. I mean, one thing that rankled me about original Lee Sedol match was that Lee had no access to the program's "record" while the machine by the nature of the AI training process had effectively studied Lee's games in great detail.

I recall a while back someone came up with a set of "anti-computer" strategies that allowed even an amateur to defeat a strong go program. These moves weren't anything like ordinary go moves (and perhaps the "loophole" has been closed now) but imo, their existence suggests that a study of programs may reveal other unexpected weakness.

I saw the same things when the OpenAI Dota bots could eviscerate humans 1v1 - even pros lost!

Until a more average player confuses the AI with an unseen behaviour (pulling creeps between the towers etc) to get an advantage.

We saw this with AlphaStar too, but ultimately it feels like simply an exploit. I expect even a relatively simple modern LLM/model working with the custom transformer would have been able to address these exploits after a game.

I don't think exploit is the right term?

Anyway. Yes if you throw examples into training it will be able to handle the situation - but handling unseen things for me is a key goal.

In a march 2026 interview David Wu (lightvector, Katago’s creator at Jane Street) noted that he doesn’t have a systematic solution for the cyclic group problem, but adding examples to the training set mostly ensures Katago during MCT rollout figures it out. I don’t think there has been a post mid 2024 verified exploit.

https://gomagic.org/david-wu-on-building-katago/

Probably a better comparison from the chess world(in reasonably modern times, though perhaps players like Capablanca and Lasker could be mentioned as well. Alas, I don't think FIDE rating existed back then) is Bobby Fischer. In the july 1972 FIDE rating list he held a rating of 2785, the highest in history at the time, with Spassky in second sitting at a "measly" 2660, and only 13 players being above 2600 even.

If I counted corrected, Fischer went 24-3 in the world championship series around then. That excludes many draws and one forfeit vs. Spassky, a few draws vs. Petrosian, and nothing at all in his sweeps of Larsen and Taimonov.

> Katago (almost certainly significantly stronger than AlphaGo)

Interesting KataGo is an open sourced Go program written primarily by David Wu in C++ and recently heavily vibe coded by Claude. It's running on four Nvidia RTX-3090 GPUs with 96GB VRAM. [1]

[1] https://github.com/lightvector/KataGo

It significantly predates Claude, and has been one of, if not the best engine in the world for many years.

I had to read your comment five times thinking "how it is written in C++ if it's a go program?" Do'h!

KataGoGo

I read this comment before looking at the article and thought that the grandmaster beat the AI even giving the AI 2 stones. Too bad. But this way around is of course more realistic.

And of course you would need to take into consideration the scale of go ratings and chess ratings when making that comparison. With top chess ratings being around 2800, being 1000 less than the top go ratings, one would have to apply a factor of roughly 3/4.

If you scaled Shin Jinseo to 2800, you would have players with extremely negative ratings. This page shows ratings of European players on a roughly aligned scale: https://europeangodatabase.eu/EGD/createalleuro3.php?country.... It still has negative numbers on it, and this only contains players who have attended a tournament (though it's more common for beginners to play tournaments in the west, since it's hard to find times to play).

It's not a comparison of the worth of the games (I play both, though I'm better at Go, and prefer it), but the dynamic range of Go is larger.

That said, any cross-game/sport comparisons of this kind are pretty tough to do properly.

I'm unfamiliar with Go ratings, but chess ratings are based on the Elo system which is a simple mathematical prediction system. Borrowing some figures from Wiki [1] we get:

The second column is your rating minus your opponent's, and the left is your predicted result. So if you are rating 1849 and your opponent is rated 1700 then you'd be expected to score about 70%. To have a 1% expected score against Magnus, you'd need a rating of about 2150.

[1] - https://en.wikipedia.org/wiki/Elo_rating_system

I was under the impression that Go also uses Elo, then I did a bit of cursory research and discovers that it varies.

Two major federations are American Go Association (AGA) and European Go Federation (EGF). EGF uses an Elo-inspired update rule since 2021. AGA uses a quite-different Bayesian system without pairwise update; they provide a paper and a C++ reference impl.

Asian countries don't bother with such numeric ratings. Instead, rankings are titles which are won through tournament promotion structures (sounds similar to Sumo to me).

Interesting, because I always thought that it was more "apples to apples", and that the higher upper limits of Go rankings was somehow indicative of the higher "dynamic range" of the game compared to chess. For example, if Elo were applied to basketball, what would the Elo of Lebron James be compared to a playground hooper (leaving aside that 1-on-1 isn't the best part of Lebron's game)... would it be higher or lower than Magnus Carlsen in chess? I don't have an intuition.

While it's not official, GoRatings (https://www.goratings.org/en/) ranks all players based on a variation of the Elo system (note that it is international, even if in practice you'd have to scroll quite far to find a few lonely Western persons).

AGA and EGF are very minor federations in the Go world, all the professionals are in Asia where the game is far bigger.

It works the same in go. Just, as the parent has said, the dynamic range of go is higher.

This is a bit of a digression, but: It's an interesting question what (if anything) that larger dynamic range means.

One thing you'll hear people say sometimes -- I've said it myself -- is that this shows that in some sense go is a "deeper" game than chess; there's more to know and understand, more variety of possible human skill.

That might well be true. It certainly feels a more elegant game, and involves longer tactical sequences, and so forth. But this may be misleading.

Consider the game of treblechess. To play a game of treblechess, you play three games of ordinary chess and look at the overall result.

Suppose that when we play ordinary chess, I win with probability W, lose with probability L, and draw with probability D = 1-(W+L). And suppose separate games are independent of one another (which might not be true in reality, but never mind). What happens when we play treblechess?

I win 3/0 with probability W^3. I win 2.5/0.5 with probability 3W^2D, because that happens when I draw any one of our three games and win both of the other two. I win 2/1 with probability 3(W^2L+WD^2), because that happens when I win two and lose one or win one and draw two, and for each of those there are three choices for which game is which. So I win at treblechess with probability W^3 + 3(W^2(1-W)+WD^2).

I can draw by getting one each of WDL (probability 6WDL) or by drawing all three (probability D^3).

Suppose that when we play chess I win 40% of the time, draw 50% of the time, and lose 10% of the time. Then our Elo difference is about 107 points. In triplechess, I will win 65.2% of the time, draw 24.5% of the time, and lose 10.3% of the time. Our Elo difference is about 214 points.

If in ordinary chess I win 65% of the time, draw 25% of the time and lose 10% of the time -- about the same odds as for treblechess in the last example -- then our chess Elo difference is about 215 points. At treblechess I will win 84% of the time, draw 11% of the time, and lose 5% of the time, and our Elo difference will be about 375 points.

If in ordinary chess I win 15%, draw 75%, lose 10%, then our Elo difference is about 17 points; in treblechess I will win 31%, draw 49%, lose 20% and our Elo difference will be about 41 points.

Treblechess Elo differences are on the order of double ordinary chess Elo differences! Clearly treblechess is a game with twice the depth of ordinary chess!

But it isn't. It's just longer and gives more opportunities for the better player to come out ahead overall.

Go is also a longer game than chess, though of course not in the same way as treblechess is. Perhaps the larger Elo range of go is more because of that than it is because of actual deeper strategy and tactics?

As an amateur (but competent) player of both, when I play go it almost feels like I'm playing multiple smaller games at the same time on the same board. Even if I might be struggling in one area, I can make it up in another. So it's very hard for a lesser opponent to beat me, even if they do gain an advantage in one area. Likewise in reverse when I'm playing a superior opponent. I might get in a nice kill, but next thing I know we're playing a different battle on the other side of the board and I get crushed.

So I think there's truth to what you're saying.

Maybe the depth of a game is related to whether it will scale. Go is played on different board sizes and still works.

If you make a Backgammon board bigger it would just be a slog and no real increase in tactical challenge.

Chess rules dictate a set size of board, which I guess has been refined over time.

If a Go board is made bigger or smaller it just adjusts the problem space, the rules and core of the game remain the same.

Perhaps interesting: Japanese chess, Shogi, is usually played on a 9x9 board, but there are many variants, which are played on bigger boards. Though games take longer and those variants are played rarely. It scales, but I think it's fair to state, that Go scales much better, due to its simplicity (simple != easy).

I’ve thought about this before and come to very similar conclusions. The ELO range between the best and worst players is meaningful, but you can’t just read off the depth of the game from it in the senses we most care about.

Depends on what chess rating you're talking about. Online at fast time controls, where many many more games get played at than OTB classical, the top super GMs are rated in the 3400-3600 range.

>Go grandmaster Shin defeats AI KataGo with a two-stone handicap

English is not my first language but for clarity perhaps the title should be:

Go grandmaster Shin with a two-stone handicap defeats AI KataGo

No. It’s not my first language too, but your version sounds odd. Usually you put an action first and only then the details

Trouble is, from the existing title it's not clear who had the handicap. So the poster is right to try and fix it.

It's not clear from the revised title either though. In fact I found it even more confusing. The issue is the ambiguity of the wording "with a two-stone handicap", not where it's located in the sentence.

I would probably phrase it as 'Go grandmaster Shin defeats two-stone-handicapped AI KataGo'

Are there ads on the page? If so, let’s A/B test that along with “Grandmaster Shin Savagely Beats Handicapped Opponent”.

Sir, have you considered a career in social networking engagement?

Edit: 10 years since i made the HN account and I only now notice you can't see the scores of other people's comments. Here's hoping the brilliance of the above comment is appreciated.

Made my day

I only saw the "revised title" and yes, it's still confusing :)

With a two-stone handicap, Go grandmaster Shin defeats AI KataGo.

Go grandmaster Shin, playing with a two-stone handicap, defeats AI KataGo.

The problem is that "with a handicap" is in itself ambiguous. People sometimes use it positively, sometimes negatively. That is, you say that Shin had a two-stone handicap, that can in itself be understood as either "Shin was given an advantage of two stones over the AI" (the reality) or "Shin was playing with a disadvantage, giving the AI two extra stones" (the more common reading of "to have a handicap").

Idiomatically a handicap is a disadvantage not an advantage. So if Shin had two extra stones, KataGo had a two stone handicap.

Handicaps can be positive, e.g. in fixed distance racing one way to handicap is to "give" a starting distance to the racers who are slower than the scratch racer; in this system a higher handicap is more of an advantage.

Since the AI was playing scratch (the normal, unmodified play style), it seems odd to say it had a handicap. Rather, the human had a positive handicap of two stones.

Youre just mixing up who has the handicap. The handicap gives starting distance to the slower racer, but its still the faster racer that has the handicap.

Yes, but the title has an ambiguous parse (a shift-reduce conflict):

X defeats (Y with Z)

(X defeats Y) with Z

with the latter usually understood as meaning X has Z, as far as I know.

fwiw, as a native speaker, your revised version sounds wrong. I'm terrible at grammer, but i think it's because when you put the "with" early in the sentence, it acts as a preposition modifying the noun, where later in the sentence it acts as an adverbial phrase modifying the verb defeat. The handicap is describing the manner of the defeat and not a trait of the grandmaster, so the with has to go at the end of the sentence.

Or that at least is my guess as to why it sounds wrong.

> I'm terrible at grammer

Also pretty bad at spelling.

lol, you are not wrong.

I don’t think so: the ELO rating difference is what matters, e.g. a 1000 player vs an 800 player has the same win probability as a 2800 vs a 2600.

You're right that Shin Jinseo is a generational talent, and more dominant than anyone since Lee Changho (peaked in the 90s and was strong into the early-mid 2000s).

However, you can't compare goratings over time, the top ranks are not nearly stable enough. https://www.goratings.org/en/history/ (I think it's believable Shin Jinseo is better than Lee Changho, but not that there has been steady progress since the days of Lee Changho, so that there are now 20 players stronger than him).

Could you just have superhuman Go AI just how good humans are somewhat more objectively?

Not without flaws of course, but probably interesting

The problem is ambient go knowledge. A top 100 player would easily beat time traveling Lee Changho in his first few matchups. Of course give peak Lee Changho a fortnight to prep with Katago and … well that would be something!

It's not obvious to me, but I lean towards saying this is false. The 2026 player would play a better AI inspired opening, but I'm not sure that would be enough to overcome the skill difference.

(This is especially true if they don't get briefed "this is time-traveling Lee Changho, he doesn't know contemporary joseki, play a trap variation").

Your point is pretty fair.

The reason I think the top 100 pro would win is that when I watch pros review they are very quick to catch when someone is playing a non-current or up to date style. I’ve also seen a few middle game situations where the classic famous go move is now “obviously” not good after we’ve seen how AI handles it.

This all adds up to many ways good sound moves to Lee Changho have now obvious responses. Do I believe he would adapt inhumanly fast? Absolutely! Legendary fighting spirit

You can't compare Elo ratings over long stretches of time, period.

Ratings drift over time, based on the total population of people competing. I think the most accurate way to view Elo ratings is as a measure of skill vs. the average rated player.

If you want to compare Magnus Carlsen's peak rating of 2882 in the year 2014 to Garry Kasparov's peak rating of 2851 in the year 1999, you have to know how strong the total pool of players (including all the amateurs who compete at lower levels) was in the year 2014 vs. 1999.

The only way to actually anchor the Elo system over long periods of time would be to have rated humans occasionally play against a set of unchanging computer players, which could then serve as static rating calibrators. You could use those games to then calibrate Elo ratings from different time periods to a common scale, by asserting that the computer ratings don't change.

Even that would be slightly malinformative because of opening theory. If you warped a very strong player from the past to the present, he'd do very poorly at first simply because of advances in opening theory. But give him a bit to catchup and he'd likely have his rating zoom on up. So modern players would do better against the static computer because of the same advantage, but that doesn't mean they're necessarily stronger in the sense that we hope to measure. The question people always want to know are things like how would a Morphy, Capablanca, or Alekhine do in modern times with access to modern theory and the like - not how well would they do against Carlsen if they went in with nothing but the knowledge of their era.

In Go, we had an extensive documentary on this, based on the strange case of a man possessed by the spirit of an ancient master Go player. I'm not going to spoil the results here; it's an enjoyable series and includes many small introductory tutorials to the game.

https://en.wikipedia.org/wiki/Hikaru_no_Go

> If you want to compare Magnus Carlsen's peak rating of 2882 in the year 2014 to Garry Kasparov's peak rating of 2851 in the year 1999, you have to know how strong the total pool of players (including all the amateurs who compete at lower levels) was in the year 2014 vs. 1999.

I imagine that FIDE has all of this data somewhere, right? They don't publish more in-depth distributional analyses of players somewhere?

The distribution of Elo ratings will not tell you how strong the average player is in an absolute sense.

Elo ratings measure differences in skill between different players. A player rated 400 points above another player will win 90% of the time. Only rating differences are meaningful. Absolute ratings aren't.

Yes, I can understand why absolute ratings are impossible to extract.

However, since the population maintains some continuity over time (players gradually enter and then leave over time), would it not be possible to reconstruct relative ratings between players that didn't play during the same era?

Only if you were to assume that a player's prowess remains constant throughout their career, which we generally know to be false. (I'm completely inventing dates here) If Fisher played Kasparov in 1990 and Kasparov played Carlsen in 2020, you can only compare Carlsen to Fisher if you assume Kasparov's skill was about the same for this entire duration, which no one believes to be the case.

A lot of people don't realize that chess players peak around the same age as athletes do. Carlsen was at peak dominance in his mid-20s. Kasparov peaked later, at 36 years old, but still, it wasn't at age 50 or 60.

Chess requires you to be really sharp. Experience increases with age, but there's some cross-over point at which the decline in calculating speed is more important than increasing experience. Heck, I'm measurably better at chess tactics in the morning after a good night's sleep than in the evening after work.

I'm sure it's also a matter of how much time you actually dedicate daily to this - and it's quite hard to maintain passion and focus for 8+h /day on the same game for 30-40 years.

> You can't compare Elo ratings over long stretches of time, period.

I agree that you cannot do it reliably, in principle.

However, Bobby Fischer peaked at 2785, and Gary Kasparov peaked at 2851. These are not far from what informed observers suggest--maybe 50 or 100 points off. They are well into super-grandmaster territory. Kasparov would be 1st today, Fischer would be 4th.

But on goratings.org, the top player of the 90s would be roughly 30th today.

My point is that the go ratings are much more unstable than the chess ratings. With chess ratings, you'll be wrong in the details. With go ratings, you'll be catastrophically wrong.

I agree that the drift in chess Elo ratings is slow enough that we can say Bobby Fischer at his peak would still be a strong grandmaster today (if he were given some time to study developments in opening theory). But I don't think we can say whether he would land at 2700 or 2800 in today's Elo scale.

This is within the realm of reasonable opinion, but I'd tentatively say it's an uncommon one--I thought it's generally agreed that the very best players of that era didn't lag contemporary players in terms of skill, only theory. I may be wrong, though.

Chess has developed significantly over the last 50 years. The best players now can be reasonably expected to be much more skilled than the best players of 50 years ago, even if you discount opening theory knowledge.

For another comparison, top world class chess players will have solid odds to beat Leela Chess Zero when given a knight odds handicap (Leela Chess Zero starts with 1 fewer knight). For human vs human, I think this would be somewhere in the ballpark of the ~10,000th best chess player having fair odds against Magnus.

I wonder if this means the best Go play is closer to theoretically perfect play or if it just happened the current computer methods didn't manage to get much farther than humans. Go has vastly more valid games but also a simpler ruleset, so I'm not sure if there is really a good way to tell beyond "keep trying and find out"?

I'm not sure if I'm allowed to name the source on this here, but someone very familiar with the top Go program scene explained that there is a fair chance that a perfect game has been played already. This is based on the probability of a mistake per move in relation to the number of games there were played by top programs. Hard to believe at first, but perchance it is true. The problem is, of course, that we don't know which games are perfect.

This was only running on 4x3090s man. The gap is far, far wider than you think. 8 5090s or 6000s and at least as many extra dedicated to doing nothing but running disgusting amounts of monte carlo in parallel from anything resembling a good choice (fuck it check some bad ones too) would lay him to waste.

I am going to go out on a limb and say part of it is a matter of respect, and part "lets not discourage and scare the shit out of anyone who knows even a little about Go"

A long time ago I ported a Go game to the iPhone for a job, the only way to control difficulty was to limit the amount of time it could spend monte carloing. On the iPhone 4, "Hard" would take 20-30 seconds a turn and drain your battery. I learned the game as I was making the app, got into single digits vs humans years after that. When the iPhone 5S came out it was capable of doing so many more iterations a second that even the easiest difficulty destroyed me until I spent enough time figuring out which moves created the worst spaces for it to search, which isn't really playing go anymore it's more like mining bitcoin by hand.

E: 4x3090 tops out at ~1.4k nnEvals/s. 4x5090 is 3.2x. 8xRTX 6000 something like 7x.

If I remember the numbers correctly, your setup seems to be wrong if you only get 1.4k e/s with 4x 3090. What network did you use?

I don't buy novideo trash, go ask the people running the benchmarks

Giving my guy the nickname Satoro Go-jo.

A small note - it's Elo scoring, not ELO. It's named after Arpad Elo, it's not an acronym of any kind.

The consequences of League of Legends. The brain trust over there use ELO and its caused everyone to think Elo is an acronym with the wrong pronunciation.

I don’t know that it’s that shocking, remember Go it’s not solved game, so the the limits of what’s really possible is not known in all cases.

For example, we don’t even know whether perfect White play can possibly overcome two correctly placed Black stones against perfect play.

In Go, there are exchanges of plays called "joseki". Professionals consider the outcome of joseki to be an equal result for both players. Most joseki are only a handful of moves, but some, such as the "flying knife" joseki have variations that continue for upwards of 50 moves. A traditional 19x19 go board has 361 intersections.

Shin's genius was to play out a complex variation of the flying knife joseki that was, in essence, a one-way path to reach an equal board position that occupied about 1/4 of the board. Due to the 2-stone handicap, the position favoured black with the game ~25% complete. KataGo could not have played any other way, where a human may have tried to foil the plan by introducing further complications.

What was truly incredible was how Shin held the advantage from that point on.

I have some questions as a chess player who barely even understands the rules of go. First these josekis sound like what in chess is called a forced tactical sequence. When you say Shin played out a long complex joseki, how does he do that? Does he have to read/calculate it out over the board(50 moves seems crazy to me unless the search tree is highly constrained by geometry/deduction/very few candidate moves, which does occasionally happen in chess endgames), or is the joseki more of a fixed sequence of moves which he's memorised, only needing to read to "punish" if the opponent diverges?

Second, if it is a fixed sequence, how position independent is it? In chess, tactical sequences end up depending on the entire board state to work when they get sufficiently long. I guess what I'm asking is, could a player with some capacity for stategic thinking recognise this idea and take steps to make the flying knife impossible?

I really should spend some more time learning go, it's such a fascinating game.

You can think of joseki as “local opening”. Like, in a vacuum, this is known by study / AI to be an even result for black and white. It’s just like a chess opening, there’s no calculation up to a certain point. And it doesn’t exist in the midgame, it’s not similar to forced sequences which exist in both games; it’s much more like choosing French closed vs open or gambit/gambit declined. The one thing is (and this is huge), since Go board is very big, existing stone formations on other parts of the board influence the value of joseki and make certain ones more advantageous for black or white. To my knowledge this doesn’t really exist in chess, because the opening is already the entire board.

However, when Shin executed the 50 move flying knife, the board was pretty much empty. So there is really no need for calculation, both Shin and the AI know it’s locally optimal. But getting to play a very long locally optimal sequence is good for the weaker player, so they have less “real” moves to lose EV on. Notably Shin probably can’t open with the flying knife in one corner past a certain point in the game, even if that corner were completely empty - the rest of the board positions would change the end values of the variants.

If the AI could know this, they might play a variant that ends 30 moves sooner but is 0.01 pts worse. Then they would have more time to mess Shin up through organic new moves (which the AI will be better at of course).

(disclaimer: only ranked 1 dan)

Sounds like the counter is a strategy Magnus Carlsen has been known for in chess: get out of theory, probably on a bit weaker foot than the other player, to then crush them on raw power.

Yes, exactly.

The feeling among the go players I know is that the match had been kind of a setup for the human to win.

It's an equivalent of playing in chess with a bishop handicap and a computer playing deterministic chess so that a human can prepare a forced sequence into a winning endgame.

Shin is an absolutely amazing player and not anyone could have achieved that, but nobody really feels like he won vs a computer team that tried its best.

I don't know absolutely anything about Go but your description of how josekis work- how their optimality makes their evolution deterministic and how they develop better on empty boards or break up against obstacles, makes me think about Conway's game of life. Cool.

Joseki are akin to book openings in chess, eg, we routinely see players going 20+ moves entirely from AI prep.

The trouble is that in chess, if one player deviates significantly that can often still mean they have a playable (perhaps slightly worse) position which has to either be memorised or understood on the spot, which can be really difficult. The main line is not the only playable line.

Is this also the case for Go's joseki? Or is any deviation easy to punish?

It is the same. If you are not a pro punishing deviations is quite difficult (but can occasionally be easy like I'm sure it can be in chess with hanging pieces). The most obvious difficult punish example is just finding unintuitive ways to capture or suppress their stones (analogy: multi move trick captures or getting a better position). But it can be more subtle, sometimes the answer to a deviation is just to not directly punish, but just accept that their shape is slightly inefficient. But if you know they deviated incorrectly, you can be tempted to try to attack hard to directly punish when the line doesn't exist (and then you overplay). Another example of a difficult punish is to just ignore their move and make progress on a different of the board, challenging them to prove their move was actually a threat. You can kind of do that in chess with parallel pawnstorms, but it is much more common in Go.

In odds chess bots, the bots would willingly take more disadvantageous positions which are more complicated--probably the bots in GO which are trained for odds do similar? Why does it not avoid such a joseki & play a worse response which it believes the human cannot read?

Go AIs tend to naturally be quite bad at playing handicap games, due to the horizon effect. To massively simplify, when the AI sees that there's a large score gap, it realizes that every move it plays has a very low/high win rate, so it basically picks one at random. The early AIs played lots of slack moves when they were ahead, often making small endgame mistakes but winning by half a point in the end.

To account for this, KataGo uses a "playout doubling factor". When the AI plays against itself to learn, the developers set one instance of the AI to use fewer playouts compared to the other one, but gives the weaker AI some handicap. This allows the AI with more playouts to learn that although it may be in a losing position, if it makes the board position chaotic enough, it may still win.

The flying knife is objectively an extremely complicated position, so the AI played it assuming that the opponent would be forced into a very complicated reading battle where they could make some mistakes. Unfortunately, Shin has memorized the flying knife joseki more thoroughly than any other human on the planet, so he could play exactly like a very strong AI. It would probably be possible to train an adversarial network specifically to beat players like Shin, but that would take a substantial amount of effort, and Shin is strong enough that it probably wouldn't make too much of a difference -- Shin won by 11.5 points in game 3 without a flying knife shenanigans, only losing 7 points of value throughout the entire game.

Oh wow, this is one of the best explanations on the thread. I guess the horizon problem is not such an issue for chess bots because reaching endgame is much shorter? And whole-piece handicaps probably snowball way faster than a potential go equivalent

NVM I learned about leelaXodds today

Because the computer team in this match for some reason didn't do it. It was like a match vs vanilla stockfish at fixed depth rather than leela chess odds.

The headline is a bit misleading, though perhaps not intentionally.

Shin took a 2-stone handicap from KataGo which means that Shin is the weaker of the two. But to give that more context, Shin is also the strongest human player to have ever lived in raw strength terms by a good margin, and is known as replicating AI move-for-move more closely than anyone else.

If they were to play even then there’s no chance any human could win (and pretty much all pros agree with that). Lee Sedol beating AlphaGo in game 4 of that series is largely considered the last time a human beat a modern AI in an even game, which is why it was so amazing.

RE the game, Katago was set to use the strongest available model and ran on a 3x 3090 GPU system, which is a lot for KataGo. 20 seconds might sound like a handicap, but that’s over 100,000 play out variations which is essentially infinite for modern KataGo models (anything over 10,000 is overkill).

Shin played well in all games, but his strategy was to avoid complexity. KataGo reads out complex fighting like an absolute monster, so Shin was trying to play very very solid and very very calm so as to not give KataGo an in.

The 2-stone handicap could be thought of as roughly 10-15 points of ‘buffer’. That’s massive in professional games, and that’s what Shin used to win. He played so overly solid that it sometimes cost a point or two, but it removed an opening for Katago to fight. He did this at the key opening and middle-game sections and never burnt through the full buffer of handicap points in the last two games. That’s why those games look kinda ‘boring’, it’s because Shin wanted them to be that way.

Also note that KataGo probably could have won if its ’variance’ was tuned higher (basically it taking risks). Standard KataGo won’t take risks, it just wins with brute force. For handicap games though you can tune its willingness to start fights higher to prevent people from just playing ultra solid (like Shin did).

Shin did an absolutely amazing job and he deserves all the recognition. Katago routinely beats professionals giving them 3-4 stones of handicap, so the win by Shin highlights to me how strong he is, but also just how well he understands how the AI ‘thinks’.

naive question--does " He did this at the key opening and middle-game sections and never burnt through the full buffer of handicap points in the last two games. " imply in retrospect he could have won with a smaller handicap? or was having the rest of that buffer in reserve guiding strategy?

Traditionally, handicaps are usually given in the form of extra stones played on the board. This is a coarser resolution than points, with one extra stone in the opening having a strategic value of about 13 points. So he probably couldn't have won if his handicap were a full stone smaller, but he may have won if it were a few points smaller.

KataGo isn't very good at exploiting weaker opponents. In chess people were convinced a grandmaster can never be beaten with a knight odds. It's just too easy to simplify the position and win. It was very easy (for a grandmaster) vs already super human Stockfish. It was still kinda easy (for a strong GM) vs 200+ ELO stronger NNUE Stockfish. And then someone made a net optimized for exploiting humans. Its games are amazing and it beats GMs with knight odds with ease. It's unreal how good it is at setting traps, playing lines that don't work in theory but the refutation is based on precise tactical sequence a few moves deep. Playing vs it feels like playing vs a spider that slowly weaves a net around you till you can't move anymore.

I predict the same thing is going to happen in Go once the engines catch up.

(You can play those chess bots on Lichess for free. Challenge LeelaQueenOdds or LeelaRookOdds if you are master level or stronger)

can the grandmasters still win with rook odds?

they can win with any odds. But, they can only barely win even with queen odds: https://lqo.leumon.com/

That's a bit misleading. I'm pretty sure grandmasters can win almost all games with queen odds at classical time controls

Sometimes, especially at long blitz/rapid but they usually lose. I am master level player (FM) and it took me almost 300 games of slow blitz to beat it with N odds and about 150 games to beat it with rook odds. I was beating it at Q odds (meaning more than 50%) but I suspect it might be even better now than it used to be.

Random data point: I'm rather weak (~1500 chess com) and I can take ~1/10 games off Queen odds starting with white (15+10) if i focus. It has some exploitable patterns still, I feel.

I follow some chess channels on YouTube (among them the usual suspects, Gotham, Chessbrah, Eric Rosen) but do you know of any video content that does game reviews of what you describe?

You need to add Chess-Network to your usual suspects.

Apparently 2 stones is a huge advantage. An estimate is that the computer is roughly 4-600 ELO stronger on an even match.

Also, the human played a strategy tailored to that huge initial advantage. He said that the AI did not handle this particularly well, and played high probability moves instead of trying to lure him into a mistake.

Also, even though this was the best Go engine, it was not running on a supercomputer, and had a relatively limited amount of time per move.

So, this was an important victory for a human, but not a sign that humans are now stronger than AIs at Go.

> Also, the human played a strategy tailored to that huge initial advantage. He said that the AI did not handle this particularly well, and played high probability moves instead of trying to lure him into a mistake.

Yeah, katago's training is not really focused at all on handicap games, because it's by nature learning from even games against similar-strength opponents.

It doesn't have specific training from playing in a way to exploit a weaker player. In a handicap game you have to give your opponent opportunities to fuck up if you want to play optimally.

If a move loses 0.0005 points if the opponent plays optimally, katago won't play it even if there's ~zero chance a weaker player would play it right.

There have been go AIs that tried to train more directly on uneven opponents, one called "sai" comes to mind, but katago has huge advantages otherwise and won out over the others (for very good reason, it's a great project).

While AlphaGo originally only had win rate as a metric, modern Go AIs have more knobs, including an evaluation of "complexity".

Just stating this off the top of my head so I could be misremembering, but I heard that the KataGo settings used were tweaked to favor complexity. This was most apparent in Game 1 which Shin Jinseo lost, where the AI had an unusual opening. However, the last game was quite plain leading me to wonder whether that setting was present in the last game (or at all).

Ah this is interesting. Essentially the idea is that the compute can try and move into positions that it can evaluate but humans might have trouble evaluating because of the board state's complexity?

By the way that is exactly what humans do when playing with white in high stone handicap games. They place their stones all around the board, start little fights everywhere and wait for the weaker player to misread or misevaluate something. Suddenly two or three fights merge in a one sided larger one and part of the handicap is gone.

That's the basic idea, yeah.

When playing white in handicap games, you want to make your opponent uncomfortable.

Play moves where the simple/safe/obvious move is just a little bit bad. Force them to choose between complex fights or a slow death of 100 slightly suboptimal moves.

It feels really wrong to defend like 10 times in a row, so if you make them do that they'll lash out at the wrong time and you can take advantage.

You also want to look for moves where...even if their best response means it's even or a little bit worse for you, there's ~reasonable responses where you win out or it goes complex.

A lot of the time it's not even crazy complex fights, it's more just situations where the judgement of what is more points is difficult.

(Note: most of this stops applying as strongly if it's a teaching game, which most handicap games are, there you have other considerations besides winning)

There are tweaks to move choice like that, but it's all evaluation time. None of that happens in training. Zero times in the RL loop does katago see a two stone game against a weaker player.

You can kind of tweak towards play this metric or that, but it's not the same.

Maybe not to the extent you mentioned earlier (I definitely agree that it's not "focused" on this) but there is definitely asymmetric playout training in the training data (see comments on lightvector/KataGo issues #39 and #162 mentioning it for example), that is presumably how they got the tweak for playoutDoublingAdvantage (i.e. actually having a few thousand of the millions upon millions of training games be games where playouts have been doubled for advantage).

Yeah, thanks, I forgot that existed. That definitely weakens my point quite a bit.

It's still not _quite_ the same thing because an advantage in playouts is not a great model for how a stronger/weaker player dynamic actually works, but it is something for sure, way more than the nothing that I said exists.

> So, this was an important victory for a human, but not a sign that humans are now stronger than AIs at Go.

Another way to look at this: Go's handicap system gives us a genuinely interesting metric for the distance between a human and a machine at this specific game. Instead of just "computers beat humans" we get a quantified gap.

The question of whether machines or humans are stronger is moot, isn't it?

In any intellectual contest between human and machine, all the machine winning implies is that the endeavor is algorithmic.

The machine can be given practically unlimited memory and compute; we consider it cheating if the human would use memory aids. The machine could be implemented as many agents cooperating; we'd think it's not right if thousands of humans collaborated to face the machine, etc.

So statements like "not a sign that humans are now stronger than AIs at Go" are pretty meaningless, IMO

> not a sign that humans are now stronger than AIs at Go

Totally wrongheaded, actually, since the computer gave the human a 2 stone advantage from the start.

I'm sorry, but Elo is not an acronym for Electronic Light Orchestra. You don't write ELO, but simply Elo.

I love this.

I watched every game years ago between Lee Sedol, even though it was late at night. I love AI and I love Go.

Off topic, but I wrote the first commercial Go program for the Apple II in the late 1970s.

Here's likely the first computer Go algorithm (in FORTRAN) and I believe the SGF is the first ever recorded human-computer game:

https://repo.autonoma.ca/repo/historical-computer-go

I should have said first commercial Go program because I actively marketed it. There were several previous Go programs.

The software industry is such a young field that we can still sit at the feet of the giants whose shoulders we stand on.

Deep Blue vs Garry Kasparov (1997) and AlphaGo vs Lee Sedol (2016) marked the first defeats of a world champion by a machine in a chess or Go match. Coincidentally, both machines played an iconic move on Move 37 of Game 2:

• Deep Blue played Be4, declining a pawn capture to achieve a long-term positional advantage and fueling Kasparov’s suspicions of human intervention.

• AlphaGo played P10, a brilliant 5th-line shoulder hit that live commentators initially dismissed as a blunder or misclick.

https://franky07724-57962.medium.com/amazing-coincidence-in-...

For those who don’t follow human vs AI Go (I don’t), this is with a 2 stone handicap in favor of the human which is apparently standard.

Yes, so calling it a "defeat" is improper to me.

The headline is misleading, but this is still huge. 2 stones against katago is insane, I'd never have guessed we'd see that, ever.

Annoyingly misleading. I read it that the human player was handicapped.

Like many random headlines they may be targeted at people who know something about a topic.

If you know anything about go or computer go from last 10 years then it is obvious which direction the handicap goes.

> "This series taught me that rather than trying to imitate AI, it is far more important to build the board according to my own style."

It's never made sense to me that so many go players study AI go play in the hope of emulating it in a human game. We're not machines. We can't do thousands of Monte Carlo tree searches per second.

They study it because it has done things that humans had long assumed were bad until AI proved otherwise. The conceptual knowledge has been valuable at the top level.

> It's never made sense to me that so many go players study AI go play in the hope of emulating it in a human game.

> We're not machines. We can't do thousands of Monte Carlo tree searches per second.

You sound so sure of that, but I have seen people catch a ball, and I am not so sure that any artificial person should be any more aware of the tremendous maths they are "solving"

From the perspective of a game with far fewer rules, "trying to imitate AI" might not mean anything like what you are thinking.

> but I have seen people catch a ball

from what i remember, dogs sometimes run in curves so that the perceived trajectory of the object they try to catch is more linear

so not everything might be in-brain math but also good trickery

What exactly do you think the difference is?

If you have an LLM add 2+2, it is doing a tremendous number of additions and multiplications it doesn't have to, same as us, and I am not sure the LLM can answer any more questions about that process than we can about ourselves.

At least when it comes to chess, Magnus Carlsen has stated multiple times that he's been inspired by AlphaZero and adjusted his own style of play after studying its games.

Last time I read about these handicapped games in Go I saw that the model isn't trained to play from behind, so while in a normal game of Go, those 50 move Joseki's or any large move combo that's an 'even trade' will inevitably let the AI claw a lead at some point, the AI is already behind.

It's similar in chess but there's a model specifically trained to play a knight down, and it's pretty cool to see the insane tactics it'll try. Though a knight down is considerably more than a 2 stone handicap in Go.

If anyone wants to know more about the chess model: https://lczero.org/blog/2024/12/the-leela-piece-odds-challen...

You can choose which piece it's missing. It's humbling to lose starting up a queen

I recommend "The Master of Go" by Kawabata, 1951. It's a beautiful story of the change of power at the apex of the Go ranking system. I read it the same year as I read "The Glass Bead Game" by Hesse published in 1943 and its hard not to believe a relationship but in truth I think they are completely independent stories of the same situational tension.

What I found interesting was that he adapted and shifted to a very unconventional strategy of play, opposed to the AI who primarily seems to play high probability moves. Does this not demonstrate the human edge against AI in novel / unconventional thinking?

> a very unconventional strategy

Correction: a very conservative strategy, so as not to lose the advantage he started with.

I don't know what the fascination with these AI versus human tournaments is. I'm old enough to remember the whole "Deep Blue" vs Kasparov exhibition, and I didn't really understand the fascination with all that either. That humans can make sufficiently strong calculators has never been a dispute in my mind. If the human wins over the calculator, good, but if the calculator wins, okay. What does that tell us exactly? The human had a bad day? The machine had a good one?

> That humans can make sufficiently strong calculators has never been a dispute in my mind.

Within our lifetimes (unless you're quite young) it was doubtful if a go ai would ever beat a decent human. Same was true for chess a generation or two earlier. It's news because it's the handoff of man to machine being the best at a particular thing.

This current news is news from the other way, this human did _exceptionally_ well.

>It's news because it's the handoff of man to machine being the best at a particular thing.

That's a way to see it. Another way to see it is that for any given event in a zero sum game, one of two outcomes, where two players are involved, will occur. Man only had to beat machines for what, 200-400 years only for it to beat man how many times for it to be "best"?

It's all narrative. Abstract strategy games with perfect information (like chess and go) look at a casual glance like they should be difficult for computers, but in fact they're rather ideal. Before AlphaGo, belligerent go players used to smugly dismiss chess as a 'solved game' (not what that means) because chess engines have been better than the best humans for years.

Surely we didn't think they were ideal before alphago--they were huge challenges back then no? Just that now we can solve them.

Kellin Pelrine already beat KataGo in 2022 (https://www.reddit.com/r/baduk/comments/13at6s5/how_many_bli...). Why is this different? Is it meaningful that Shin is also extremely good at beating professional humans?

That kind of blind spot is pretty brilliant but more of a technical hack from a Go perspective. Shin is world #1 and won “the normal way”, i.e. playing good moves

I'm glad that the best humans are still good enough at Go that holding these matches at all is still viable.

The article is hilariously triumphalist given the AI had a handicap (which it only mentions in passing unlike the title here).

Obviously its an impressive human intellectual feat but the hubris is instructive perhaps for our wider interactions with AI as its abilities accelerate around us.

Only two stone handicap against the machine with much deeper tree searching capabilities. This means that KataGo is lacking in strategy compared to Shin and he could leverage this to achieve this significant win.

Given all the interest in sudden interest in go in this thread, anyone have a sure-fire method to improve at go from double-digit kyu to single-digit kyu and amateur dan level?

At double-digit kyu level, you should just play every day until you feel you are stuck.

After you get stuck, you should pick one thing at a time to focus on improving. But the one thing should probably be related to tesuji or life and death.

Thanks!

One follow-up: Any recommended go servers to play on? OGS, Fox, Pandanet?

Probably Fox would be the server with the most players but at DDK basically any would work and I think getting exposure to different playing styles would be worthwhile.

OGS (online-go.com) is the predominant server for players in the Western hemisphere, and provides a modern browser based UI. Fox has a larger player base, but that player base is almost entirely in Asian timezones. Fox requires downloading a not so modern client whose UI is lacking English translation. I get the feeling that Fox has a lot of bot players in the lower ranks.

It can take a few minutes to get a match on OGS. When that happens, I take the opportunity to warm up with a few tsumego drills on goproblems.com

Binge Hikaru No Go and pay close attention to whatever Sai does. /s

Handicap Go by Nagahara Yoshiaki helped me tremendously.

I started playing I believe in March of 2025. I had the very good luck that one of my high school best friends is a 4 dan EGF player, and was in the French team 20 years ago. We played constantly against each other with 9 stones handicap and him teaching me a lot.

But when I read that book, in a few weeks/months we went from 9 stones to only 3 stones by August 2025. and we have stayed at 3 stones since, playing very even games, with only a handful or couple of points differences. Only very recently did I win by 9.5 points. Which made me hope I had made new progress, but I had to resign in the very next game.

Spaced repetition: https://onlymove.net/

I'm 2 dan.

For a double digit kyu there are two simple things than can give you 2-3 stones boost overnight if you understand them.

- don't play aji keshi.

- endgame starts earlier than you think. If nothing is urgent, play the largest endgame in the corner / side of the board.

"The 26-year-old South Korean grandmaster became the first human to win an official series against a state-of-the-art Go engine under a two-stone handicap, a margin considered the absolute boundary for human competition against modern AI."

What a powerful story. Humans have a chance of remaining superior because emotions are the fuel for our intellect and wisdom.

A strange take. Everything indicates that while the best human chess and Go players will continue to be incrementally better than all previous humans, computer players will improve much faster and leave all humans behind in very short order.

As with chess, roughly equal matches between computers and human players are probably happening for the very last few times.

Is there an equivalent of LeelaKnightOdds for Go? That might be harder to tackle.

This would be exactly that. The handicap stones in go allow it, it also works between humans of different strength levels. Now, if you mean a NN specifically trained to play at disadvantage, that would be extremely interesting and I hope they make it!

That's exactly what I meant . LeelaKnightOdds is specifically trained for that, and it makes a world of difference.

I read it wrong, I really hoped that he gave AI a handicap :)

Great, now next match against blue spot[0], starting even and adjusting the handicap each game.

Blue spot is an adversarial ai and it's managed to beat average professionals on five handicaps, which is absolutely insane.

[0]: https://codenamebluespot.com/

What does "on five handicaps" mean? The human received a 5-stone handicap?

Yes. More info here: https://codenamebluespot.com/open-test (including commentary of the games)

Neo...

Okay, now remove the 2-stone handicap

We're so back

I'm struggling to believe it's not good enough to beat a mere mortal.

This was with a two-stone advantage for the strongest-ever human player, apparently.

Thinking machines were finally defeated and will now be prohibited.

The fact that it's news that a human beat an AI, post-AlphaGo, demonstrates the current norm.

This is the bed all you AI hypers made. Now all manner of disparate tools fall under the banner of AI, with something like this that are not LLMs or transformer type tool, they happen to get lumped with all the rest of them.

Not sure what you're talking about. Playing games has been a research topic in AI for much longer than transformer models have existed.

For all four of you that are like me and understand Dota 2 a lot better than Go, and are wondering what impact a “two-stone handicap” has and what it means, ChatGPT Pro claims that to analogize this scenario to a professional team playing against OpenAI Five:

> The professional human team begins from a legal eight-to-ten-minute game state in which it has decisively won the laning stage: roughly a 6,000–8,000 team-net-worth lead, a 4,000–6,000 team-XP lead, two enemy Tier 1 towers destroyed, the third badly damaged, and all three friendly Tier 1 towers standing.

I don't get it, can you explain in StarCraft II?

It's like 1.5x zerglings rushing 20s earlier.

Reply on news.ycombinator.com