Hacker News Reader

Comment by ronaldx | Hacker News Reader

One additional issue for computers is that humans don't play the game tree out to its conclusion - humans recognise winning and losing positions rather than playing a much longer (perhaps infinite) prove-out game.

A naive game theory strategy runs into a disasterous problem - the game tree is extensively longer than the 200 moves that a human will play.

So, a computer has to recognise a winning position, in a heuristic way, as well as the best players do. But, even as a total beginner, it's easy to notice Go programs sometimes get the end-of-game scoring totally wrong.

Once you've overcome this first problem, you've reduced Go to "just" a 200-move game tree.

View on HN

I don't know what you're trying to say. Monte Carlo based programs play out entire games, and evaluate them by scoring the end of the game--no heuristics needed for the evaluation, only for the playouts. And yes, they can accurately score a completed game.

I think the above is trying to say that the percentage of total possible trees that a human would bother considering is much lower in Go than most games, like chess. So the computational resources needed to do Monte Carlo and get results competitive with human players are much greater.

But the poster is wrong in saying that this means computers need to use heuristics. Indeed, the recent breakthroughs in Go-playing programs come from removing heuristics that previously helped the programs prune the tree space and instead just using Monte Carlo to simulate all possibilities.

Actually, computers use relatively knowledge heavy playouts these days. http://lifein19x19.com/forum/viewtopic.php?f=18&t=9371.

There's a tricky balance, because incorporating too much knowledge/heuristic use can make the program miss good but surprising moves. Nonetheless, the trend seems to be heavy playouts.

Yes, the strategy for rollouts (the rest of the game after you leave the currently explored tree) are still an area of research. The successful programs usually involve some sort of local shape matching or other heuristic, sometimes trained using ML techniques.

One of the surprising results is that having stronger play between opponents in the rollout phase does not always equate with a successful search. Having a balanced strategy -- that is, one that does not introduce a bias for either player -- appears more important.

I'm trying to express why Go is a difficult problem to solve.

I'm saying this (scoring, play-out) has been a difficult problem, which was not solved even a small number of years ago, and is still not considered straightforward. Even if it is solved correctly now, that's only a start.

I would also note that Monte Carlo is by definition a heuristic method - it is statistical and not guaranteed to be optimal.

I would also note that Monte Carlo is by definition a heuristic method - it is statistical and not guaranteed to be optimal.

So is the endpoint evaluation in chess programs. This isn't a difference between Go and the other games. (In fact, pretty much nothing in your original post is)

Anyone who has a modest understanding of chess can codify very simple rules for identifying a winning endgame.

There are extensive wikipedia articles on what constitutes a winning endgame in chess, as the board simplifies towards the end of the game:

https://en.wikipedia.org/wiki/Endgame_tablebase says "All chess positions with up to 6 pieces have been solved"

So, I disagree. Even if you don't count this as a theoretical difference, it is a practical difference.

Anyone who has a modest understanding of chess can codify very simple rules for identifying a winning endgame.

The exact same thing can be done in go, scoring a finished game is trivial. The strength of the program simply isn't determined much by correctly identifying and scoring terminal positions, but the intermediate ones. And for both chess and go, one very much uses heuristics which are often wrong.

If the heuristics were never wrong, you wouldn't have to do the tree search part at all.

Endgame databases only have a tiny effect on the strength of chess programs (a common misunderstanding!) just because it's not very common for the game to be still "flippable" by the time they become relevant. In all the other positions, you need the heuristics.

The situation for checkers on the other hand, is very different. There the endgame databases were critical for solving the game, but due to mandatory capturing rules the search space reduces much faster.

Edit: Not sure why HN won't let me reply. But anyway: the error that you're both making is assuming that the positions in which humans stop and score are "endgame" or "finished" positions. That's not at all the case! A game is finished if there are no more legal moves besides filling one's own eyes. Counting at that point is trivial because all the life & death situations are "resolved". Monte Carlo programs play until those positions, not the ones where a human would stop the game.

(if you now bring up seki, you get half a cookie)

It seems you've studied this more than me, so perhaps I'm just misunderstanding you, but it sounds like you are equating a finished game with endgame. Scoring a finished game is obviously trivial, but even when a small number of positions remain open on a Go board, the game tree is still combinatorically vast, and I don't think there are any "simple rules" for determining optimal play.

As it turns out, Go endgames are surprisingly tractable with clever mathematics. See for example http://senseis.xmp.net/?MathematicalGo

I wouldn't say scoring is trivial -- it depends on both players agreeing which stones are alive or dead. Experienced Go players "just get it" but computers don't: http://www.andromeda.com/people/ddyer/go/scoring-games.html

Scoring is trivial - if the game is actually finished. No human wants to play go games to the finish so essentially all human games are scored by agreement. But a Monte Carlo program just simulates the game vs itself. See the addenum to my post above.

I don't know what you mean by "the game is actually finished".

AGA Rules: 9) Ending the Game: Two consecutive passes signal the end of the game.

New Zealand rules: The game is finished when both players agree that there are no more worthwhile moves.

Those rulesets weren't written by mathematicians. What do they say about life & death disputes while scoring "after the game has ended"? I'm going to guess: Resume the game.

Humans agreeing on a score and outcome has little to do with what a program has to do to calculate this score.

It's complicated. Most rulesets (AGA, New Zealand, I believe chinese) say "resume the game". The Japanese ruleset is a lot more complicated.

I still don't see your argument. Humans and humans are capable of agreeing on a score with those rulesets. Computers are also capable of following the ruleset and agreeing on the score. Computer vs. human games are not playing by a different ruleset. By that ruleset, the game is over when both sides pass.

Edit: Not sure why HN won't let me reply

I've also encountered this bug. Please bring it up with dang.

It's not a bug. It is a delay on replying to nested comments.

Click the [link] link to get a reply box quicker.

> Monte Carlo based programs play out entire games

Would that be pretty much impossible with Go on current hardware and software?

Not sure what you mean? That's exactly how go programs work today. They don't need to play out the games with an optimal strategy. Just an unbiased, sufficiently random one.

No, it's just 300 or so moves. It's an entire game, not an entire game-tree (all possible moves).

> humans recognise winning and losing positions rather than playing a much longer (perhaps infinite) prove-out game.

If true, does it mean a computer (or an Human) could win by just not conceding?

No.

"Conceding" means passing. If you don't pass, you have to play. If you play when the game is effectively over, you either play in your opponent's territory and get captured, or your own, which reduces your score. If you play in your own territory enough then you can actually end up losing your eyes, and then be captured.

I guess the remark comes from the "perhaps infinite" proof game. I don't know why they put "perhaps infinite" there as most Go rulesets have rules against repeating board positions.

A mathematically established lower bound on the longest possible go game is 10^10^48 [1]. While this isn't infinite, it's certainly large enough to be considered infinite in practice.

[1] http://en.wikipedia.org/wiki/Go_and_mathematics

Of course there are situations in mathematics where the tightest proven bound is 10^10^48 and the conjectured value is 7.

For what may or may not be an example, Graham's number is astonishingly huge, but math world (http://mathworld.wolfram.com/GrahamsNumber.html) claims the answer to problem it is an upper bound for may even be larger than 11:

"Graham and Rothschild (1971) also provided a lower limit by showing that N must be at least 6. More recently, Exoo (2003) has shown that N* must be at least 11 and provides experimental evidence suggesting that it is actually even larger."*

In this case, I think it is safe to claim that the answer is at least 361!/8, though (but that may already include many truly silly games with suicidal moves in the opening or games that continue way past the time experienced players think they are over)

Sure, but that number is a lower bound, not an upper bound.

Oh, wow, I misread that. So what's the protocol for dealing with trolls who refuse to concede?

At some point they fill in their territory and lose all of their stones. Eventually they will have no where to play.

A computer would only win by playing out pre-proved positions (joseki) against an amateur who has studied them positions without understanding them.

As for infinite games, they don't really happen much; in rulesets that make it possible for them to happen, the game is usually called "no result", and this has happened only very few times in hundreds of thousands of recorded games. Modern rulesets have "patched out" this by implementing superko or similar rules: playing a stone that would put the board in a state it was in previously (positions of the stones and which player's turn it is) is an illegal move.

No, a game is only over when both sides concede, and players concede when they recognize that any move they could make either does not change the score or would cost them a point (playing in territory that is already yours effectively reduces your territory as only empty territories are counted as points (in standard japanese rules)).

If one player would never concede you would keep playing until the board is full except for the unfillable eyes of the player that wins. Obviously that would take a long time and many negative moves of the losing player.

So you stop playing when both players are confident the game is over and agree on who has won.

That problem of "negative moves" is solved by the Chinese rules. A point is scored under them for each surrounded intersection plus each stone on the board. So you can defend any supposed weakness without losing points (you still lose sente and points if you needlessly defend when the score is not settled yet). You can even reduce your territories to two eyes each if you feel like.

By the way, the Chinese rules let solve neatly all those cases handled by special rules in the Japanese rules. The bent four in the corner is the most notable one. Playing it out with the Chinese rules is a neat explanation why that corner is defined to be dead: the surrounding player defends any weakness without losing points and starts the ko. The other player has no ko threats and dies. Those defensive moves lose points under the Japanese rules so they have to make a special rule for that shape and many others.

The only problem with the Chinese rules is that scoring takes longer and completely destroy the shape of the game: you fill in the territory of one player, take out the other's stones and count by grouping the stones in convenient shapes. Furthermore if you want to count during play you must remember how many stones have been captured because prisoners are returned to their bowl and are not stored in plain view (the score penalty is paid by not having those stoned on the board). Japanese rules are a shortcut that makes scoring easy but the tradeoff is the dictionary of special cases at the end of the game.

Elwyn Berlekamp, the first to study mathematical Go, says that 'What are the rules? is 'a dangerous question' (video: http://www.msri.org/realvideo/ln/msri/2000/gametheory/berlek..., start at 6:30).

For each of the (several) traditional rulesets there are positions that the ruleset can't deal with. So in order to allow a mathematical or computer analysis the rules often need to be tweaked.

Reply on news.ycombinator.com