Hacker News Reader

Comment by astrofinch | Hacker News Reader

There's also the fact that some of the unexpected moves were apparently more about solidifying against a loss than increasing the magnitude of a win. Which has its own kind of eerie implication: since AIs (like all computer programs) do what you say, not what you mean, the "intelligent species" can sometimes work really intelligently towards a goal that wasn't quite what you had in mind. (Gets especially interesting for any AlphaHuman/AlphaCEO/AlphaPresident successors that are given goals more complicated & nuanced than "maximize Go win probability regardless of ending score". BTW, if you haven't already read the Wait But Why series on the future of AI, I recommend it: http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...)

View on HN

About a year ago I wrote an AI to play the board game "Hive" (shares some similarities with chess). Because I scored all wins equally, it behaved almost exactly like this. It would simply try to minimize my advantage while always keeping open the possibility for it to win, almost like a cat toying with prey. It never actually would make the winning move – however obvious – until it had no other options!

I fixed this behavior by scoring earlier wins higher than later wins. Now it will actually finish games (and win), but almost invariably its edge is very small, no matter how well or poorly I play. Because of the new win scoring, it willingly sacrifices its own advantage if it means securing a win even one turn earlier. (And since scoring is symmetrical, this has the added advantage of working to delay any win it sees for me, thus increasing the possibility of me making a mistake!)

I suppose I could try modifying the scoring rules again, to weight them by positional advantage. A "show off" mode if you like :) And again, with the flip side of working to create the least humiliating losses for itself.

In go, the purpose is to have more territory than the opponent. There is no point in humiliating the opponent by having a big advantage. I think the aim of the strange moves was to increase the confidence of the program in its advance, not to increase the advance.

But you could indeed humiliate the opponent by actually capturing ALL of his stones. But that won't happen, if the enemy knows at least the basic concepts ... Still, if you play well, you cover much ground - while trying to supress the area of the enemy and even crushing him. But classic go is nice in a way, that it gives weaker opponents a start bonus of some stones - so the game is balanced and domination usually won't happen ...

My brother once played the (then) British Youth Go Champion on a 13x13 board, and lost by around 180 points - literally scoring worse than if he hadn't played at all.

> I think the aim of the strange moves was to increase the confidence of the program in its advance, not to increase the advance.

Absolutely. Also worth noting that it may be simply unable to distinguish between good and bad moves if both outcomes lead to a win, since it has no conception of the margin of victory being important.

So it might not be that it increased win probability, but that both paths led to 100% win probability and it started playing "stupidly" due to lacking a score-maximizing bias.

Sorry, I didn't mean the intent would be to humiliate, just the appearance.

Humans, I think, have the natural instinct to "hedge" themselves in games like go and chess, by creating positional/material advantages now to offset unknowns later. Of course, that advantage becomes useless in the end game, when all that matters is the binary win/lose.

An AI, which may have a deeper/broader view of the game tree than its human opponent (despite evaluating individual position strength in roughly the same manner), may see less of a need to "hedge" now, and instead spend moves creating more of a guaranteed advantage later (as you suggest). And indeed, my experience with my AI is that during the endgame (in which an AI generally knows with certainty the eventual outcome of each of its moves), it tends to retain the smallest advantage possible to win, preferring instead to spend moves to win sooner.

> Humans, I think, have the natural instinct to "hedge" themselves in games like go and chess, by creating positional/material advantages now to offset unknowns later. Of course, that advantage becomes useless in the end game, when all that matters is the binary win/lose.

That's actually an excellent way to win chess games. Keep your eye on the mate while the other person is focusing on position and material.

> It never actually would make the winning move – however obvious – until it had no other options!

I'm confused. Why would 'make the winning move' not be the way to maximise probability of winning?

That happens if the winning probability of the other move is 1 as well.

maybe he's defining a "winning move" as something with > 50% chance of winning

Usually it's because as it searches the move tree, it finds ways for the opponent to maximize their own winning probability and so has to hedge against that. In minimax games sometimes the evaluator finds a long chain of moves that leads to a win, and once it finds that, doesn't necessarily bother trying to find a shorter one. It can be frustrating to tune that out.

The AI is based on the minimax algorithm [1]. Because of the way Minimax works, the only way for a possible next move to be designated a "win" is if it is a guaranteed win. (The tree is (effectively) fully explored, and the opponent is given the benefit of the doubt in the face of incomplete information.) So, if there are multiple such winning moves, and care is not taken to distinguish the "magnitude" of the win, the AI will choose one arbitrarily.

I suppose that, in Hive, it is more likely that a path to a win is longer rather than shorter. Hence, when my AI was arbitrarily choosing "winning" moves, it statistically chose those that drew the game out.

[1] https://en.wikipedia.org/wiki/Minimax

But once you have guaranteed winning moves, why not pick the shortest one available (in terms of turns)?

Yes, that's what I did after I found the design flaw which effectively threw that information away.

Thank you for this.

Your post should be required reading in this discussion.

People forget how literal computers are.

> There's also the fact that some of the unexpected moves were apparently more about solidifying against a loss than increasing the magnitude of a win.

Humans play that way too. Everyone wants to maximize the chance of leading by >=1 stone. The difference is that AlphaGo is better at calculating a precise value of a position, so that when uncertainly plays in, AlphaGo can play for, say, "1-3 stone lead", while a human can only get confidence in "1-7 stone lead", and thus needs to play excessively aggressively to overcome the uncertainty.

> the "intelligent species" can sometimes work really intelligently towards a goal that wasn't quite what you had in mind.

That's called programming

Right. Skynet and Terminator are science fiction, but the slippery, unpredictable reality of how computers actually behave is right in front of your eyes as a programmer every day. Sometimes I wonder if science fiction writers do more harm than good: once they make a movie about some possible future, people feel free to dismiss it as "just science fiction", even if they have easily available empirical evidence that something vaguely like the scenarios described actually kinda has the potential to occur.

Not unlike the Simpsons episode where the military school graduation speech tells them the wars of the future will be fought with robots and that their jobs will be to maintain those robots.

thats .. unlikely. this could only happen if two wealthy and highly developed nations nations want to make a spectacle out of a war.

if you have fully autonomous robots which can fight your war, you'd be able to launch a massive offensive within hours. properly mobilizing defenses and responding to that invasion would take too long, as any command centers would've already been wiped out by the first attack.

I wasn't saying it will literally happen exactly as a Simpsons episode predicted, just that it is interestingly relevant for joke from 20 years ago.

Reply on news.ycombinator.com