Hacker News Reader

Comment by philh | Hacker News Reader

> The fact that AlphaGo beats someone beaten by Ke Jie 8 times out of 10 doesn't say enough about AlphaGo vs. Ke Jie.

8:2 implies the difference between them isn't so large. (If someone only beat me 8:2, I'd expect a professional to beat them 10:0.) Do we have any particular reason to think that AlphaGo would fall within that narrow skill band?

E.g. if Google specifically challenged Lee, thinking that AG had just about passed his level.

Alternatively by looking at the games and saying "Ke beat Lee by more than AG did, so Ke will beat AG as well". But many people have pointed out that AG doesn't optimize for winning by the greatest amount, so this probably isn't very informative.

Alternatively if AG's playstyle is strong against Lee but weak against Ke. But could Ke know that? AG's only played against two high-profile opponents, if it can adapt its playstyle we won't have seen that in action.

View on HN

A couple background info:

For people outside of Go community, Ke Jie is the new star of Go, currently Jie's Elo rating is #1, http://www.goratings.org/, Lee Sedol is #4. Media touted Lee Sedol as the greatest player of past decade, it was true couple years ago but Lee Sedol has been clearly in decline in recent years (as seen in the Elo rating).

2) Ke Jie is 19, at the peak age of his game. Lee Sedol is 33 and is seemingly in decline in his game, hence the 2:8 record vs. Ke Jie.

3) Ke Jie probably wanted to be challenged since he is the current world's #1 Go player. Sure there aren't much differences in terms of skill levels between top Pro 9 dan players but challenging a current world's #4 player is different from challenging a current world #1 player, isn't it?

It's a shame that the Western world doesn't know Ke Jie well(as his wikipedia page has very limited info and his award records.)[1] But these are moot points. After the 2nd defeat, it's increasingly clear that AlphaGo will defeat human player - it's a matter of time or a matter of a couple more games.

[1] https://en.wikipedia.org/wiki/Ke_Jie

Not so sure about the extra mile. It could happen that the AI couldn't be able to go beyond a certain level. That is this AI could be walled. We have a wonderful opportunity to see an example in which current AI is measured to be near but not far from best human competence.

> It could happen that the AI couldn't be able to go beyond a certain level.

That's your wishful thinking.

As we've seen the matches with Fan Hui in the past, unfortunately and fortunately, the CNN will improve drastically game after game and making less mistakes whereas human always do at a certain level.

Another big point I want to point out is human emotion and state of the mind. With 0:2, Lee Sedol lost his confidence ("speechless" in his own words) [1] and quite possibly increasing fear of losing in a landslide 0:5. Losing confidence and fear of losing are VERY TERRIBLE things in any competitive sports.

[1] http://www.theverge.com/2016/3/10/11191184/lee-sedol-alphago...

it is very interesting to measure at what rate AI improves. If the AI can improve a lot, a very interesting problem is to develop a scale or elo for measuring this progress. I am more interested in what lesson we can take home from this match, I don't have a wish here. We have many thinks to learn about this. In a situation of great uncertainty about the pace at with AI can improve its capabilities, measuring the velocity or rate of change can gives a hint about the future and applications of this new technology.

Agreed. Each 100 point of ELO diference translate into a 2 times stronger player.

The one to beat is Ke Jie because he is currently two times stronger than Lee Sedol.

I prefer the marketing narrative of beating a top pro then challenging the #1...or better yet have said #1 more or less ask for the challenge himself. It's more impact than going for the #1 directly and also allows for some extra optimization cycles.

Transitivity doesn't necessarily apply in most of these cases. Just because A can beat B and B can beat C, that doesn't mean A can beat C.

That's especially true when one of your players is a robot, which learns in a different way than humans. Yes, it's running a neural network, but a neural network still isn't a brain. Every person's brain is slightly differently organized on the neural level. On the other hand, every NN is running more or less the same structure and backpropagation algorithms. If you can find where those algorithms are strategically weak, you'll be able to beat them consistently.

In this case, we have A > C and it's looking like B > C. For these three players, transitivity holds whether A > B or B > A.

But yes, I spoke about playstyles. Do you think Ke can discover exploitable weaknesses in AG, given the games AG's played?

The actually values in the NN depends on both the algorithms and the data input during training phase. You cannot find how these algorithms are strategically weak unless you've seen all of what NN have seen.

Not necessarily. Unlike the human brain, NNs are extremely organized. They can change their weights but they cannot easily change their fundamental structure. Their organization can make them bad at certain tasks.

For example a CNN does well at object recognition from photos, but does poorly at recognizing a pencil sketch of an object it hasn't previously seen a sketch of. Or recognizing a scene at night that it has only seen daytime photos of. Those tasks, humans do very well because the trained human brain combines cultural understanding, physics intuition, and visual cortex at the same time, and would be able to use this to their advantage to easily beat, say, a massively trained CNN image recognition program that lacks cultural understanding and physics intuition.

Some other more complex NN structure may be able to tackle these kind of tasks, but as long as its neural structure is rigid, it will have yet other deficiencies. The human brain is still able to structurally adapt in ways that NNs cannot, yet.

"8:2 implies the difference between them isn't so large. (If someone only beat me 8:2, I'd expect a professional to beat them 10:0.)" Not sure I follow the logic here. They're both professionals, and one was much better than the other in the 10 games they played!

My point is that 8:2 isn't much better. Not compared to the range of skills available. If someone beats me 8:2, that puts fairly strong bounds on their skill level, such that most players would either beat both of us or lose to both of us.

Another way to look at it, if Ke beats Lee 8:2 then Lee-plus-four-stones probably beats Ke more than half the time. (I'm pretty confident about four stones; I suspect two or three would suffice.) Four stones is significant, but it's still a fairly narrow gap to say "I think AG falls within this skill range" until you've seen it lose a game.

I get your point, but four stones is beyond ridiculous. Two stones is already a really large handicap. At two stones, Ke would have no chance whatsoever.

Games at this level are decided by a few points. The closer two players come to optimal play, the harder it is to be very much more efficient than your opponent over the course of a game. To be so much more efficient, at this level, that you could overcome a 4 stone handicap is unthinkable.

I would very much like to see computers improve to the level that they could take on a top professional giving a 2 stone handicap. That would be a sight!

Ke Jie would lose to or at best be even with Lee Sedol with one stone handicap (taking white with no komi) because his ELO is not THAT much higher. Since Lee Sedol has a better record against Park Junghwan who also has wins vs. Ke Jie etc. ELO is a better predictor than just 10 games in a vacuum.

8:2 says a lot more when the players are the 2 best in the world (or close to that)

I agree with what you say, but disagree with what I think you mean.

Weaker players are more inconsistent. Someone else says Lee-plus-two stones beats Ke. I don't think me-plus-two-stones dominates someone who beats me 8:2.

So with weaker players, 8:2 is consistent with a wider range of skill gaps (measured in handicap stones). It's consistent with one player being only a little better, and it's consistent with one player being a lot better.

With two of the best players in the world, 8:2 isn't consistent with more than two stones difference between them.

So in this case, 8:2 rules out a lot of possible skill gaps, which does say a lot; in the same way that "beats me 8:2" says a lot more than "beats me 10:0".

(Alternatively we can measure skill gaps in points. Games between weak players will often be decided by tens of points. Games between strong players will be decided by a handful. 8:2 between Lee and Ke suggests their points gap is smaller than between two amateurs who get 8:2.)

Reply on news.ycombinator.com