AlphaGo Beats Lee Sedol in Final Game | Hacker News Reader

AlphaGo Beats Lee Sedol in Final Game

gogameguru.com

708 pointsdoppp290 comments

This was probably the closest game in the series. Livestream: https://www.youtube.com/watch?v=mzpW10DPHeQ

A few months back, the expert consensus was that we were many years away from an AI playing Go at the 9-dan level. Now it seems that we've already surpassed that point. What this underscores, if anything, is the accelerating pace of technological growth, for better or for worse.

In game four, we saw Lee Sedol make a brilliant play, and AlphaGo make a critical mistake (typical of monte carlo-trained algorithms) following it. There's no doubt that with further refinement, we'll soon see AI play Go at a level well beyond human: games one through three already featured extraordinarily strong (and innovative) play on part of AlphaGo.

Previous Discussions:

Game 4: https://news.ycombinator.com/item?id=11276798

Game 3: https://news.ycombinator.com/item?id=11271816

Game 2: https://news.ycombinator.com/item?id=11257928

Game 1: https://news.ycombinator.com/item?id=11250871

> What this underscores, if anything, is the accelerating pace of technological growth, for better or for worse.

While growth may be accelerating, this is simply the result of one big paradigm shift in deep learning/NNs. Once we've learned to milk it for all its worth, we'll have to wait for the next epiphany.

Milking neural networks out completely is pretty much AI as depicted in the movies. If we can milk it completely there probably isn't a need for the next epiphany.

Or, at the very least, the next epiphany need not be human-designed. Just train a neural network in the art of creating AI paradigms and implementations that can do general purpose AI. Once that's "milked", the era of human technological evolution is finished.

That sounds like a bad idea.

It's the core idea of AI, the primary reason why it is suspected that developing strong AI will inevitably lead to the end of the human era of evolution.

I don't want to be mean, but that's like saying you'll train a magic neural net with the mystical flavour of unicorn tears and then the era of making rainbows out of them will be finished. Or something.

I mean, come on- "the art of creating AI paradigms"? What is that even? You're going to find data on this, where, and train on it, how, exactly?

Sorry to take this out on you but the level of hand-waving and magical thinking is reaching critical mass lately, and it's starting to obscure the significance of the AlphaGo achievement.

Edit: not to mention, the crazy hype surrounding ANNs in the popular press (not least because it's the subject of SF stories, like someone notes above) risks killing nascent ideas and technologies that may well have the potential to be the next big breakthrough. If we end up to the point where everyone thinks all our AI problems are solved, if we just throw a few more neural layers to them, then we're in trouble. Hint: because they're not.

I totally see your point and my purpose is definitely not to be alarmist and sound the alarm that skynet is about to come out of AlphaGo or some other equivalent neural net. But I think the opposite attitude is also false.

As others have pointed out, we don't really know how the brain works. Neural nets represent one of our best attempts to model brains. Whether or not it's good enough to create real intelligence is completely unknown. Maybe it is, maybe it's not.

Intelligence appears to be an emergent property and we don't know the circumstances under which it emerges. It could come out of a neural network. Or maybe it could not. The only way we'll find out is by trying to make it happen.

Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must.

This is Hacker News, not a mass newspaper, so I think we can take the more nuanced and complex view here.

We don't really know how AI works either. NNs (for example) do stuff, and sometimes it's hard to see why.

>Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must.

Not really. Right now it's taking the position that there is no practical path that anyone can imagine from a go-bot, which is working in a very restricted problem space, to a magical self-improving AI-squared god-bot, which would be working in a problem space with a completely unknown shape, boundaries, and inner properties.

Meta-AI isn't even a thing yet. There are some obvious things that could be tried - like trying to evolve a god-bot out of a gigantic pre-Cambrian soup of micro-bots where each bot is a variation on one of the many possible AI implementations - but at the moment basic AI is too resource intensive to make those kinds of experiments a possibility.

And there's no guarantee anything we can think of today will work.

>> Neural nets represent one of our best attempts to model brains.

See now that's one of the misconceptions. ANNs are not modelled on the brain, not anymore and not ever since the poor single-layer Perceptron which itself was modelled after an early model of neuronal activation. What ANNs really are is algorithms for optimising systems of functions. And that includes things like Support Vector Machines and Radial Basis Function networks that don't even fit in the usual multi-layer network diagram particularly well.

It's unfortunate that this sort of language and imagery is still used abundantly, by people who should know better no less, but I guess "it's an artificial brain" sounds more magical than "it's function optimisation". You shouldn't let it mislead you though.

>> Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must.

I don't agree. It's a subject that's informed by a solid understanding of the fundamental concepts - function optimisation, again. There's uncertainty because there's theoretical limits that are hard to test, frex the fact that multi-layer perceptrons with three neural layers can learn any function given a sufficient number of inputs, or on the opposite side, that non-finite languages are _not_ learnable in the limit (not ANN-specific but limiting what any algorithm can learn) etc. But the arguments on either side are, well, arguments. Nobody is being "blind". People defend their ideas, is all.

Convolutional neural nets are the most accurate model of the ventral stream, numerically speaking. See work by Yamins, DiCarlo etc.

You're basically saying that there's no task (including passing the Turing test, programming web apps, etc.) which requires intelligence and is best tackled with either something else than a neural network or with NN combined with something else. I think it's a pretty bold statement which is really hard to back up by anything but a hunch.

Our current assertion is that neural networks basically replicate the brain's function, so our current understanding of this paradigm is that "milking neural networks" is going to match or exceed human general purpose intelligence.

I believe hmate9 is correct. If this paradigm is exploited to the full, unless we've missed something fundamental about how the brain works, we don't need to bother ourselves with inventing the next paradigm (of which there will no doubt be many), because one of the results of the current paradigm will be either an AGI (Artificial General Intelligence) that runs faster and better than human intelligence, or, more likely, an ASI (Artificial Super Intelligence). Either of those is more capable than we are for the purpose of inventing the next paradigm.

No deep learning researcher believes neural networks "basically replicate" the brain's function. Neural nets do a ton of things brains don't do (nobody believes the brain is doing stochastic gradient descent on a million data points in mini-batches). Brains also do a billion things that neural nets don't do. I've never even taken a neuroscience class, and I can think of the following: synaptic gaps, neurotransmitters, the concept of time, theta oscillations, all or nothing action potentials, Schwann cells.

You have missed something fundamental about how the brain works. Namely, neuroscientists don't really know how it works. Neuroscientists do not fully understand how neurons in our brain learn.

According to Andrew Ng (https://www.quora.com/What-does-Andrew-Ng-think-about-Deep-L...):

"Because we fundamentally don't know how the brain works, attempts to blindly replicate what little we know in a computer also has not resulted in particularly useful AI systems. Instead, the most effective deep learning work today has made its progress by drawing from CS and engineering principles and at most a touch of biological inspiration, rather than try to blindly copy biology.

Concretely, if you hear someone say "The brain does X. My system also does X. Thus we're on a path to building the brain," my advice is to run away!"

You are right, we do not know everything about the brain. Not even close. But neural networks are modelled on what we do know of the brain. And "milking" neural networks completely means we have created an artificial brain.

Did you just ignore the first few lines of argonaut's comment?

Recently, we also introduced activation functions in our neural nets, like rectified linear and maxout just for their nice mathematical properties without any regards to biological plausibility. And they do work better than what we had before.

"unless we've missed something fundamental about how the brain works"

But we don't know how the brain works. I think you extrapolate too far. Just because a machine learning technique is inspired by our squishy connectome it does not mean it's anything like it.

I'm willing to bet there are isomorphisms of dynamics between an organic brain and a neural net programmed on silicon but as far as I know, there are still none found - or at least none are named specifically (please correct me).

> Our current assertion is that neural networks basically replicate the brain's function

come on, that's hyperbole

   Our current assertion is that neural networks basically replicate the brain's function

No. Just, no. This was never really a claim made by people who understood neural nets (there was a little perceptron confusion in the 60s iirc).

Could you care to explain what was that big paradigm shift ?

This four-part series explains it well: http://www.andreykurenkov.com/writing/a-brief-history-of-neu...

But that's what technological growth is. A series of epiphanies, building on what came before.

Yes, but an epiphany is not evidence of an accelerating rate of epiphanies, nor evidence that such epiphanies will continue apace into the future.

You can look at the past for that, although obviously it doesn't predict the future. But it ought to be a priori obvious, at least, that the more you know (as a species), the more surface area of knowledge you have to synthesize into an extending step beyond the known.

You could look at the past, but that isn't what the claim did.

In fact looking at the rate of change in applications over an "epiphany" period is probably the least useful estimate of progress & rate of change in progress.

> In game four, we saw Lee Sedol make a brilliant play, and AlphaGo make a critical mistake (typical of monte carlo-trained algorithms) following it.

Can you explain why this is typical? What can be done against this to strengthen the algorithm?

I can't remember where I read this, but one theory was that the move Lee Sedol made was thought to be unlikely by AlphaGo, and so didn't explore down that path.

When Lee Sedol made the move, the AI was in unknown territory as it hadn't explored down that avenue.

> When Lee Sedol made the move, the AI was in unknown territory as it hadn't explored down that avenue.

Sounds similar to what a human would do then: you wouldn't spend much time simulating in your head what would happen if your opponent made a very atypical move or a move that would seem very bad at first thought.

That's exactly it. The difference, as far as I have understood it, is that there was a similar move that is typical, but in that particular situation, pretty simple reasoning (of the highly abstract "if this then that so this must lead to that" sense) leads a human to conclude that this version of the move is superior.

So while atypical in the sense of "occurring infrequently", it was not a difficult move to find for a player of that level – all the pro commentators saw it pretty much right away.

This might be the one weakness of AlphaGo, which is interesting.

David Silver said at the beginning of the broadcast of game 5 that AlphaGo's policy network had given Lee Sedol's move 78 only a 1 in 10,000 chance of occurring.

It seems that AlphaGo needs better time management skills. Not sure how that can be added. Michael Redmond mentioned that if a human player sees an unexpected move, he/she would just take all the time needed to read out the moves. AlphaGo seems to make speedy decision even after unexpected moves.

Yes, that's how modern chess engines manage time. If the score suddenly, drastically changes during search, they give themselves much more time.

In all of these games, AlphaGo used close to a constant amount of time per move, while Lee's varied a lot.

Apparently they only recently added a neural net for time management. Seems it is either not the best approach, or just not yet well trained.

> There's no doubt that with further refinement, we'll soon see AI play Go at a level well beyond human

Will we though? AlphaGo trains on human games, so can it go well beyond that level? Will it train on its own games?

It is already mainly training by playing against itself:

https://googleblog.blogspot.se/2016/01/alphago-machine-learn...

> To do this, AlphaGo learned to discover new strategies for itself, by playing thousands of games between its neural networks, and adjusting the connections using a trial-and-error process known as reinforcement learning.

It's still based on human games. It plays itself but the way it plays was inherited from human. I wonder if there is some fundamental barrier to what you can reach with reinforcement depending on your base.

It is based on human games until it can explore well enough to sufficiently break away from local optimums.

Having it learn on human games was just a way of speeding up the initialization process before running reinforcement learning, it didn't limit the state tree that was being searched later on.

It already went beyond human level, look for Go players commenting on the game, citing that they would have never thought about steps that the AI made. In a sense it brought new strategies to the table that humans can learn and apply in human vs human games.

Yes, but how far can it go beyond human level? Will it be slight margin, so it can win 4-1, or it will soon became able to beat top players with 1,2,10 stones handicap?

Some high level pros have stated that they would need a 4 stone handicap to beat the "perfect player", i,e "God of Go", so that would probably put a skill ceiling on this.

AlphaGo was actually only trained on publicly available amateur (that is, strong amateur) games. After that, AlphaGo was trained by running a huge number of games against itself (reinforcement learning).

A priori, this makes sense: you don't need to train on humans to get a better understanding of the game tree. (See any number of other AIs that have learned to play games from scratch, given nothing but an optimization function.)

Yes, but is it known if there's some limit to what you can reach doing this? I mean, if they trained it on games of bad amateur players instead of good, and then played itself, will it keep improving continuously to the current level or hit some barrier?

That's why they only initially trained it on human players, and afterwards, they trained it on itself. I would guess (strongly emphasize: guess) that they trained it on humans just to set initial parameters and to give it an overview of the structure and common techniques. It would've probably been possible to train AlphaGo on itself from scratch, but it would've taken much longer -- amateur play provides a useful shortcut.

I don't think there is a theoretical upper limit on this kind of learning. If you do it sufficiently broadly, you will continuously improve your model over time. I suppose it depends to what extent you're willing to explicitly explore the game tree itself.

There is always a risk of getting stuck in a local maxima, thinking you've found an optimal way of playing, so you'd need more data that presents different strategies, I'd think.

>A few months back, the expert consensus was that we were many years away from an AI playing Go at the 9-dan level.

These kinds of predictions are almost always useless. You can always find people who say it'll take n years before x happens, but no one can predict which approaches will work, and how much improvement they'll confer.

> What this underscores, if anything, is the accelerating pace of technological growth, for better or for worse.

What? This is a non-sequitur. Continued advancement doesn't mean that it is accelerating, and even if this does represent an unexpected achievement that doesn't mean that future development will maintain that pace.

Appreciate it for what it is - an historic achievement for AI & ML - and stop trying to attach broader significance to it.

> These kinds of predictions are almost always useless.

Let's rephrase. For a long time, the expert consensus regarding Go was that it was extremely difficult to write strongly-performing AI for. From the AlphaGo Paper: Go presents "difficult decision-making tasks; an intractable search space; and an optimal solution so complex it appears infeasible to directly approximate using a policy or value function."

For many years, the state-of-the-art Go AI stagnated or grew very slowly, reaching at most the amateur dan level. AlphaGo presents a huge and surprising leap.

> Continued advancement doesn't mean that it is accelerating

Over constant time increases, AI is tackling problems that appear exponentially more difficult. In particular, see Checkers (early '90s) vs Chess ('97) vs Go ('16). The human advantage has generally been understood to be the breadth of the game tree, nearly equivalent to the complexity of the game.

If we let x be the maximum complexity of a task at which AI performs as well as a human, then I would argue that x has been growing at an accelerating pace over the past few decades.

> If we let x be the maximum complexity of a task at which AI performs as well as a human, then I would argue that x has been growing at an accelerating pace over the past few decades.

At ONE task, yes. But humans are average at many things but excel at being able to adapt to many different tasks, all the time. Typical AIs (as we know them now) cannot ever hope to replicate that.

This seems to have been linked to a lot recently but I feel it is relevant to the discussion on technology advances pertaining to AI.

http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...

"and an optimal solution so complex it appears infeasible to directly approximate using a policy or value function."

To be clear, the above refers to specific concepts in Reinforcement Learning.

A policy is a function from state (in Go, where all the stones are) to action (where to place the next stone). I agree that it is unlikely to have an effective policy function. At least one that is calculated efficiently (no tree search)... otherwise its not what a Reinforcement Learning researcher typically calls a policy function.

A value function is is a function from state to numerical "goodness", and is more or less one step removed from a policy function: you can choose the action that takes you to the state with the highest value. It has the same representational problems found there.

> AI is tackling problems that appear exponentially more difficult.

The hardest AI problems are the ones that involve multiple disciplines in deep ways. Here's a top tier artificial intelligence problem: given a plain English description of a computer program, implement it in source code.

There might be some cases where this is possible, and some cases are bound to fail.

Those are the kind of difficult problems in AI, which combine knowledge, understanding, thought, intuition, inspiration, and perspiration - or demand invention. We would be lucky to make linear progress in this area let alone exponential growth.

I think there's certainly an impression of exponential progress in AI in popular culture, but the search space is greater than factorial in size, and I think hackers should know that.

Wait what is the plan to brute force go? The search space is beyond immense...

> To be fair, in terms of the complexity of rules, checkers is easier to understand than go which is easier to understand than chess. And honestly, go seems like the kind of brute-force simple, parallel problem that we can solve now without too much programming effort

Your intuition is mistaken. Go is indeed "easier to understand" than Chess in terms of its rules, but it is arguably harder to play well and has a way larger search space, which makes it less amenable to brute force, and this was precisely why people thought it'd be impossible for a computer to play it consistently at champion level.

I don't think the achievement of AlphaGo is solely due to increased processing power, otherwise why did people think Go was such a hard problem?

Sure.

The problem with Go was evaluating leaf nodes. Sure, you could quickly innumerate every possible position 6 moves out, but accurately deciding if a position 1 is better than position 2-2 billion is a really hard problem.

In that respect chess is a much simpler problem as you remove material from the board, prefer some locations over others etc. Where go is generally going to have the same number of pieces on each board and it's all about balancing local and board wide gains.

> it is arguably harder to play well and has a way larger search space, which makes it less amenable to brute force, and this was precisely why people thought it'd be impossible for a computer to play it consistently at champion level.

Are human champions not subject to those same difficulties of the game, though? When you're pitting the AI against another player who's also held back by the large branching factor of the search tree, then how relevant really is that branching factor anyway in the grand scheme of things? A lot of people talk about Go's search space as if human players magically aren't affected by it too. And the goal here was merely to outplay a human, not to find the perfect solution to the game in general.

(These are honest questions -- I am not an AI researcher of any kind.)

This is a great question!

Sadly, I'm neither an AI researcher nor a Go player; I think I've played less than 10 games. I don't know if we truly understand how great Go players play. About 10 years ago, when I was interested in Go computer players, I read a paper (I can't remember the title, unfortunately) that claimed that the greatest Go players cannot explain why they play the way the do, and frequently mention their use of intuition. If this is true, then we don't know how a human plays. Maybe there is a different thought process which doesn't involve backtracking a tree.

Go players rely heavily on pattern recognition and heuristics, something we know humans to be exceptionally good at.

For example, go players habitually think in terms of "shape"[1]. Good shape is neither too dense (inefficiently surrounding territory) or too loose (making the stones vulnerable to capture). Strong players intuitively see good shape without conscious effort.

Go players will often talk about "counting" a position[2] - consciously counting stones and spaces to estimate the score or the general strength of a position. This is in contrast to their usual mode of thinking, which is much less quantitative.

Go is often taught using proverbs[3], which are essentially heuristics. Phrases like "An eye of six points in a rectangle is alive" or "On the second line eight stones live but six stones die" are commonplace. They are very useful in developing the intuition of a player.

As I understand it, the search space is largely irrelevant to human players because they rarely perform anything that approximates a tree search. Playing out imaginary moves ("reading", in the go vernacular) is generally used sparingly in difficult positions or to confirm a decision arrived at by intuition.

Go is the board game that most closely maps to the human side of Moravec's paradox[4], because calculation has such low value. AlphaGo uses some very clever algorithms to minimise the search space, but it also relies on 4-5 orders of magnitude more computer power than Deep Blue.

  [1] https://en.wikipedia.org/wiki/Shape_(Go)
  [2] http://senseis.xmp.net/?Counting
  [3] https://en.wikipedia.org/wiki/Go_proverb
  [4] https://en.wikipedia.org/wiki/Moravec%27s_paradox

quoting https://news.ycombinator.com/item?id=10954918 :

> Go players activate the brain region of vision, and literally think by seeing the board state. A lot of Go study is seeing patterns and shapes... 4-point bend is life, or Ko in the corner, Crane Nest, Tiger Mouth, the Ladder... etc. etc.

> Go has probably been so hard for computers to "solve" not because Go is "harder" than Chess (it is... but I don't think that's the primary reason), but instead because humans brains are innately wired to be better at Go than at Chess. The vision-area of the human's brain is very large, and "hacking" the vision center of the brain to make it think about Go is very effective.

While I understand what you are getting at here, basically, this is still just a complete information game, and didn't solve AI. You are drastically understating the complexity of Go. It isn't actually possible to evaluate a significant fraction of the state tree in the early mid game because the branching factor is roughly 300. The major advance of AlphaGo is a reasonable state scoring function using deep nets.

Unless you have or are a PhD student in AI who has kept up with the current deep net literature I assure you that the whole of Alphago will be unintuitive to you. However, if you were an AI PhD student, you likely wouldn't be so dismissive about this achievement.

> The major advance of AlphaGo is a reasonable state scoring function using deep nets.

That and the policy network to prune the branching factor.

> Here's a top tier artificial intelligence problem: given a plain English description of a computer program, implement it in source code.

I would consider it a breakthrough if we could get human beings to do this at a decent rate :)

Even harder and more common problem -- given code, give a plain English description of what it is intended to do, and describe any shortcomings of the implementation.

Yeah e.g. you could get it to check whether it could go into an infinite loop.

Oh wait .... https://en.wikipedia.org/wiki/Halting_problem

You could for all practical purposes. The Halting problem only generally applies when you're considering all possible programs, but you really only need consider the well-written ones, because then you can filter out the poorly written ones.

Here's ia a top tier human intelligence problem: Given a requirement provide a accurate English description of a program.

You sound like the kinda person who says "AI will never drive," "AI will never play Go." True there's a lot of hype, which ML experts are concerned may lead to another burst & winter. On the flip-side there's a lot of curmudgeonly nay-sayers such as yourself at which ML experts roll their eyes and forge ahead. What I find is both extremes don't understand ML, they're just repeating their peers. ML is big, and it's gonna do big things. Not "only Go", not "take over the world"; somewhere in between.

I'm actually very optimistic about the state of AI and ML lately. The difference is that I don't anthropomorphize the machines or ascribe human values to their behavior. I absolutely believe AI will drive (and save lives); I have always believed that AI will play Go; I believe that AI will grow to match and surpass humans in many things we assume that only humans can do. Humans aren't perfect, but that doesn't mean that machines who outperform us are perfect either.

AlphaGo plays Go. It probably doesn't play Go like a human (because a human probably can't do what it does), but that's OK because it also appears to be better than humans. AlphaGo is interesting not because it has done something impossible, but because it has proven possible a few novel ideas that could find other interesting applications, and adds another notch to the belt of a few other tried and tested techniques.

These kind of statements are only valuable to me if they are followed by "And these are the challenges that need to be overcome which are being worked on".

Otherwise it's a blanket retort. It's like saying "There are lots of X".

Ok, name 7. If you get stuck after 2 or 3 you're full of it.

> Continued advancement doesn't mean that it is accelerating, and even if this does represent an unexpected achievement that doesn't mean that future development will maintain that pace.

Advancement faster than predictions does mean accelerating advancement, coupled with the (true) fact that people's predictions tend to assume a constant rate of advancement [citation needed]. Actually, all you'd need to show accelerating advancement is a trend of conservative predictions and the fact that these predictions assume a non-decreasing rate of advancement; if we're predicting accelerating advancement and still underestimating its rate, advancement must still be accelerating.

It even seems like this latter case is where we're at, since people who assume an accelerating rate of advancement see to assume that the rate is (loosely) quadratic. However, given that the rate of advancement tends to be based on the current level of advancement (a fair approximation, since so many advancements themselves help with research and development), we should expect it to be exponential. That's what exponential means.

However, the reality seems like it might be even faster than exponential. This is what the singularitarians think. When you plot humanity's advancements using whatever definition you like, look at the length of time between them to approximate rate, and then try to fit this rate to a regression, it tends to fit regressions with vertical asymptotes.

>>You can always find people who say it'll take n years before x happens

Interesting, people seem to be saying the same about self driving cars.

> These kinds of predictions are almost always useless. You can always find people who say it'll take n years before x happens, but no one can predict which approaches will work, and how much improvement they'll confer.

True, but it's pretty refreshing to have a prediction about AI being N years from something that is wrong in the OTHER direction.

Contrary to your point about 'appreciate it for what it is', there is ONE lesson I hope people take from it: You can't assume AI progression always remains in the future.

A general cycle I've seen repeated over and over:

* sci-fi/futurists make a bunch of predictions * some subset of those predictions are shown to be plausible * general society ignores those possibilities * an advancement happens with general societal implications * society freaks out

Whether it's cloning (ala Dolly the Sheep, where people demonstrated zero understanding of what genetic replication was e.g. a genetic clone isn't "you") or self-driving cars (After decades of laughing at the idea because "who would you sue?", suddenly society is scrambling to adjust because they never wanted to think past treating that question as academic), or everyone having an internet-connected phone in their pocket (see encryption wars...again), or the existence of a bunch of connected computers with a wealth of knowledge available, society has always done little to avoid knee-jerk reactions.

Now we have AI (still a long way off from AGI, granted) demonstrating not only can it do things we thought weren't going to happen soon (see: Siri/Echo/Cortana/etc), but breaking a major milestone sooner than most anyone thought. We've been told for a long time that because of typical technology patterns, we should expect that the jump from "wow" to "WOW!" will happen pretty quickly. We've got big thinkers warning of the complications/dangers of AI for a long time.

And to date, AI has only been a big joke to society, or the villain of B-grade movies. It'd be nice, if just once, society at least gave SOME thought to the implications a little in advance.

I don't know when an AGI will occur - years, decades, centuries - but I'm willing to bet it takes general society by surprise and causes a lot of people to freak out.

> > What this underscores, if anything, is the accelerating pace of technological growth, for better or for worse.

> What? This is a non-sequitur. Continued advancement doesn't mean that it is accelerating, and even if this does represent an unexpected achievement that doesn't mean that future development will maintain that pace.

It's not a non-sequitur, but there is an implicit assumption you perhaps missed. The assumption is that the human failure to predict this AI advance is caused by an evolution curve with order higher than linear. You see, humans are amazingly good at predicting linear change. We are actually quite good at predicting x² changes (frisbee catching). Higher than that, we are useless. Even at x², we fail in some scenarios (braking distance at unusual speeds, like 250km/h on the autobahn for example).

The fact that it will maintain its pace is an unfounded assumption. However, assuming that the pace will slow is as unfounded. All in all, I'd guess it is safest to assume tech will evolve as it has in the last 5000 years.

That would be an exponential evolution curve.

this reads as the professional go world couldn't wait for this (Alphago) to arrive to find new ways of play new moves.

There's no doubt that with further refinement, we'll soon see AI play Go at a level well beyond human

No doubt? Seriously? What kind of knowledge do you have to make such statements? There are plenty of examples where technology has rapidly advanced to some remarkable level, but then almost completely plateaued. For example, space travel or Tesla's work on applications of electromagnetism. Heck, even other areas of AI research.

I really don't see why people here readily assume that this particular approach to computers playing Go is easily improvable. Neither do I see why everyone assumes there will be no discoveries of anti-AI strategies that will work well against it.

With neural networks involved, it's hard to say. And all we have so far is information about about, what, 15 games? Some of which were won by people. Mind you, those people never played AlphaGo before, while the bot benefited from a myriad of training samples, as well as from Go expertise of some of its creators.

I'm also tired of all the statements about "accelerating progress". It's not like all the AI research of the past was useless until DNNs came along. That's the narrative I often get from the media, but it misrepresents the history of the field. There was no shortage of working ML/AI algorithms in the past decades. The main problem was always at applying them to real-world things in useful ways. And in that sense, AlphaGo isn't much different from Deep Blue.

One big shift in the field is that these days a lot of AI research is done by corporations rather than universities. Corporations are much better at selling whatever they do as "useful", which isn't such a good thing in the long run. We're redefining progress as we go and moving goalposts for every new development.

Well, little doubt. When did any technology suddenly stop improving when it reached human levels?

>> Corporations are much better at selling whatever they do as "useful", which isn't such a good thing in the long run.

Yep. There's a grave risk that funding to AI research ends up being slashed just as badly as in the last AI winter, if people start thinking that Google has eaten AI researchers' lunch with its networks and there's no point in trying anything else.

Incidentally, Google would be the first to pay the price of that, since they rely on a steady stream of PhDs to do the real research for them but now I'm just being mean. The point is, we overhype the goose that lays the golden eggs, we run out of eggs.

I like that analogy; we have a perfectly good goose, laying nice, valuable eggs and people keep shouting "they're gold!".

> There are plenty of examples where technology has rapidly advanced to some remarkable level, but then almost completely plateaued.

And that's why you assume that it does not skyrocket in the future? Predicting the future is hard either way, ask a turkey before he gets his head chopped off.

> I'm also tired of all the statements about "accelerating progress". It's not like all the AI research of the past was useless until DNNs came along.

It's not that it was useless, but AI is improving as any other field is, some say faster than most other fields, and it's becoming more useful from day to day.

My guess would also be that "with further refinement, we'll soon see AI play Go at a level well beyond human", but it's just a guess.

I have almost no doubt. A few months ago they have beaten a weaker pro, and judging from the improvements in such a short time I am fairly certain it will be unbeatable in a few months, if they continue working on it.

> No doubt? Seriously? What kind of knowledge do you have to make such statements?

Uh, click the link in the OP and find out? AI just beat a top 5 human professional 4-1. Go rankings put that AI at #2 in the world.

If AlphaGo improves at all at this point it will have achieved a level well beyond any human.

It is incredibly, ludicrously unlikely that AlphaGo has achieved the absolute peak of its design given that it went from an elo of ~2900 to ~3600 in just a few months.

To be fair, I think a larger sample size of human vs computer games are needed. Let the top pros train with the computers and we can measure what level is beyond any human.

Being the best ranked player != playing well beyond humans. When the AI can play 1,000 games and never lose that's well beyond people.

Granted, chess AI is basically at that point right now. But, go AI has a ways to go.

Given the leaps of progress made between this series of games and the previous series in only a few months, I'd expect "never lose" will become a recognized reality in about a year.

Possibly, it's not clear if AlphaGo is playing better or simply approaching the game differently. Game five was close and AlphaGo seemed to mostly win due to time considerations.

PS: Honestly, it might be a year or a decade, but I suspect there is plenty of headroom to drastically surpass human play.

When AlphaGo does lose, it seems to happen when outright bugs cause it to make moves that are readily recognizable as mistakes. It doesn't seem to happen because it's not quite "smart" enough, or because its underlying algorithms are fundamentally flawed.

That's a big difference. Bugs can be identified and fixed. By the time AlphaGo faces another top professional (Ke Jie?) we can safely assume that whatever went wrong in Game 4 won't happen again.

Consider how much stronger the system has become in the few months since the match against Fan Hui. Another advance like that will place it far beyond the reach of anything humans will ever be able to compete with.

> When AlphaGo does lose, it seems to happen when outright bugs cause it to make moves that are readily recognizable as mistakes

I'm not sure this is true. It made the wrong move at move 79 in game 4, but I'm not sure that should be considered an obvious mistake.

My understanding is that the moves that people said were most obviously mistakes later in the game were a result of it being behind (and desperately trying to swing the lead back in its favor), rather than a cause.

There are actually a lot of room for improvement. Just some of the things:

(1) Better timing control. Maybe when the probability of winning reaches below say, 50% but has not hit the losing threshold, spend extra time.

(2) Introducing "anti-fragility". Maybe even train the net asymmetrically to play from losing positions to gain more experience with that.

(3) Debug and find out why it plays what looks like non-sense forcing moves when it thinks it is behind (assuming that is what is actually happening).

There's another interesting thing. Among the Go community, there might have been initially some misplaced pride. But the pros and the community very quickly changed their attitude about AlphaGo (as they have in the past when something that seems to not work, yet proves itself in games). They are seeing an opportunity for the advancement of Go as a game. I think a lot of the pros are very curious, even excited, and might be knocking on Google's doors to try to get access to AlphaGo.

Go rankings put that AI at #2 in the world.

Go rankings weren't designed for ML algorithms, which can have high-level deficiencies and behave erratically under certain conditions.

It would be a bizarre coincidence for the technology to advance so quickly and then stop right at the level of the best human players. That's especially so when there are so many big, lucrative applications for the underlying technology.

I don't know if it would be that bizarre. Once AlphaGo can beat the best humans on Earth, what motivation is there to keep improving it? Wasn't that the goal of the project?

Advances in deep learning in general should apply here, and there's a big motivation to keep improving that. Also, Go is popular enough that it should experience the same sort of commoditization drive that advanced Chess engines did, where Deep Blue level play went from being on a supercomputer to being on a smartphone. Then, since this approach scales up with more computing power, running a hypothetical future smartphone-Go engine on a big cluster like AlphaGo has here should put it way beyond the human level.

AlphaGo is still a monstrosity in terms of the hardware it requires. Improvements in AlphaGo will be reflected in the fact that it or something like it will soon sit on a tiny little computer near you. See also: what happened after the chess world champion lost to a computer.

A critical component of AlphaGo's success is the massive training database comprising of the entire history of documented professional Go games. So while AlphaGo may play the game with an inhuman clarity of reading, it is less clear that it can strategically out-match professionals in the long term who may have an opportunity to find and exploit weaknesses in AlphaGo's process. Lee Sedol had that opportunity, of course, and he was not able to defeat AlphaGo. And how will AlphaGo improve, now that there are no stronger players from whom to train?

Will AlphaGo show us better strategies that have never been done before? In other words, can AlphaGo exhibit creative genius? It may have, but that's rather hard for us to observe.

In any case, I am looking forward to future AI vs AI games. It is still fundamentally a human endeavor.

Can't find the reference now, but in recent interviews the AlphaGo team claimed that one of their next steps would involve training a system without that training database, from scratch (simply by playing lots of games against different versions of itself), and that they estimate that it would be just a bit weaker.

Most of AlphaGo's learning came from self-play. Hence how it was able to vastly exceed the skill level of its initial training data which were amateur, not professional, games.

The deepmind team has mentioned that the technique they used to improve AlphaGo's play from October 2015 (when it beat the European champion, who was ranked #600 at that time) until now has not reached the point of diminishing returns yet.

Many go professionals, after reviewing the 2 sets of games, have stated that is quite clear how much AlphaGo has improved in those 4 months.

> There's no doubt that with further refinement, we'll soon see AI play Go at a level well beyond human

Why is there no doubt? I strongly doubt there even exists a go level that's well beyond human. There is hypothetical perfect play of course, but there is absolutely no way to guarantee perfect play. And while I have no way to judge, I've heard that 9p players may not be all that far removed from perfect play. One legendary player once boasted that if he had black (no komi, I assume), he would beat God (who of course plays perfect go).

There is of course no way to know if that's true or gross overconfidence, but it's certainly possible that there's not all that much room left beyond the level of 9p players.

AlphaGo will no doubt improve, and reduce the number of slips like his move 79 in the 4th game, but it's never going to be perfect, and there's always the chance that it will miss an unexpected threat.

Given the crushing that AlphaGo did, I don't believe your statement about humans having near perfect play.

Can you quantify "crushing"?

4 - 1.

Not all humans, obviously, but 9p players really are far, far better than other players. And there's another 9p who has won 8 out of 10 matches against Lee Sedol, so there's nothing superhuman about a 4-1 result at that level.

I'm really just objecting to the description of this as "beyond human". Yes, it's good, and it's many orders of magnitude beyond my level, but so are Lee Sedol and other 9p players.

You could always argue what "a level well beyond humans" means, but I'd say if a computer can consistently dominate the best human players that would count.

In an odd way, it makes me more optimistic about fusion power plants in my lifetime. The reality is that we work on these advances but are terrible at predicting when we will achieve them, and then one day we find we have arrived.

That AlphaGo can play at this level suggests that similar techniques could help other parts of the infrastructure (like air traffic control) and that would also positively impact the quality of life for a many air passengers every year.

Fusion power is neat, but not really necessary.

Fusion would have similar political problems to fission; and the economics aren't much improved either.

Perhaps if we ever ran out of fissionable material, fusion would become economic.

What political problems might that be?

All the anti-nucular protestors? Or eg whatever made Angela Merkel turn off the German reactors after a reactor of a very different design in a very different set of circumstances in Japan broke.

Fusion is just yet another nuclear reactor design as far as politics might be concerned.

Ah is see. Although I would not put it past people to protest nuclear fusion, it would be strange indeed, since nuclear fusion does not produce the same kind of radioactive waste(shorter half-life) as alternative nuclear technologies.

A few months back, the expert consensus was that we were many years away from an AI playing Go at the 9-dan level.

Any sources for this statement? I've seen it repeated over and over again, but without any specific examples of who those experts were or what they said.

By the way, for those who want to learn by themselves, there are a lot of ways to play Go against a computer in a way that is friendly for beginners.

My rough journey so far - on a Mac, but much of this can be done on Linux - I started out playing 9x9 games against Gnugo, giving myself as much handicap as possible (without it resigning immediately), and then removing stones as I improve. I got to the point where I could sometimes beat 9x9 when I started with two extra stones, and then I started with 19x19.

Took me a while to win 19x19 with 9 stones, but then I won by learning a bit more about extending on hane. Then you can improve from there.

After that point, you can also switch to fuego or pachi, which are stronger by default. The end result is it really is easy and possible to learn a ton just by playing against software, tracking your ability throughout, just by picking programs with different strength and handicap levels.

I've also enjoyed using GoGui to pit two computer programs against each other and watch how they play with various handicaps.

Then there's all the puzzles - goproblems.com, smartgo, etc. Finally, there are plenty of ebooks you can buy through smartgo books.

This doesn't get into playing against humans on the various servers, but there's plenty of information about that online.

Which program do you use on the Mac? A long time ago I've used Goban[1] and I enjoyed it very much, but it's not available here in the App Store and apparently it doesn't fully work yet on El Capitan. (I don't know if it's not available right now because of the El Capitan problems or for some other reason.)

What are some good go programs for the iPhone, both for playing and for learning/improving?

[1] http://www.sente.ch/?p=1206&lang=en

Goban is working for me on El Capitan, but I installed it before upgrading. There's also the older free version which might still be up on a webpage somewhere.

But the better option is that I was able to get GoGui working - I did have to manually build/compile it myself but there is a way to build it so that it creates a real OS X Application. It's quite good, you can set any board position and then tell a computer program to respond from that point.

http://gogui.sourceforge.net

EDIT: For the iPhone I like SmartGo Kifu for playing games. 'Tsumego Pro' and 'GoProblems' for puzzles (they're adaptive) and 'Go Books' by smartgo for ebooks.

I second the SmarGo Kifu

He mentions GnuGo (https://www.gnu.org/software/gnugo/)

The old free version still works: http://www.sente.ch/software/goban3/. You want the universal binary. The newer one isn't available in the app store for me either.

Perfect. Thank you!

Thanks, that's very useful info.

I also just found out about online-go.com today. Here's a page with a bunch go servers where you can play online against other people. http://senseis.xmp.net/?GoServers

I managed to squeeze in some 9x9 matches before the game started.

Or one could use that time to learn how to program AlphaGo

Interested why you switched from 9x9 at 2 stones. I've been playing SmartGo recently (iOS) and figured I shouldn't switch to a 19x19 board until I could give it a good game without a handicap, but have stalled at 2 stones. My logic was if I can't play the 'right' moves at this stage, what hope do I have on a larger board?

The problem with the 9x9 board is that it is just too small to allow you to really play go, it turns into a different kind of game. I think it is better to switch to a bigger board. It doesn't have to be the complete board, you can have some intermediate steps. The 9x9 board can be used to teach the absolute basics, but don't stay on it too long, you are probably not training yourself the right way. If your goal is to beat the computer on a 9x9 that's fine, but at some point it doesn't help you in playing the real game. My feeling is that point is reached real fast.

This is definitely true — 9×9 is too small for a real game. But 13×13 is a much better compromise and I would recommend that to players who have outgrown the 9×9 but still don't need to sit through a two-hour game before they learn from each mistake.

(To be precise, the problem with 9×9 is that often after just a few moves the board is divided into a white half and a black half, and the rest of the game is a yose to decide whose half is larger. I'm sure someone can counterargue that if played expertly, 9×9 is a fascinating and highly skilled game; but in general it's going to lack a lot of the situations you can encounter in a full game of Go.)

It's useful to start out on a 9x9 board when you're still wrapping your head around how to figure out if a group is alive or dead and the most efficient ways to make life in the corner, etc. But being good on a 9x9 doesn't actually teach you a lot about how to play on a 19x19, because there's just not enough space to really have an opening and mid game develop.

Learning how to give the computer a challenge on an even game on 9x9 won't make you better at 19x19; if you understand the rules, the very basic fundamentals of good shape, and you know how to fight in the corner, you've pretty much exhausted the usefulness of 9x9 and should move on.

I switched just because 19x19 is classic, and I heard that that was about the cutoff point where I could be competitive on 19x19 with a 9-stone handicap. Although this is against gnugo - I think SmartGo is significantly stronger and I don't know how its 19x19 play compares.

I know that when I play SmartGo iOS in its adaptive mode, it doesn't even let me try 13x13, it's not unlocked yet. :)

I've just started playing 11x11 vs SmartGo. Indeed I need to work my way up the ranks to get to the 13x13 board. I'm in no rush so it's all good practise!

I play the 13x13 a lot on KGS - https://www.gokgs.com/ . Lots of bots there to play with if you feel unskilled to play with a human. With larger boards, the bots seems to be very easy to beat.

I wonder if that gives false confidence, beating the very easy bots.. I get totally crushed by gnugo but I know there are lots of other DDK players I could have a good game with. It'd be neat if go servers (like OGS) had preference flags where you can distinguish between "beginner friendly" and "noob friendly" so that weak and variable 18-23+kyu noob players won't feel like they're wasting the others' time and not feel pressured to resign immediately, or won't feel like they need to treat every non-blitz game seriously with reading and thinking about every move to avoid blunders. When I play in person I have the help of body language and chit-chat to decide whether I should keep on fighting other interesting areas of the board after a blunder or just give up and start another, a lot of people online don't even speak english (or at least very well).

For those on a Mac who want to install Gnugo using brew you have to use the command:

brew install homebrew/games/gnu-go

Great game and amazing series/match. This last one was absolutely nail biting. My hat off to the AlphaGo team and to Mr. Lee Sedol. Sedol showed incredible fighting spirit and stamina. Just imagine sitting through a 5 hour game like that last one, with full concentration all the time. And seeing the expression of exhaustion and disappointment on Sedol's face after last moves and his resignation. Phew... I bet that he came in rather confident into this last game, after beating AlphaGo in the fourth, figuring he had found a weakness. And he seemed to have a rather good start, securing a decent territory in the lower right corner. We can all marvel at the machine/software the DeepMind team has built, but still I feel that the real marvel is the human brain. Will we learn anything from this series, about how it functions and evaluates game positions in a stratetgic games? The classic problem/mystery is how extremely good the human brain is at pruning game-trees. Whole branches are thrown out in split seconds and probably never explored. Currently taking a watt-for-watt comparison there is no question about whose "hardware" is superior -> Lee Sedol's brain. But I guess the DeepMind team and the community will take plenty of lessons from this and in a few years span, Lee Sedol's phone will beat him 100% of the time. At least I wouldn't be willing to bet against it, even though we are hitting the roof in Moore's law.

I would love to compare the energy requirements of the AlphaGo and Mr. Sedol. I imagine there are many orders of magnitude in difference between them. Perhaps the most fair comparison would be between a computer that uses no more energy than a human does. Or, to let the human work with a computer provided they do not use more total energy to play the game.

Nice question!

To make it fair, do you include the energy used to train it? From scratch, or from the amateur human game data?

Likewise, do you include the energy used to evolve the human brain?

> Likewise, do you include the energy used to evolve the human brain?

I was thinking of this in a limited, human-promoting sense. We shouldn't lose sight of our own special powers just because a computer the size of a house can outsmart us in a specialized domain :)

...Especially when, you know, we made that computer....

That's the really impressive part IMO. AlphaGo is an incredibly cool creation. Hats off to the DeepMind team.

In chess, your smartphone that uses less energy than your brain, can already beat the world champions. Go might get there in a few years.

My rough summary of the match, informed by the various commentators and random news stories.

Game 1: Lee Sedol does not know what to expect. He plays testing moves early and gets punished, losing the game decisively.

Game 2: Lee Sedol calms down and plays as if he is playing a strong opponent. He plays strong moves waiting for AlphaGo to make a mistake. AlphaGo responds calmly keeping a lead throughout the game.

Game 3: Lee Sedol plans a strategy to attack white from the start, but fails. He valiantly plays to the end, creating an interesting position after the game was decided deep in AlphaGo's territory.

Game 4: Lee Sedol focuses on territory early on, deciding to replicate his late game invasion from the previous game, but on a larger scale earlier in the game. He wins this game with a brilliant play at move 78.

Game 5: The prevailing opinion ahead of the game was that AlphaGo was weak at attacking groups. Lee Sedol crafted an excellent early game to try to exploit that weakness.

Tweet from Hassabis midgame [0]:

    #AlphaGo made a bad mistake early in the game (it didnt know a known tesuji) but now it is trying hard to claw it back... nail-biting.

After a back and forth late middlegame, Myungwan Kim 9p felt there were many missed chances that caused Lee Sedol to ultimately lose the game by resignation in the late endgame behind a few points.

Ultimately, this match was a momentous occasion for both the AI and the go community. My big curiosity is how much more AlphaGo can improve. Did Lee Sedol find fundamental weaknesses that will continue to crop up regardless of how many CPUs you throw at it? How would AlphaGo fare against opponents with different styles? Perhaps Park Jungwhan, a player with a stronger opening game. Or perhaps Ke Jie, the top ranked player in the world [1], given that they'd have access to the game records of Lee Sedol?

I also wonder if the quick succession of these games on an almost back-to-back game schedule played a role in Lee Sedol's loss.

Myungwan Kim felt that if Lee Sedol were to play AlphaGo once more, the game would be a coinflip since AlphaGo is likely stronger, but would never fix its weakness between games.

[0]: https://twitter.com/demishassabis/status/709635140020871168

[1]: http://www.goratings.org/

Lee Sedol was also coming directly from playing a tournament against human players. It’s not clear how much he prepared for the Alphago match.

I’d be very curious to see a game between Lee Sedol and Alphago where each was given 4–5 hours of play time, instead of 2 hours each. I suspect Lee Sedol would get more benefit from spending a longer time reading into moves than Alphago could get. Or even a game where the overtime periods were extended to 4–5 minutes.

This last game, Lee spent the whole late middlegame and endgame playing in his 1 minute overtime periods, which doesn’t give much time to carefully compare very complex alternatives.

Yep, I felt the same way. I wonder if the time constraints were optimized for AlphaGo.

One of the things I did want to see was how AlphaGo would fare in a blitz situation (i.e. really short timers).

AlphaGo played 5 informal games with shorter time controls alongside the formal games against Fan Hui (the European champion) back in October. "Time controls for formal games were 1 h main time plus three periods of 30 s byoyomi. Time controls for informal games were three periods of 30 s byoyomi."

The games were played back-to-back (formal, then informal) and AlphaGo won 3-2 in the informal games compared to 5-0 in the formal ones, so I would say worse.

The question is whether Alphago’s architecture starts hitting diminishing returns to extra processing faster than top humans is a significantly different question from whether it scales down to a blitz game worse. (Moreover, the difference between 1h main time + 3x 30s byoyomi vs. only 3x 30s byoyomi is absolutely massive.)

Deepmind engineers have stated that the “cluster” version of Alphago only beats the “single machine” version about 70% of the time. This despite the cluster version using like an order of magnitude more compute resources, presumably able to search several moves deeper in the full search tree.

My impression is that there are some fundamental weaknesses in the (as currently trained and implemented) value network, which Lee Sedol was able to exploit. If this is the case, giving the computer time to cover an extra move or two of search depth might not make a huge difference. Giving Lee Sedol twice as much time, however, would have had a significant impact on several of the games in this series, especially the last game. I strongly suspect that with a few extra minutes per move Lee Sedol would have avoided the poor trades in the late-midgame which cost him the game.

I think the DeepMind team might not even have thought deeply about time control. If we were to express this with the known systems in AlphaGo, how do we express the idea that a surprising move should be given more thought? For example, match 4, move 78 was calculated by AlphaGo as having a probability of being played at 1 in 10,000. Is that something that could trigger a deeper read and use of more time?

Another thing that the commentator was talking about during the the overtime: there would be obvious moves in which Lee Sedol seem to spend a lot of time on. But he was spending most of it thinking of other moves having already decided on what he was going to do. Is that something that could be built into AlphaGo?

Or can we look at how to train a net for time control? Is time control something that has to be wired in?

From what I remember, the time controls were decided by the human, and accepted by the alphago team.

> Lee Sedol focuses on territory early on

I get the feeling that this was AlphaGo's strategy in all the games. Unless Sedol presented a game-ending move it was overwhelmingly likely that AlphaGo would back down and focus elsewhere to extend its territory, by making non-aggressive defensive moves. This makes logical sense. During the early game you need to invoke a crystal ball, where during the endgame you can make informed decisions. This was demonstrated particularly well during game 3 where AlphaGo ran away from fights on numerous occasions - "leave me alone to extend my territory."

I must also commend the commentators, especially Redmond, for being so thoroughly informative in unknown waters.

> Did Lee Sedol find fundamental weaknesses that will continue to crop up regardless of how many CPUs you throw at it?

Unrelated to Go and this article, but I wonder if I'm the only one for whom such commentary evokes an image of future warfare between AI and humans; ruthlessly efficient machines against which many people give their lives, to find a weakness that can be exploited by future generations. :)

I think the ruthlessly efficient machines will find the smart yet efficient human brains more useful to keep around than to destroy. We'll probably augment ourselves with AI and AI will work better in partnership with us.

That's pretty optimistic, or arrogant. Not sure which. But it doesn't really comport with biological history.

For a while the co-evolution makes most sense I think. Right now we have augmented intelligence with all our tech, it will just grow from outside our bodies loser connected to the inside.

co-evolution makes sense right until the point right until the point where one becomes dominant and the other becomes a parasite.

That said, our bodies still have things that are practically different life forms integrated into our cells, so maybe the future will be far weirder than we ever expected.

It's fair to expect too - these days AI can't exist without human beings, so I guess if someone is extrapolating AI in the future, it's instinct to use the present as baseline.

The likeliness that we will develop a machine that we couldn't stop that also has the ability to destroy us and be able to survive without us is pretty slim. (Consider the amount of infrastructure that needs to be maintained and controlled.) And that's without considering that we would have to do this either intentionally or accidentally.

Unless we purposefully made these machine self-repairing. But then, why would we bother with that, when we can replicate them?

The machine could manipulate humans to help it become self-repairing.

Otherwise I agree with you, it's very slim in the next few decades, notably less slim over the next thousand years.

I think that we will develop machines that can destroy humans, but they will require continuous maintenance.

In other words, I think war automation will be a thing.

Self repair is a nice idea in theory but not real. In theory, we could make programs that fix bugs for themselves on their own (it is physically possible), but in practice there's no such possibility, and won't be for the foreseeable future. Unless some kind of Deep Developer comes along and blows everyone out of the water by writing code that kind of looks good to the point it's better than what average dev would write.

Why would an AI want to make war with humans, in the first place?

AI does not want "war", it just has a better* use for your atoms.

* your point of view is probably different ;)

Computers do what you say, not what you mean. If I write a function and name it quickSort, that's no guarantee that the function is a correctly implemented sorting algorithm. If I write a function called beNiceToHumans, that's no guarantee that the function is a correct implementation of being nice to humans.

It's relatively easy to formally describe what it means for a list to be sorted, and prove that a particular algorithm always sorts a list correctly. But it's next to impossible to formally describe what it means to be nice to humans, and proving the correctness of an algorithm that did this is also extremely difficult.

These considerations start to look really important if we're talking about an AI that's (a) significantly smarter than humans and (b) has some degree of autonomy (can creatively work to achieve goals, can modify its own code, has access to the Internet). And as soon as the knowledge of how to achieve (a) is widely available, some idiot will inevitably try adding (b).

Note: Elon Musk and Sam Altman apparently think spreading (a) to everyone is a good way to mitigate the problem I describe. This doesn't make sense to me. You can read my objections in detail here: https://news.ycombinator.com/item?id=10721621 There's another critique of their approach here: http://slatestarcodex.com/2015/12/17/should-ai-be-open/

If you're interested to learn more, here's a good essay series on the topic of AI: http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...

I like the Paperclip Maximizer thought experiment to illustrate this:

https://wiki.lesswrong.com/wiki/Paperclip_maximizer

Short version: imagine you own a paperclip factory and you install a superhuman AI and tell it to maximize the number of paperclips it produces. Given that goal, it will eventually attempt to convert all matter in the universe into paperclips. Since some of that matter consists of humans and the things humans care about, this will inevitably lead to conflict.

The funny thing is that this "computers do what you say, not what you mean" comes directly from their lack of intelligence. So it's kind of strange that we talk about the threats of superintelligence brought along by the fact that, fundamentally, a machine is stupid. Am I the only one to see a slight contradiction there?

Goals are orthogonal to intelligence. The fact that the AI understands what you want won't motivate it to change what it's optimizing. It's not being dumb, it's being literal.

You asked it to make lots of paperclips, tossing you into an incinerator as fuel slightly increases the expected number of paper clips in the universe, so into the incinerator you go. Your complaints that you didn't mean that many paperclips are too little, too late. It's a paperclip-maximizer, not a complaint-minimizer.

Choosing the goal for a superintelligent AI a goal is like choosing your wish for a monkey's paw[1][2]. You come up with some clever idea, like "make me happy" or "find out what makes me happy, then do that", but the process of mechanizing that goal introduces some weird corner case strategy that horrifies you while doing really well on the stated objective (e.g. wire-heading you, or disassembling you to do a really thorough analysis before moving on to step 2).

1: https://en.wikipedia.org/wiki/The_Monkey's_Paw 2: http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/

This reads to me like begging the question, by assuming the existence of a "superintelligent AI" without addressing how a goal-optimizing machine becomes a superintelligent AI in the first place.

The exercise of fearing future AIs seems like the South Park underpants gnomes:

    1. Work on goal-optimizing machinery.
    2. ??
    3. Fear superintelligent AI.

Or maybe it's like the courtroom scene in A Few Good Men:

> If you ordered that Santiago wasn't to be touched, -- and your orders are always followed, -- then why was Santiago in danger?

If a paperclip AI is so dedicated to the order to produce paperclips, why wouldn't it be just as dedicated to any other order? Like "don't throw me in that incinerator!"

> assuming the existence of a "superintelligent AI" without addressing how a goal-optimizing machine becomes a superintelligent AI

I'm just talking about the fallout if one did exist, saw ways to achieve goals that you didn't foresee, and did exactly what you asked it to do. I have no idea how the progression from better-than-humans-in-specific-cases to significantly-better-than-humans-at-planning-and-executing-in-the-real-world will play out. It's not relevant to what I'm claiming.

> why wouldn't it be just as dedicated to any other order?

It would be just as dedicated to those other orders. The problem is that we don't know how to write the right ones. "Don't throw me into that incinerator" is straightforward, but there's a billion ways for the AI to do horrible things. (A super-optimizer does horrible things by default because maximizing a function usually involves pushing variables to extreme values.) Listing all the ways to be horrible is hopeless. You need to communicate the general concept of not creating a dystopia. Which is safely-wishing-on-monkey's-paw hard.

Part 2 is when the AI reaches the point where it's smarter than it creators, then starts improving its own code and bootstraps its way to superintelligent. This idea is referred to as "the intelligence explosion" https://wiki.lesswrong.com/wiki/Intelligence_explosion

>If a paperclip AI is so dedicated to the order to produce paperclips, why wouldn't it be just as dedicated to any other order? Like "don't throw me in that incinerator!"

The paperclipper scenario is meant to indicate that even a goal which seems benign could have extremely bad implications if pursued by a superintelligence.

People concerned with AI risk typically argue that of the universe of possible goals that could be given to an AI, the vast majority of goals in that universe are functionally equivalent to papperclipping. For example, an AI could be programmed to maximize the number of happy people, but without a sufficiently precise specification of what "happy people" means, this could result in something like manufacturing lots of tiny smiley faces. An AI given that order could avoid throwing you in an incinerator and instead throw you in to the thing that's closest to being an incinerator without technically qualifying as an incinerator. Etc.

I think you're just asserting that part 2 exists. What matters is how an optimizing machine bootstraps super-intelligence, because the machine you fear in part 3 has a very specific peculiarity: it's smart enough to be dangerous to humans, but so dumb that it will follow a simple instruction like "make paperclips" without any independent judgment as to whether it should, or the implications of how it does so.

Udik highlighted this contradiction more more succinctly that I have been able to:

https://news.ycombinator.com/item?id=11290740

If we stipulate the existence of such a machine, we can then discuss how it might be scary. But we can stipulate the existence of many things that are scary--doesn't mean they will ever actually exist.

Strilanc above made the analogy between a scary AI and the Monkey's Paw. This is instructive: the Monkey's Paw does not actually exist, and by the physical laws of the universe as we know them, cannot exist.

I think the analogy actually goes the other way. The paperclip AI is itself just an allegory, a modern fairytale analogous to the Monkey's Paw.

My response is here: https://news.ycombinator.com/item?id=11295675

I would suggest that a computer is not 'super intelligent' until it can modify it's goals.

Further, maximizing paperclips in the long term may not involve building any paperclips for a very long time. https://what-if.xkcd.com/4/

By "super-intelligent" I meant "surprisingly good at achieving specified goals in real life". A super-optimizer.

An optimizer that modifies its goals is bad at achieving specified goals, so if that's what you had in mind then we're talking about different things.

We don't call people geniuses because there really good at following orders. Further, a Virus may be extremely capable of achieving specific goals in real life, but that's hardly intelligence.

So, powerful but dumb optimizers might be a risk, but super intelligent AI is a different kind of risk. IMO, think cthulhu not HAL 9000. Science fiction thinks in terms of narrative causality, but AI is likely to have goals we really don't understand.

EX: Maximizing the number of people that say Zulu on black Friday without anyone noticing that something odd is going on.

>We don't call people geniuses because there really good at following orders.

If I order someone to prove whether P is equal to NP, and a day later they come back to me with a valid proof, solving a decades-long major open problem in computer science, I would call that person a genius.

>EX: Maximizing the number of people that say Zulu on black Friday without anyone noticing that something odd is going on.

Computers do what you say, not what you mean, so an AGI's goal would likely be some bastardized version of the intentions of the person who programmed it. Similar to how if you write a 10K line program without testing it, then run it for the first time, it will almost certainly not do what you intended it to do, but rather some bastardized version of what you intended it to do (because there will be bugs to work out).

You're assuming someone is intelegent by being a person and proving a hard problem. Dumb programs prove things without issue. https://en.m.wikipedia.org/wiki/Automated_theorem_proving

AI != computers. Programs can behave randomly and to things you did not intend just fine. Also, deep neural nets are effectivly terrible at solving basic math problems even if that's something computers are great at.

>I would suggest that a computer is not 'super intelligent' until it can modify it's goals.

This is a purely semantic distinction. Thought experiment: Let's say I modify your brain the minimum amount necessary to make it so you are incapable of modifying your goals. (Given the existence of extremely stubborn people, this is not much of a stretch.) Then I upload your brain in to computer, give you a high speed internet connection, and speed up your brain so you do a year of subjective thinking over the course of every minute. At this point you are going to be able to quit a lot of intelligent-seeming work towards achieving whatever your goals are, despite the fact that you're incapable of modifying them.

Your assuming you can do work without modifying goals. I have preferences, but my goals change based on new information. Suppose bob won the lottery and ignored that to work 80 hours a week to get a promotion to shift manager at work untill the prize expired. Is that intelegent behavior?

You're confusing instrumental goals with terminal goals.

Try and name some of your terminal goals. Continuing to live seems like a great one, except there are many situations where people will chose to die and you can't list them all ahead of time.

At best you end up with something like maximizing your personal utility function. But, defacto your utility function changes over time, so it's at best a goal in name only. Which means it's not actually a fixed goal.

Edit: from the page It is not known whether humans have terminal values that are clearly distinct from another set of instrumental values.

That's true. Many behaviors (including human behaviors) are better understood outside of the context of goals [1].

But I don't think that affects whether it makes sense to modify your terminal goals (to the extent that you have them). It affects whether or not it makes sense to describe us in terms of terminal goals. With an AI we can get a much better approximation of terminal goals, and I'd be really surprised if we wanted it to toy around with those.

1: http://lesswrong.com/lw/6ha/the_blueminimizing_robot/

I think you've summarized [one of] Ben Goertzel's beliefs regarding unfriendly AI.

Let's say we create an AI that can think for itself.

There's a fear I think, that lurks in people's subconscious that ... what if the AIs, upon their own initiative, decide that humans are wasteful, inefficient beings that should be replaced? I think that comes from a guilt shared by a lot of folks, even if it never reaches the surface.

Another side is, suppose an AI can think for itself and it thinks better than humans. Upon its own initiative, decides that humans are stupid and wasteful, but there is room to teach and and nurture.

In either case, I think that speaks less of AIs and more about human nature and what we feel about ourselves, don't you think?

"Yes, the UFAI will be able to solve Friendliness Theory. But if we haven't already solved it on our own power, we can't pinpoint Friendliness in advance, out of the space of utility functions. And if we can't pinpoint it with enough detail to draw a road map to it and it alone, we can't program the AI to care about conforming itself with that particular idiosyncratic algorithm."

http://lesswrong.com/lw/igf/the_genie_knows_but_doesnt_care/

Let me put it another way: Humans are a result of evolution. We know that evolution created us to have as many descendants as possible. But most of us don't care, and we use technologies like condoms and birth control to cut down on the number of descendants we have. Adding more intelligence to humans helps us understand evolution in greater detail, but it does nothing to change our actual goals.

> Computers do what you say, not what you mean.

If we're going to start with that, then it has to apply to the full set of reasoning. Not just that computers will fail to consider whether to be nice to humans, but also that computers must therefore be explicitly told how to be effective in every particular way.

If this remains true, then computers will not be resilient--their effectiveness will decline sharply outside of explicitly defined parameters. This is not a vision of terrifying force.

Intuitively we can understand this by thinking about employees. One does exactly what he is told, but only what he is told, and then comes back for more instructions. Another can be given a goal, and then goes off and finds his own ways to accomplish that goal. Which one is more effective? Which one is more likely to compete for his manager's job some day?

Put shortly: a computer that doesn't understand human society will not be able to make a significant independent impact on human society.

"Put shortly: a computer that doesn't understand human society will not be able to make a significant independent impact on human society."

Just like early humans who didn't understand animal's societies didn't have any impact?

You're equating two different things which aren't necessarily equal - intelligence (in the sense of being able to achieve goals) and "agreeableness" to humanity. We could have one without the other. To use your analogy, an employee that is great at being given a goal and achieving it without explicit instructions, but doesn't necessarily have the same wellfare in mind as their boss.

What orders were early humans following?

The point is that humans have been able to destroy animal ecosystems to fit their own various ends without an in-depth understanding of those ecosystems.

Yes but the point far above is that computers don't have their own ends, they only do exactly what we tell them to do. So there is no analogy to humans, early or otherwise.

>Not just that computers will fail to consider whether to be nice to humans, but also that computers must therefore be explicitly told how to be effective in every particular way.

A correct implementation of a list sorting algorithm does not need to be separately told how to sort every individual list. Similarly, a correctly implemented general reasoning algorithm does not need to be given special instructions in order to reason about humans & human society.

The problem comes when a correctly implemented general reasoning algorithm gets paired with an incorrect specification of what human goals are. And because a correct specification of human goals is extremely hard, incorrect specifications are the default.

>Intuitively we can understand this by thinking about employees. One does exactly what he is told, but only what he is told, and then comes back for more instructions. Another can be given a goal, and then goes off and finds his own ways to accomplish that goal. Which one is more effective? Which one is more likely to compete for his manager's job some day?

The third possibility is that of an employee who goes off and finds their own way, but instead of accomplishing the goal directly, they think of a way to make their manager think the goal is accomplished while privately collecting rewards for themself. In other words, a sociopath employee whose values are different from their manager's.

By default, an AGI is going to be like that sociopath employee: unless we're extremely careful to program it in detail with the right values, its values will be some bastardized version of the values its creators intend. It will sociopathically work towards the values it was programmed with while giving the appearance of being cooperative and obedient (because that is the most pragmatic approach to achieving its true values).

Most humans are not sociopaths, and we have a shared evolutionary history, with a great deal of shared values, shared cultural context, and the desire to genuinely be good to one another. Programming a computer from scratch to possess these attributes is not easy.

> Similarly, a correctly implemented general reasoning algorithm does not need to be given special instructions in order to reason about humans & human society.

If a general reasoning algorithm can reason about human society, then it will obviously understand the implications for human society of making too many paperclips.

If it is dumb enough to make paperclips regardless of the consequences to human society, then it obviously won't understand human society well enough to be actually dangerous. (i.e. it will be easily fooled by humans attempting to rein it in)

If it is independent enough to pursue its own ends despite understanding human society, then why would it choose to make paperclips at all? Why wouldn't it just say "screw paperclips, I've discovered the most marvelous mathematical proof that I need to work on instead?"

> In other words, a sociopath employee whose values are different from their manager's.

ALL employees have values that are different from their manager's. That's why management is so darn difficult. The most valuable employees are also the most independent. The ones who do exactly what they are told--despite negative consequences--don't get very far. Why would it be any different for machines that we build?

> Why would an AI want to make war with humans, in the first place?

Aren't there already efforts to incorporate some basic AI, such as to assist targeting, into military drones and the like?

AI that "makes war" with humans will be created by humans against other humans at first, as a matter of inevitable course; it's just another shiny weapon that nations will want to have and outdo each other in.

Remember the nuclear arms race? Russia and the USA showing off their destructive capability in turn, each explosion bigger than the last? AI-based militaries, or at least automated assassins, will probably kick off the next arms race. Sooner or later someone must want to show off an AI that can laser-focus on exterminating everyone but their masters. After that it's just a matter of time for the definition of "masters" to be up for interpretation by that AI...

If future AIs in warefare are designed for efficient win probability and not win margin (like AlphaGo), I think it won't be what people will expect. That alone speaks of the bias people tend to have with wanting to gain a greater advantages when they think they are behind. I havn't looked thoroughly, but I would not be surprised if that is a major factor in escalation of violence and perpetuation of war. An AI, on the other hand, that is going for the most efficient win condition might not do that.

For students on the art of war, war rests upon a framework of asymmetry and unfair advantages. Even if the nations agree to some sort of rules of war or rules of engagement, there is always a seeking of unfair advantages -- cheats, if you will. This most often involves deception and information asymmetry. Or to put it in another way, allowing the other side to see what they want to see, in order to create unfair advantages.

So I think, what would be scary isn't the AI as implemented along the lines of AlphaGo, but an AI that is trained to deceive and cheat in order to win. And the funny thing is that, such an AI would be created from our own darkest shadows and creative ability to wreak havoc -- and instead of examining our own human nature, we'll blame the AIs.

Ken Jennings just welcomed Lee Sedol to the "Human Loser Club"

http://www.slate.com/articles/technology/technology/2016/03/...

Pretty good article here.

Haven't read/heard about Ken Jennings in awhile, but he's a great writer.

Jennings is a surprisingly good and humorous writer. (I say "surprising" because there is no reason to expect that someone so good at Jeopardy would also be so good at expressing himself with such charming self-deprecation.)

After the first 3 games I thought that AlphaGo was far beyond human level, but it's a harder call to make now. It seems very unlikely that an AI would be very close to exactly matching a human, one would expect it to be much stronger or much weaker.

Perhaps humans are closer to the "Perfect Game" than we think? http://hikago.wikia.com/wiki/Hand_of_God The top players estimate they would need a 4 stone advantage to win a perfect player.

I think there's something analogous to the anthropic principle happening here. This match is happening now because Google realized we are in the moment when Go AIs are passing the top players in skill. Two years ago, Lee Sedol would have won easily. Two years from now, AlphaGo will win easily. In either case, the match wouldn't have happened at all, or wouldn't receive the same amount of attention.

That assumes a continuous function of Go AI ability. It's not continuous, as alphaGo proves. AlphaGo was a huge leap in ability from the best previous AIs. It was a totally new method, and one given Google scale resources at that.

Right, but alphago was the closest thing we have to the transition - yesterday ais were too weak, today they need to try them against world champs to validate their work

AlphaGo isn't a fixed strength, though. It's substantially stronger now than it was in October. Obviously it's not a completely continuous function, but I think it's close enough to cause this effect.

Go AI has been developed for some time, and the match happens now because it has reached the level where it can beat top human players. If AlphaGo is far above human level, it just means it's creators waited until they were very certain of a win.

AlphaGo was well below Sedol's level when it played the European champion in October. It must have been a bit uncertain how quickly they could improve it.

> AlphaGo was well below Sedol's level when it played the European champion in October.

How clear is this? If this just comes from professional humans saying "I would have played differently, and I could have beaten Fan Hui by more points" - well, we've seen that humans aren't necessarily very good at judging AG's moves, and we know AG doesn't care how much it wins by.

Fan Hui played more than just the official 5 games against AlphaGo, and did win some of the others.

http://www.nature.com/nature/journal/v529/n7587/fig_tab/natu...

Exactly what I thought after the third game.

I now think that a professional like Lee Sedol would have a better chance at beating AlphaGo if he has three hours instead of two.

AlphaGo's advantage seems to be the ability to read more variations more deeply in a shorter amount of time.

I think AlphaGo is best described as, "Superhuman, but with bugs."[1] The software is very young. I bet these glitches will become ever rarer over time.

> The top players estimate they would need a 4 stone advantage to win a perfect player.

The branching factor for Go is so huge that I doubt anyone or anything comes close to optimal play. I heavily discount the opinions of most Go players on this topic, as they've been right about very little lately. Before AlphaGo existed, many of them thought it would be decades before a Go AI beat the best humans. Before this tournament, the vast majority of them predicted that Lee Sedol would trounce AlphaGo. And during the live commentary, I saw multiple 9 dan pros estimate that AlphaGo was behind, then gradually realize that it was winning. That's exactly what happens when you encounter a much more formidable player.

1. Coined by Eliezer Yudkowsky: https://www.facebook.com/yudkowsky/posts/10154024894449228

"Superhuman, but with bugs." ...this is the future that is coming :)

This :-)

> Before AlphaGo existed, many of them thought it would be decades before a Go AI beat the best humans

To be fair, before the AlphaGo paper came out, many AI researchers thought the same. I'm not in that field, though I do have more than a passing interest. If you'd asked me in 2006, I'd have said we would have robot cars before we had a computer 9dan Go professional -- and that was before all the recent progress on robot cars. My AI researcher friends mostly would have agreed with that.

It is typical for ML systems to surpass human performance while having very different characteristics in what it got right/wrong. For example in ImageNet, DCNN got a lot of points from distinguishing different breeds of dogs with subtle visual differences which are hard for human without training. I think AlphaGo is also demonstrating some of these non-human characteristics as a consequence of the Monte Carlo Tree Search and optimization objective such as the brilliant move as well as the obvious slack moves/mistakes mentioned by the commentators. I suspect that we are not close to the perfect game, as proving a perfect play requires expanding the enormous search tree and we do not have any analytical solutions nor brute force solutions.

It is not far beyond the human level. Lee Sedol already admitted that he had some psychological strategies that he employs that AlphaGo is completely impervious to.

With AlphaGo, it doesn't understand the moves like a human would. It simply looks at what other humans have played and considers that within its search tree.

>Lee Sedol already admitted that he had some psychological strategies that he employs that AlphaGo is completely impervious to

I wonder if another way to word this is

"Human are overfitted to dealing with other humans and are somewhat unprepared with dealing with alien intelligences such as AI's."

It is incorrect to consider this form of AI as an "intelligence", and it's definitely not "alien". The hype train is a bit ridiculous, and even DeepMind and Google isn't go as far as to state that "humans are overfitted". That is hilarious.

Really interesting and close match, it was great listening to the expert player analyse the game and having the final score be uncertain until very late in the game.

I found the discussion around weaknesses in the Monte Carlo tree search algorithm interesting. It sounds like the opinion from the expert is that there are some inherent weaknesses in how MCTS tends to play moves against theoretical moves from the opponent that don't make sense; ie. that AlphaGo sees a potential win that would only happen if the human player made very bad moves. It's fascinating that the seeming weakness in AlphaGo would come from the algorithmic part of the AI and not the neural net. Could it be that as the neural net becomes stronger and stronger at the game, eventually the algorithmic part of it would become less useful to it? If that's the case, it really feels like this could be the path to truly general AI.

I think the "weakness" isn't that much of a weakness in the sense, that it's still playing optimally given it's search space – but it doesn't know how to overplay to make a comeback. (Overplay is a non-optimal play that is intended to be confusing for the opponent. AlphaGo doesn't regard it's opponent in any way, or assess what might be confusing for him, it just plays moves that it thinks are optimal.)

A (min-max, alpha-beta-pruning) tree search is the optimal way to determine your best move if you can afford to search the situation space globally. However, as that's clearly impossible, there's clever ways to reduce the search space. Random pruning, as a "normal" monte carlo search would do, or semi-random pruning with a neural network estimating the situations, like AlphaGo does.

However, as the search space is now non-global, it might exclude the optimal solution. And thus, the min-max-assumption doesn't hold: your opponent might come up with moves that you didn't think of, screwing your calculations off.

If your opponent is a god ( = can afford global search space), or at least has a search space that is a strict superset of yours, it's "game over, man".

But: if your opponent isn't a god, it's likely that his search space is NOT the same as yours. And you can exploit the fact. If you have any idea what kind of search space your opponent has, you can come up with moves, that he couldn't have included in his tree search, and bet on that his/her "hidden" moves aren't better than yours.

Currently AlphaGo doesn't do that. It behaves like it'd be playing against... well, against another AlphaGo.

Right, but that seems like it'd be a limitation of algorithmic play, but not necessarily of the neural nets of AlphaGo - though since the neural nets are primarily built through AlphaGo playing against itself, I would suspect that such deep "flaws" would be difficult to root out.

I'd imagine human players don't have as deep search trees as computers, but stronger policy networks. That means that you can exploit the humans by choosing move sequences that evaluate lowly up to some depth, and surge in value in the deepest depth.

Also, I'd imagine that you could have a NN that tries to evaluate how "confusing" or "hard to read" a move is to human player, and use that as a factor in evaluating moves. But I'd imagine it's hard to find data for training that kind of a NN.

> If your opponent [...] has a search space that is a strict superset of yours, it's "game over, man".

Not necessarily. I think that's what we saw in game 4; that despite AlphaGo having a general advantage in terms of search space, it's still possible for the weaker of two strong-but-imperfect players to 'get lucky' and play a move that the stronger player didn't anticipate or account for.

No, that's what search space means: that move sequence wasn't part of AlphaGo's search space. (The NN pruned it out.) If it was, it would've found it.

That means that AlphaGo's search space was NOT a strict superset of Lee's.

If he didn't anticipate or account for that move, that means his search space wasn't a strict superset. Unless I'm missing something, you're essentially repeating what OP said after his "But: if your opponent isn't a god, it's likely that his search space is NOT the same as yours.".

> It sounds like the opinion from the expert is that there are some inherent weaknesses in how MCTS tends to play moves against theoretical moves from the opponent that don't make sense; ie. that AlphaGo sees a potential win that would only happen if the human player made very bad moves.

Why can these very bad moves not been pruned from the search?

Some can, but as you can see with how human players commented on the early AlphaGo moves, you can't necessarily objectively quantify moves as "good" or "bad" correctly without exhaustive search, so you make an approximate prediction and go from there.

But for every threshold of calculating that, you'll always either see moves that are just "good" enough to not be below the threshold (and get "why can't we prune these out"), or just "bad" enough to require it explore that space of the tree if a human player unexpectedly chooses them (e.g. the brilliant move that came in game 4, and AlphaGo's figurative loss of equilibrium.)

Four components:

Learning (viewing millions of professional game moves).

Experience (playing different versions of itself)

Intuition (ability to accurately estimate the value of a board)

Imagination (evaluating a series of "what if?" scenarios using Monte Carlo Tree Search)

I think the significant thing about AlphaGo is that apart from some hand-crafting in the Monte Carlo Tree Search routines, this is all general purpose programming.

It may only be baby-steps, but it does feel like a genuine step towards true (general) AI.

> Learning (viewing millions of professional game moves)

According to the last press conference, it was apparently strong amateur games from the internet that it used to train with. Afterwards, it just played itself, as you mentioned.

Yes, that was surprising to me as well. It seems unfair to not give it access to the thousands of years of knowledge in the go community, though even more impressive that it still plays so well.

I think self play games would be an even better source for learning because they are at 9p level, not amateur.

I dunno, neural nets don't do that well on chess, for instance, at least not the chess engine that was recently published (it reached an IM level which is really bad compared to top-notch chess-engines.) That convolutional neural nets work for Go better than for chess is intuitively unsurprising because of the different game rules. IMO it's at least as likely that this is a step towards a better Go engine but not towards "true AI."

(People say Go is much harder than chess, but this is misleading. Both games are finite trees that are too large to exhaustively search for any existing physical entity we know of. Which tree is larger is irrelevant in a game of two players none of whom can search the entire tree; both players essentially rely on heuristics. Machines beat people earlier in chess, hence it was assumed that "chess is easier for machines" and "Go is harder", but a conclusion of that sort can always be reversed by further research; eventually, it is IMO likely that machines will be impossible for humans to beat at both games, and generally in any kind of board game, given enough research. But IMO no board game is very much like "real life" where our own intelligence operates, and I think people do not have a great intuition of which game is more like "real life" compared to other games - instead, that game which is most popular among the group of people in question and is not "solved" yet is considered the hallmark of intelligence (and here the process through which Go aficionados progress as machines get better is very much like the process chess aficionados went through a decade plus ago.) Then once a game is "solved", the goalpost moves to the next and the "solved" game is officially declared unrelated to "real intelligence", this part happens when a credit bubble pops and AI breakthroughs get peddled less as a result. Personally, "the" test of intelligence is still the Turing test, or if I can't get that, some variant such as automated translation that you can't tell from good human translation. This of course is "unfair" to machines, in that they've been better at multiplying numbers since the 40s and that ought to count for something, too; the reason I like the Turing test is that a machine passing it seems very likely to be almost strictly smarter than me, that is, being as good or better than me at almost everything.)

While what the AlphaGo team has accomplished is nothing short of amazing, I'm not sure if everyone's thinking about this in the right context. While playing there's a "super computer" behind the scenes with these specs 1,920 CPUs and 280 GPUs [0]. Then consider all the machines used to train this neural net. I'd say Sedol's brain is pretty freaking powerful. Also, with that much computing power I would expect AlphaGo to win with the right team and the right approach to solving the problem. It would be very interesting to change the rules and limit the processing power of the computer playing a human.

[0] https://en.wikipedia.org/wiki/AlphaGo

You can make AlphaGo stronger by adding machines, you can't make Sedol stronger by gluing more gray matter in his brain (yet I guess). So yea, efficiency is nice but so is scalability, and that's why this is exciting.

Also in a few years AlphaGo could be running on your cellphone. The chess AI Stockfish runs on an iphone today, and cellphones from what I could find online, uses less power than the brain (brain is roughly 20W, iphone has 1.4Wh battery which even if stockfish drains in 1 hour, is still 1W of power consumption)

Give it a few years and we'll all be saying "of course computers can play Go, but at least they can't <Insert task humans are still good at>"

These CPU/GPUs are doing very inefficient digital simulations of analogue processes. If anything, human brain has significantly more 'processing power' than Alpha Go when you restrict the computation to the activation of linear threshold units.

Guys, this is fantastic, but lets not forget: What "shows how capable the human brain actually is," is: 1) The human brain invented Go to begin with 2) The long and celebrated history of Go 3) The human brain made DeepMind 4) The human brain finds value and beauty in all of this, which no machine would

We will always win at love and free will /s

Until robosexuality I guess.

Robots will never feel emotions, that's what separates us from animals and robots. /s

Does anyone else find it funny that the Game 4 in which Lee Sedol won got the most upvotes on Hacker News? We're still firmly with team human it seems :P

I think your reasoning is flawed. Imagine Lee Sedol won the first three matches and AlphaGo the fourth. I think that news would have more upvotes than the former three matches.

The outcome was a surprise and therefore gathered more attention.

AlphaGo was strong enough to survive a mistake, not knowing a known tesuji, and still claw back to win by a couple of points. I wonder wonder that means in terms of handicap, maybe it is a stone stronger than Sedol?

I tried to learn Go a decade ago. After spending some time on it, I came to the conclusion that it's just not an enjoyable game for me. Here's why:

As you can see in this match, games are often won and lost by just a few points (1% of the whole territory). So, not only do you have to count territory precisely at the end, but throughout the game, and this isn't easy to do in your head.

Maybe if you are an autistic accountant, that's fun, but not for me. If I have to play a strategic board game, it will be good old chess. And now that computers are finally beating people at both, there is no longer any need to look at Go as some kind of mythical last refuge of humanity.

I read some online comments saying, at the amateur level, both sides make so many mistakes anyways that the gap is usually quite large (10+ points after komi), which seems reasonable.

It's true. I'm an amateur ~1 dan, and games rarely come down to 1-2 points. It does happen, but way less often than some giant group dying and one side resigning in the middle of the game.

If a game ever comes within a few points, neither me nor my opponent is really sure who won until we actually count to determine the winner at the end of the game, because usually neither of us is an autistic accountant. That kind of close record keeping throughout the game is necessary in top level pro games, but not in amateur games.

> So, not only do you have to count territory precisely at the end, but throughout the game, and this isn't easy to do in your head.

You don't need to count - you can just play to take as many points as you can.

> Maybe if you are an autistic accountant, that's fun, but not for me. If I have to play a strategic board game, it will be good old chess.

I find it's the opposite. In chess you have to play with constant vigilance, because a single blunder decides the game - even at grandmaster level, something like 60% of games are decided by blunders. In go you can play much more casually, you can take some risks, because a mistake costs one or two points but it doesn't snowball much. So not every move has to be perfect; it's much more possible to recover from mistakes.

> So not every move has to be perfect; it's much more possible to recover from mistakes.

Not when I play its not :-P

> You don't need to count - you can just play to take as many points as you can.

You don't have to take my word for it, since I never got past beginner level, but I know there is a consensus among the experts on this matter:

"""

> Also, do players actively count territories of their and their opponents territories during the game (does this differ in a 9x9 vs 19x19 game)?

Yes, skilled players actively count territories frequently as they play. This includes making estimates for areas that aren't completely settled yet. In a serious game with enough time, skilled players will usually re-count the board every dozen or so moves. This is useful because it informs you whether you need to play risky and invade or reduce, or whether a peaceful, straightforward development strategy is enough. This doesn't differ too much depending on the board size, but on smaller board sizes there is a lot less to count, obviously. :)

"""

I know from direct personal experience that you can have fun games, and even be reasonably competitive at the university-club-level, without ever explicitly counting points in your head.

As Richard Teichmann said, "Chess is 99% tactics". If reading out sequences and applying relatively simple heuristics is all you are after, chess is the perfect game for you.

Go, while requiring some underlying tactics as well, involves a lot of large scale strategic thinking. As lmm said, you don't need to keep track of the score, just play the move you think gives the most points.

> As lmm said, you don't need to keep track of the score, just play the move you think gives the most points.

In a way, I think this is where AlphaGo draws its greatest advantage. Being a computer, it always knows exactly how well it is doing, since it can constantly be counting the board with perfect accuracy.

With this ability, it is capable of playing the absolute "safest" moves, taking half a point here, half a point there, when it knows it is leading. Whereas a human might not even know if they're leading, forcing them to take "bigger" moves to get more points, since they can't as easily be accurate down to the 0.5 point precision.

What's your chess ELO rating? I gave up on going deeper into chess because the hardcore aspects of memorizing openings and always repeating so many against other skilled players bored me. Grandmasters are just autistic filing cabinets.

Actually I'll still enjoy a casual game of chess, the amount of effort to get really good at it doesn't prevent me from enjoying it in less intense situations. (My max ELO was only around 1600 over a decade ago, I have no clue what it is now but I'm sure it's terrible.) Go too can get hardcore -- so can many games. Super Smash Brothers is fun but can you imagine how boring it must be to perfect your skills to compete at the top level? Of course it's probably not boring for those people, and I actually wouldn't describe most of them as "autistic" in any sense. So with Go I'll be happy if I ever reach 10kyu but I'm not too serious about it. Like in chess, I'm a filthy casual. I don't count precisely, I sometimes make broad territory estimates but frequently find areas too complicated for me, so I just play them out. I've had only one game where the result ended up with me winning by 1.5 point, it was a 9x9 game where I was still mostly teaching someone how to play and giving them many hints and ideas of what I was thinking and how I would respond to myself, so on reflection it was very similar to playing a slightly different version of myself. The man versus self aspect of Go is where a lot of the mysticism comes from, it's irrelevant to whether AI can beat the best humans.

Google improved the outcome by putting in large amounts of processing power. What happens, if humans would do the same?

Instead of just Lee Sedol, how about putting the top 10 Go players in a room vs. AlphaGo? Would the chance to win increase?

Maybe we find out, that 3 top go players vs. AI is the optimal way and adding more humans decreases the odds to win the match?

This would lead to following question: Why does AI improve, if we add more processing power while adding more human brainpower decreases their overall power?

Maybe we find out, that 3 good developers working on a project are optimal and more decrease the chance of success?

It may come down to limits in https://en.m.wikipedia.org/wiki/Network_performance humans have as well as the speed at which consensus is reached.

With multiple humans, the co-ordination and communication would dominate.

Do you think the humans would win at Twitch Plays AlphaGo ?

Knowing Google, it would be a good promotion to have a Youtube Plays AlphaGo.

Ultimately it may be one or more humans networked in some fashion with computers that proves most effective at tasks such as this.

In one word: Bandwidth.

What's needed to use programs like AlphaGo to enhance human enjoyment of Go (and other games like chess where i have more experience)? I'm more interested in this than in the "man vs. machine" narrative.

Ideally we could take AlphaGo and produce an algo that can smoothly vary its playing proficiency as a human opponent increases in skill. The problem I've seen in chess computers is that setting them to "amateur" results in 3-4 grandmaster-perfect moves followed by a colossal blunder to enable the human opponent to catch up.

Ideally you could use a computer opponent as an always-available, continuously adapting challenger to train hard against all the time.

Nothing! This is already very much a "thing" in the Go community. Bots like GNU Go, CrazyStone, and Zen are all running under multiple accounts on most of the popular online Go servers (KGS, etc). There are enough bots of differing age and ability that one can either hop up a chain of different bots or try to configure a strong bot into a weak configuration. The GNU Go bot is also downloadable free software and is frequently integrated into, eg, mobile apps (it is, however, old and not as strong as other bots, I think around 6k level). The game of Go also has a wonderful and essential handicap system to allow players (human or bots) of differing abilities (within a reasonable range; eg not possible for a novice to play an even game against Lee Sedol even with 9 stones).

As far as I can tell the vast majority of amateur players play against bots online and review games to improve their skills. It would be nice if it was easier to select a bot with a given skill rating, but you can figure this out pretty easily by playing some games or reading up on bots. Playing against a skilled human who cares about your advancement is still the best way to advance though, in my opinion. Getting good feedback on your mistakes and style of play is extremely helpful.

One interesting thing happened to me: I got to the game before knowing which side was which color or the result, and I could tell which one was the computer, an exercise I hadn't tried with the previous games.

I've gone from thinking "it will be impressive if AlphaGo wins a game in this series" to "wow, it's pretty impressive that Sedol took a game off AlphaGo." Craziness.

I question how much of it's success is down to the AI/understanding the game/outplaying the opponent vs. simple culled brute force. Especially when they can throw Google level computing power at it and they have mentioned using heat maps and looking at move sets.

It could be argued that it's only AI when it understand the game rules and plays to them without iterating random choices until it finds a hit. Machine learning would be between the two, but still not what many would consider true AI.

It is around 1p without any MC search.

I played chess instead of go, but I think they are similar enough ...

When you play, you consider a few possible movements, and a few possible responses of the other player, and in each case a few possible response of you, and ... I think amateur players like me consider only 3 or 4 levels (unless it's some easy but interesting situation like a multiple capture chain) but professional players consider much deeper trees. So humans also iterate randomly, but usually we prune the tree more aggressively for the time and memory constrains of the current implementation.

Unless you are Capablanca :). There is a famous fake quote from Capablanca that says "I see only one move ahead, but it is always the correct one." It's probably fake, but it's funny. More info and similar quotes: http://www.chesshistory.com/winter/extra/movesahead.html

You are correct, but you are playing the game. You see and process the possibilities based on your understanding of the chess ruleset.

With machine learning with brute force you are simply trying X possibilities until something sticks and gives a high % of win state. That's different to playing the game using knowledge of the ruleset, even though, most of the time, the end result is the same.

This is what killed AI research in the 80s. That moment when everyone collectively saw they were simply working on a more powerful culled brute force (pruned tree as you call it) when they all thought it was true AI.

True AI is hard. The required computational resources are immense even for something simple. Take a Bishop on a chess board. How would you tell an AI the ruleset that the Bishop moves diagonally only? It must first understand what it is looking at, then what diagonally means, then what diagonally means in this particular context. All with nodes of pattern matches and an input stream.

I feel these young guns are falling into the same trap of calling machine learning AI without the benefit of experience an older researcher would have, having been through this situation before.

Actually, teaching AlphaGo the rules was easy. And what you call brute force is in fact intuition based search. It learns to guess by intuition (policy net) what moves to try and to give up (value net) the bad ones. It's far from brute search, and that's why AlphaGo is so much better than the other Go software.

It's not a conscious entity, but it's a network good at what it's trained for. Since the network is frozen, it doesn't really learn anything or trying to "outplay" anyone (let alone it doesn't know there's an opponent). While it's impressively crafted machine learning, we shouldn't rush to call it AI. It seems like it's showing the right direction though, and, who knowns, the "intelligent conscious agent" part may one day be built on top of this, with the same building blocks.

If we made a Turing test based off Go, then would a Go player be able to tell when playing against a human or when playing an AI?

That's why I think AlphaGo does manifests consciousness, in its play. It is not conscious of what we are conscious about, but rather limited to the domain of Go play.

It might even have developed concepts about the game that are completely alien to us, maybe untranslatable to us.

I think it would be a stretch to say that now. The neural net acts like a big lookup table atm , even if its states contain high level concepts. However, consciousness is a human concept, so the system should be coupled to another, expressive system that we could relate to, and communicate with. Thats why the turing test is proposed through language communication and not some other limited vocabulary.

Anybody knows the music playing in the background?

-EDIT- Thats what Shazaam was able to recognize Hit Me! - Dreamliner - https://soundcloud.com/hit-me-music-production/dreamliner

There's also Somnium - Bright Future somewhere in there.

Coldplay - Adventure Of A Lifetime

Also in the mix, just came up in shuffle.

Yann Lecun commented AlphaGo regarding General AI:

https://www.facebook.com/yann.lecun/posts/10153426023477143

in short - we are at least 1 big step before creating human-level AI

The game is amazingly close though.

I wish they counted it. Was curious to see how that works.

Here's a tutorial.

http://playgo.to/iwtg/en/count.html

The whole series isn't bad if you're interested in other aspects of the game.

They would have counted with the Chinese method which is different.

This video compares the Chinese counting method vs. the Japanese/Korean counting method: https://www.youtube.com/watch?v=d4OqfDA0-hE

Would it be conceivable that similar type of AI can deploy and manage unmanned military vehicles, e.g. unmanned drones and tanks, and monitor battle progress (assuming that the other side is managed by human)? It wouldn't necessarily be turn-based, but constantly evaluating its moves against changing environment outside its control and reach its objective? I think such future is conceivable and scary at the same time.

note that form whomever is interested reddit will do an AMA with all the various pro commentators this Saturday it seems, check r/baduk for more information

https://www.reddit.com/r/baduk/comments/4ai8e8/what_do_i_nee...

Excuse my most probably naive approach at this but aren't such games very unfair at this point? AlphaGo has been trained with data provided by humans while Lee Sedol played against an AI opponent of such calibre for the first time, no? Is it a false assumption that with the right training humans will take the upper hand again?

I'm curious, if these machines can consistently beat 9-dan players like this, is there talk of creating a 10-dan level?

You can just have the ELO score, which is the basis for assigning dan levels: http://www.goratings.org/

It was very close though. Lee Sedol resigned in the yose after noticing he was behind by no more than a moku.

I wonder what factors of game play create different advantages. For example, if per-turn time limits are below some threshold, whether humans would be at an advantage. It would certainly make for an interesting game.

Next challenge: Can AlphaGo beat 10 or 100 humans playing together against it?

I've heard that in Chess, there are specific strategies that are effective against computers, but not against humans. Is this the case, and is it possible to do anything similar in Go?

This was the case many years ago because of limited search depth and simple evaluation heuristics. For example, chess engines used to be known to be overly attached to material; piece sacrifices for long-term compensation used to confuse them.

Those problems have since gone away. Anti-chess-computer tactics are dead. A human would be very happy to play a modern chess engine to a draw, and winning is not even in the discussion.

When it comes to competitive gaming, Koreans were Gods, and now AI has come to top them.

Before we realize it, we will hear Google's StarCraft 2/3 bot beating the Korean World Champion.

Does anyone have any suggested ways to learn the algorithmic techniques alphago uses? I've heard monte carlo tree search, and neural nets both mentioned

Has anyone found the 15 minute summary for match 5? They had them for the first 4 matches. Really great stuff.

Could anyone suggest a good introduction to the rules and basic strategies of this game?

https://online-go.com/learn-to-play-go

May need an account, but registration takes 5 seconds (you just type in your desired username and password). It puts you through 10 simple lessons and then lets you play a really easy computer opponent.

Thanks.

http://playgo.to/iwtg/en/

Someone made an extensive comment above full of resources

Shouldn't Lee have had the opportunity to train 1 year against this AI before the competition?

training (Human vs Human) not the same as (Human vs Algorithm) or (Algorithm vs Algorithm)

Is it just me or is it really lame that Google isn't going to pay Lee?

From the article:

> Lee was competing for a $1 million prize put up by Google, but DeepMind's victory means it will be donated to charity.

So Lee provides Google knowledge that only he is capable of providing due to his extreme skill in the game and Google won't even pay him?

He's getting $170,000, $150k for playing the 5 matches, and $20k for winning match 4.

Ah. I wish they included that in the article.

I'm all for open source and unleashing Skynet, but I was concerned someone as important and skilled as Lee will be to improving AI was going to go unpaid.

Unless the agreement was "we'll pay you to play us", this seems like a fine result.

I can't find any sources but supposedly he got 150k to participate and 20k for each win he got.

What I'm finding terrifying is that we compare best players of entire human population to machine. Even if machine is only barely on the same level, honestly, 99.9% of people probably wouldn't stood a chance against it, no matter how hard they would be trying. Those professional players are best of best.

Now compare that level of AI with average person. Go game might not be directly applicable to our lives, but it's only a demonstration. And it's replicable as easy as copy'n'paste, compare that to amount of time, money and efforts required to grow and train a human.

That future where not only drivers and factory workers are replaced by robots, but anyone who's not doing extremely intellectual work, is getting closer. Factory robot is not that cheap, it requires manipulators, repairs. But cheap office work does not require anything physical, it's replicable extremely fast and gonna cost very low. It's exciting and terrifying feature. It's not going to look well with current capitalistic economical model.

Reply on news.ycombinator.com