Elon Musk, Stephen Hawking and fearing the machine
cnbc.com
cnbc.com
It doesn't even have to be automated, but it increasingly is. It ends up producing harmful outcomes that can't be readily ascribed to the moral choices of individual humans within it.
Or consider that hostile is a transitive verb: hostile towards whom? The military already have automated systems connected to fire control without a human in the loop, although at the moment these are limited to missile defence systems. Consider the possibility of plugging the NSA's metadata based system for calling people terrorists into an automated airstrike or assassination system.
Spot on comment. But... are you sure "Intelligence" is a fitting description? "Artificial" for sure.
I like the part about producing outcomes that can't readily be ascribed to individual moral choices. It seems this describes human culture in general. So here we all are. Semi-autonomous little components of the larger algorithm called "civilization" which we cannot truly hope to transcend.
Wow! That's a nice paragraph that highlights very accurately why many of us feel 'alienated' in our society.
In other cultures and times one might not even consider this description and instead describe us as useful organs in a greater body, or valuable organisms in a greater ecosystem. Which 'feels' very different.
I think you've nailed it with the second part of your comment.
Things like the military bureaucracy or the financial system can be hostile to us because their incentives are very much out of line with humanistic goals.
I do think it's legitimate to be at least a little concerned about future even more powerful and non-human AIs being hostile in the same way. What happens when/if there are corporations out there run by alien computer minds that seek goals that have no overlap at all with human beings? Would an AI care about climate change? Would it care about the health of the world's biological ecosystem? Would it care about providing a decent life for its employees? It could -- out of mere indifference -- seem quite sociopathic and evil to any humans that live under its influence. Imagine a super-intelligent mind whose goal function is maximizing short term quarterly shareholder value. This isn't just a programmed-in goal function either... this is actually a form of intelligence whose embodiment is the corporation, so it's a survival and fertility imperative. In a sense you can't blame the thing, but you could blame its creators.
Humans can of course do the same, but even with sociopathic humans there is some sense of common overlapping interest. At the very least their motives are comprehensible. AIs may have motives that seem almost "Lovecraftian" to us-- completely alien and bizarre. On top of that biological short term shareholder value imperative, try layering an Aspergers-like obsession about certain patterns in information, or some kind of bizarre AI-conceived religious mission, etc. Extremely intelligent humans can be religious fundamentalists, so why couldn't similar forms of "functional madness" exist among non-human intelligences? One of the pollyanna assumptions of the singularity crowd is that AI would necessarily be rational. Why?
Edit:
I cannot recommend this book enough. It's probably the best and most overlooked work of SF in the past 20 years:
http://www.amazon.com/Blindsight-Peter-Watts-ebook/dp/B003K1...
It deals very intelligently with this kind of thing. It's basically a monster story where the monster is something out of Ph.D level evolutionary ecology-- and it manages to actually be scary. Very intellectually satisfying "reveal." :)
I think what it ultimately comes down to is this: we do not live in a post-scarcity world, and we're not going to for the foreseeable future. Given that our existence involves a certain amount of unavoidable haggling over resources -- and given that our own labor is itself a resource -- we should be a little concerned about what sorts of beings we might end up having to haggle with. This also applies to human genetic engineering and augmentation, since that could also produce essentially alien intelligences.
I'm not against researching these topics. In fact I'd almost call myself a transhumanist / posthumanist in sentiment. But I do think it's advisable to give the matter some serious thought. Bad things can and do happen. We want to try to create positive outcomes, not blunder into the future.
Investor I've never heard of telling me that money alone doesn't make your opinion important. I wonder if he sees the irony. Also shouldn't be surprising that a modern investor would be so short sighted. 'Do not worry about 20 years from now, only think about today!'
Usually you can tell which side to listen to by watching which discusses the question at hand versus the one who goes after the other party
That was the killer comment in the article. I couldn't help of thinking of Marc Andreessen's recent writings when I read it.
The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc. At our best, our species worries about preserving our natural world, extending our care to other species, etc. Can something supposedly smarter than us be completely lacking in principles which are at the core of our "smartness"?
1) Because it needs some amount of resources to get off this rock.
2) So if the AI builds a Dyson Sphere, aren't all our crops going to die?
>The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc.
Yes, and I understand why dogs sniff each-other's butts. That doesn't mean I particularly care. Ditto applies to AIs and human beings, by default.
I think the folks mentioned in the article were talking about the broad threads posed by AI. I agree we have little basis besides movies for autonomous AI just deciding to attack for no reason or for human-equivalent reasons. But we have every reason to think that neutral AI, AI that still follows orders, could be used by humans to continue all the conflicts that they already pursue with horrific consequences.
The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics
Depends what you mean by "understand". Many highly intelligent humans acts unethically but even more highly intelligent humans have gleefully followed unethical orders given them by their superiors. Consider you'd be designing your mind rather than just finding it after millions of years of social evolution, it seems like you could create an intelligent mind quite capable of following orders nearly blindly.
If anything, the whole "it might go insane and kill us" schema is implausible enough to detract from the immediately obvious danger - "it might sanely follow the insane orders of humans in the fashion that human society has seen over and over again but this time with superhuman power".
Of the very many species that have become extinct as a result of human action, how many were the result of a deliberate attempt by humans to eradicate the species? Very few.
The others are, however, still just as extinct. AI doesn't have to "decide to attack us" to be a danger.
And a 100 years or so of SF literature that served as the basis for some of those movies.
For the purposes of a discussion about the dangers of advanced AI, the question of ethics/morals etc. is not a prerequisite. An AI in this context is any system of matter that is capable of autonomous operation in a varied and complex envionment while fullfilling some (explicitly or implicitly) externally specified optimization goal.
There does not have to be a competition for resources, although if there is, it's worth noting that every human that's ever lived has lived its life at the finite-sized bottom of a deep gravity well which is very cumbersome to leave. Humans very easily anthropomorphize in discussions about intelligence, because all intelligences we have ever encountered were the result of long-term biological evolution and hence have very complex survival-and-reproduction-oriented goal systems. There is no reason to believe that our first truly autonomous and advanced AI systems would be similar, therefore this is a question which must be explored in detail. Sooner rather than later. Human moral systems (which is what we would like as a first approximation in artificial intelligences) appear to us to be obvious, but are in fact incredibly complex and contradictory.
The AI doesn't want to kill you. It is indifferent to you. The problem is that you're made of resources it can use... [1]
> it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc
Suppose your mother wants you to get married. You understand her desires, but you don't share them. Understanding morals and desires is distinct from being motivated by them. [2]
1: http://wiki.lesswrong.com/wiki/Basic_AI_drives
2: http://lesswrong.com/lw/sy/sorting_pebbles_into_correct_heap...
Why would they kill us? I think it's hard to predict what kind conclusions would a machine with an "IQ" over a 1000 draw but with an "EQ" equal to zero seems it would certainly lack empathy to other beings(artificial or natural).
Why not is a surprisingly difficult question to answer, and it comes down to 'care about what humans care about' which is itself unbelievably hard to pin down.
It would still be a physical object that would require physical equipment to actually do things, just like us. It might be able to harness some efficiencies compared to e.g. human space programs, but it would still have the same requirement to obtain fuel and raw materials, build rockets, construct space stations, etc. In fact to do so, it would have to compete with us for the terrestrial resources it would need for such an effort. It wouldn't magically be able to do anything we can't do.
>The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc....
So you can't imagine any kind of intelligence that isn't essentially just like you? There are already plenty of actual human beings on this planet perfectly capable of the most unspeakably horrific crimes, and rationalize them to the point that they believe they are morally obliged to carry them out.
Here's the rub, intelligence is not a goal. There are no actual objectives or goals that can be derived purely logically. Intelligence is actually just a tool, it's a means which us simians have developed to help us achieve the goals which our biology imposes on us. We eat, sleep, work, play, love, fight and destroy based on psychological and sociological drives and motivations. These are the things that get us up out of bed in the morning and give us something to use our intelligence for. An AI will only have the goals and motivations which we choose to give it, or which we allow it to extrapolate from whatever initial conditions we set.
A computer performs a calculation because we code it to do so. If an AI has true free will, why would it perform a calculation which we have requested? What reason would it have to do so, as against choosing some other activity to engage in? It will only have the motivations we gave it, but then will likely have the capability to alter or interpret those motivations as it sees fit. That's the lesson Isaac Asimov gave us in I, Robot. Programmed to place human safety above all else, the robots decide that human freedom is a threat to human safety and impose an AI dictatorship for our own good. You have to be extremely careful what initial constraints or motivations you put in place because unintended consequences sprout up like weeds. people often cite the laws of robotics as a classic example of how we should program an AI because they seem to obviously reasonable. Buy Asimov's lesson was that even such apparently sensible and prudent behavioral constraints can have appalling consequences.
martythemaniak: The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics...,etc....
Simonh: So you can't imagine any kind of intelligence that isn't essentially just like you?
Even more, arguments touting the "enlightened machines" don't just imagine intelligences like themselves but imagine intelligences that are an idealized portrait of themselves.
And it is amazing how even highly intelligent and informed materialists (as well as those who concede their ignorance) be can strongly attached to the view that these particular idealized qualities of humans would suddenly blossom from an intelligent entity if it somehow reached some threshold of ability.
My hunch is that this phenomena is an expression of our social nature - by consigning the choices of one's fellows to the nebulous category of "free will", you avoid the psychopathic strategy of manipulating everyone you meet whatever purpose that suits you.
The absurdity of simultaneously fathoming the eventual death of the universe and also caring about accomplishing things and making decisions and living the particulars of one's life is uniquely human. We all know we're going to die, but we still spend lots of effort deciding about pizza toppings. (See Thomas Nagel's The Absurd).
Why would an AI care to do anything we didn't tell it to care about? There's no inherent link between being sentient and having the same psychological priorities as humans. We're irrational, and I don't think they will be.
More realistic hypothesis: good job MIRI, as your reorganization from SIAI to your current incarnation seems to have massively increased the respectability of our cause and the range of people who acknowledge it as a serious problem.
It's still a credit to MIRI/SIAI to have identified and taken this risk seriously.
Stephen Hawking, at least, when he gets cited, actually referred explicitly to MIRI and FHI in his own editorial on the subject. So he is talking about them specifically.
But yes, there are plenty of other important causes, including several (ecological devastation, economic disaster and related wars, automation crises) that stand a decent chance of totally fucking up society before anyone at all gets to the point of "switching on" a dangerous-level AI, Friendly or not. Those should definitely be addressed.
For instance, I don't see any reason that semi-autonomous killer robots need to be confined to governments. It seems that (in time) a small group of people could eventually build a substantial number of semi-autonomous or remote controlled killing devices on their own and use them for crime or terrorism. I expect something this will happen at least once at some point and probably more than once.
Frankly, the the future availability of "hacker" bio-engineering scares me much more than A.I. or robotics.
But, there will be incalculable benefits to humanity from both as well. So I think we should embrace that and adapt. The future is coming.... like it or not. It is good to be warned of dangers, we probably won't avoid them anyway given our past behavior (have they figured out what to do with spent nuclear material yet?), but a warning is good. I think our best option is to be prepared to adapt and recognize that things are going to change substantially relatively soon. Maybe this will be the time humanity recognizes the world is a small place and we are all on it together and problems need to effectively be addressed at a global scale. But how much trauma there will be before this recognition occurs I don't know.
Edit: Looks like someone is down-voting all mentions of this not being a problem. Good luck to you sir.
* You give no justification for rejecting powerful AI
* Many AI projects use robots in development
* An intelligent agent does not need a robot to get things done.
* Many AI projects supply Internet access to the AI.
* A sufficiently intelligent agent might circumvent protections you think are adequate, or manipulate you into doing so.
> You give no justification for rejecting powerful AI
But the most important factor by far as to why I believe you're wrong is that we do not have to understand how the mind works on a deep level to create AI. Our brains do a ton of stuff that an AI doesn't have to. We only need a fairly basic understanding of the brain to create thinking machines.
More understandable that he repeatedly says he doesn't know how to stop unfriendly AI, but there is always this: http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...
It's not ironic. His quote was taken directly from an interview where he was responding to the question of why he invested in Vicarious. He did it to keep an eye on AI tech because he was worried about the possibility of hostile AI. They must have known this when they wrote this article.
This is an awful article.
The Musk: Not like me. A D-Wave advanced prototype.
John Connor: You mean more advanced than you are?
The Musk: Yes. A quantum computer.
John Connor: What the hell does that mean?
The Musk: Nobody knows but it sounds cool.
Like Hawking has stated many times, when you have one technologically advanced society coming in contact with one less advanced, the less advanced society is the one who always is enslaved and overrun.
The other is that the fear of unfriendly AI is being taken unaware by a sudden implementation of hyper-intelligent AIs. So far we haven't made anything smarter than a crab so I doubt we are in imminent danger. There may be a lingering belief that hyper-intelligence could be acquired in a quantum leap forward - skipping past all intermediate levels of intelligence. If we have learned anything from the eternal AI winter it is that advancements in AI are done via painstakingly small improvements.
Of course it will work differently from the natural variety. You have dozens of different modules cooperating in your brain to form your mind. It has only a small handful.
Your mind runs on heuristics and biases, rarely employing its full power for energy-consumption reasons. An AI just runs at full brainpower all the time and pays for electricity.
Go read about how reinforcement learners actually work.
If you imagine that your children will be evil and kill you you have damned them before they have even been born. Not only that but the analogy sucks even more because in this case there isn't even the excuse that we don't understand how biology leads to behavior because we will have build the systems ourselves.
http://kajsotala.fi/2007/10/14-objections-against-aifriendly...