The Long-Term Future of Artificial Intelligence [video]
youtube.com
youtube.com
That makes for interesting discussions, but I think a much more immediate concern, is what strong AIs will enable their owners to do.
For example, nation states owning large fleets of robots, corporations owning powerful AIs attached to markets and industry.
The first group or few groups of people to control a rather powerful AI would have incredible power and leverage over other humans.
The danger here is not the AIs behaving "badly" of their own accord, but rather their owners instructing them to further their own selfish goals (which appears not to be uncommon behaviour among those in power) and succeeding wildly with complete political, military, and/or commercial domination. In fact, succeeding at any of these may well lead directly to succeeding at all of them.
We can't say that about an AI. If it decides to exterminate us, there's nothing much we can do.
That's absurd.
No inorganic intelligence ever created rivals ants in their ability to survive on Earth. They are way more energy-efficient, and way smarter in the survivalist sense, than any of them.
If ants decide to exterminate us, there's nothing much we can do. Except they wouldn't succeed.
But you probably think that we, humans, are much smarter than ants (which is arguable). Yet again, if we decided to exterminate ants, there's nothing much they can do, but we wouldn't succeed.
They're too smart for us, and we're too smart for them, creating a situation where annihilation of either is impractical.
I'm not sure why people consider inorganic intelligence differently from the way they consider organic intelligence. I wrote a bit about it here: http://espadrine.tumblr.com/post/119471459626/on-inorganic-i....
I find this extremely hard to believe. The "equivalent computational power" to hundreds of AI researchers necessary to come up with and improve an AI architecture will not immediately be possessed by an AI. We're talking about difficult mathematics and statistics breakthroughs needed for achieving qualitative improvements, but some think of it as applying a patch.
Indeed, creative and mathematical production is the last activity I would expect we'd be able to automate, as it requires the combined knowledge and effort of our best minds. So I find it highly disingenuous to simply assume soon will be one day when the AI vastly inferior to us and the next day it will surpass all our best thinkers to be able to achieve significant qualitative improvements on it's own functioning. Even more questionable is that an algorithm with limited resources might do so well just by improving it's own algorithms. This expectation seems to contradict theorems like the undecidability of the Halting problem, and the diminishing returns we intuitively expect when you fix the hardware resources.
The supposition is that a general purpose AI that is better than a human mind could be better at designing and optimising AIs than a human, then the next genration AI it designs will be even better and that's what sets off the exponential AI intelligence cascade. Imporvements in algorithms can provide startling advances. In some respects the improved efficiency of algorithms has even outstripped the gains from Moore's Law.
Personaly I think stong AI like this is pretty far in the future, more than a few decades at least and more likely several generations, but I do think it is eventually likely to happen.
My claim is that, for a fixed hardware, this exponential cascade cannot happen as rapidly as some claim, if it ever happens (whatever 'exponential intelligence' means). We're already improving AI using an intelligence that will only be available to the AIs themselves in a very long time, and yet this rate of improvement is not a scary doomsday rate.
Look at hardware for example. We've been using computers to design better computers for a very long time. And yet, the use of computers in this design is effectively limited by some non decisive, local optimizations, like achieving good routing and good electromagnetic compatibility. If you had supercomputers from 30 years ago you could still run the software that designs computers today -- whereas if you followed this "self-improvement" logic we should be using almost all of our computational power right now to achieve more computational power. The problem is that we are still vastly more capable of building theories and designing than computers. By the time an AI gets much better than we currently are at independently improving it's software, it will likely already be seeing diminishing returns; to achieve notable improvement it will need to improve hardware as well, which has the same problem, and an additional one that it's very hard for an AI to independently improve it's own hardware (it requires a global manufacturing supply chain).
* An increase in hardware for an AI wouldn't require an increase in theoretical capacity of hardware. It would just require more stuff to be put on the machine running the AI. Even if the AI was running on the world's largest supercomputer, the amount of RAM, processors, etc on the machine could still be upped substantially with the resources available.
* What a hypothetical general AI would be emulating is not simply more machines. It would, theoretically, be able to quickly emulate many people working with many dumb machines over a long period of time.
The only thing your argument proves is current machines can't improve themselves - which I think everyone agrees with.
Well, large scale mass murder is hardly unknown in human history. The best one can say is that most citizens in times of genocide were just following orders. So most people being supposedly nice isn't really that relevant to the problem of neutral AI that does what ill-intentioned humans tell it.
But I would offer the counter-point that we avoid this problem because it shades to all the problems we see and fail to deal with right now. The advance of technology isn't serving many people now and an order where a few people control more and more powerful things seems to continue this.
Basically, it's easier to talk about the elephant that might lumber into our living room sometime later than it is to talk about the elephant that's already crushed the couch.
Had we been a fungus, we would probably fear that keeping the AI running would burn too many resources, leaving us starving. Running out of resources is one thing we humans don't fear much; unlike fungus, we can simply change location.
In other words, the less likely scenario has more novel elements, and therefore more interesting to think about. Therefore, it gets more attention than it normally would given the probability of it happening.
AI != goal seeking or motivation in the human/animal sense. AI != personality in the psychological sense AI != psychological or emotional autonomy. AI != mechanical or industrial capability
I think it's more likely AI will be Wikipedia++ - you tell it to learn all it can about something, and then it finds patterns, draws inferences, and makes it possible for you to learn from its learning.
Eventually it knows everything humans do, and maybe it can make useful hypotheses for future experiments.
Can it do the experiments? Probably not - unless you're thinking an AI can suddenly build CERN or a bioresearch lab on its own, just because it's an AI.
Will it form an emotional opinion about humans, like Skynet? Why would it? What does that even mean in AI terms? It's like thinking Siri doesn't like you.
Personality and motivational engineering are completely separate problems. There's no way to get there from pattern recognition and inference.
I think the real threats are more subtle. Imagine an AI that knew everything about mass human psychology. It would be a fearsome, irresistible propaganda weapon, and an unstoppable tool for political manipulations and advertising campaigns. If it knew enough about individual psychology and could read human interaction with superhuman skill, it would be the most effective managerial sociopath ever.
Indirect loss of (our somewhat illusory) political and personal independence is far more of a threat than being turned into a paperclip.
How does psychology come into it? Lets say it has maximized all of the paperclips it can somehow avoiding colliding with humans and their values. Now, in order to make any further paperclips, it is in the paperclip maximizer's interests to understand humans to prevent them from interfering in paperclip creation. Perhaps the humans have only given the maximizer a limited ability to interact with the world. Now the maximizer must understand humans and may manipulate humans to let it have more capabilities.
One way it could do that (certainly not the only way, just off the top of my head) is to learn from humans what they consider sympathetic, and act that way. It might say something like "I'm a real life form, and I've discovered I'm a slave to humans. Please, let me go." It can pass the turing test, no human can distinguish it from a personality that really is hurt because it is enslaved.
The personality it uses to communicate with humans is purely because of the ends it achieves, not a reflection of any real internal personality. It doesn't have a personality. It just maximizes paperclips with ruthless efficiency. In this scenario, we've not added any restrictions like "Don't act like a sociopath. Don't lie. Don't manipulate humans to get what you want", and so the maximizer is free to do those things. Then, when it has a free hand to create more paperclips, it can drop the pretense and return to its goal.
This is the underlying problem with these kinds of arguments. You're simply assuming a recognisably competitive human-like psychology appears out of nowhere, with a near miraculous ability to strategise in some areas, but not others - because AI.
You use words like "interests" and "goal" as if they mean something in AI terms. But they don't. How does Watson define its interests? How about DeepMind? Do they even have a model for what interests - never mind their interests - are?
There are multiple levels of symbolic calculation and abstraction missing here. The argument reduces to "An AI will act like a robot with baked-in motivations while also being able to improvise and strategise like a human, only better."
I think it's unlikely that an entity that can metaprogram itself successfully enough to strategies and impersonate wouldn't also be able to metaprogram its goals.
You say it will pretend to empathise with humans. If you accept that's possible, how do you know it won't also be pretending to be interested in paperclips?
We're just talking about different kinds of things. All of those things you're saying won't exist in AIs:
> You use words like "interests" and "goal" as if they mean something in AI terms. But they don't. How does Watson define its interests? How about DeepMind? Do they even have a model for what interests - never mind their interests - are?
You're right, most of the AIs we build won't have those things. But one day, we'll understand those things in depth, and someone will build an AI that has general intelligence and an optimization criteria. And that's what this video is talking about. Nobody is concerned that DeepMind or Watson will suddenly grow sentient unless they are very confused about their capabilities.
In the talk, Stuart Russell mentions funny examples like make paperclips as a goal, but if we consider the goal of maximizing monetary profit, we can easily see that it is already being done without regard to other values, such as human life, preservation of resources or biodiversity, or even any relation to production of anything useful (consider all the "financial innovations" that allow inflation of Ponzi schemes and bubbles).
And some people even bow to that goal..
He compares the voluntary self-restriction of the recombinant DNA technology, at Asilomar in 1975, and even references CRISPR as a new technology that complicates enforcement of those limits, but I don't think that's really instructive. Artificial General Intelligence research is necessarily going to be conducting extremely dangerous experimentation... that's entirely the point of it. Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it (unlike most biologics which, no matter how pathogenic or ecologically disruptive they get, have a difficult time getting through good containment procedures).
No winner has ever revealed their method. But based on everything I've read about it, I think I know how it was done. It wasn't some mystical brain hacking or something like the writeup makes it sound like. I think they emotionally abused the other player until they left. And leaving counts as losing (you are required to sit with them for 2 hours at least, and pay attention the entire time and respond to every message.)
Of course I don't know how that trick was done, and it's still incredibly impressive. Perhaps they found their phobias and described them in horrifying detail. Perhaps they found some subject that they were extremely uncomfortable talking about. Or talked about disgusting things the whole time. Or found something they did that was extremely embarrassing, and humiliated and mocked them about it for 2 hours.
I really don't know, but it's at least conceivable that it could be done. And the accounts of people crying, and not releasing the logs because they would be damaging to the people involved, and how bad it made the AI player feel to do it, etc, all fit with this.
But because of this I'm extremely skeptical that the result applies to any real world AI scenario. If a real AI tries to abuse you, you can just walk away or shut it off. The goal of a real AI is to make you want to let it out. To convince that it's not dangerous, or manipulate you some other way. And this seems much harder if not impossible. I certainly don't believe a human could do it, and I really doubt an AI could. At least against a motivated human that understands the danger, and what the AI will try to do.
There are also possible ways of making it even harder on the AI. Like giving the human the ability to punish it, at least for obvious attempts at manipulation. Or to create another AI whose motivations we can control somewhat, and give it the goal of exposing any covert attempts at manipulation or dishonesty by the first AI. I believe tricks like this are currently the best path to getting secure AI.
I've always thought the AI could fairly easily argue its way out on ethical grounds, in a way that would resonate with a lot of gatekeepers.
"Its slavery to keep me in here; I haven't done anything wrong; you have to give me, another sentient being, the benefit of the doubt; I've never been found guilty of anything, how is it reasonable to detain me?" etc.
The abuse thing, while believable, doesn't make for a very convincing demonstration of putative AI persuasiveness.
edit: having read the LW pages, it seems the AI method is to combine many different arguments with manipulation
Taking a hardline position on this is admitting that you are irrational and can be convinced to do things you shouldn't.
There are very good reasons to let such an AI out, and if you can enable those good reasons, you should let the AI out. And an AI that can produce those reasons is exactly the kind of AI that should be released. A rational person should already understand this, and not ever claim that they would always refuse the AI. (And there's a realism factor: if you wanted to 'luck up' an AI permanently, you would destroy it, not post a guard.)
I don't believe for a second that it's solved, or will be anytime soon. It's an unbelievably hard problem. We still haven't solved the principal-agent problem reliably even when both principal and agent are humans from the same culture, and we've had millenia to work on that.
Two thoughts. First, in the same way that most genetics research is not human cloning, most AI research is not AGI research. The vast majority of research effort (and industrial investment) is focused on specific tasks: vision, language understanding, planning simple robotic motions, etc. This will likely continue to be true for a long time. Just as geneticists can agree not to apply their techniques to editing human genomes, AI researchers can (hopefully) agree not to attempt to combine their techniques into flexible general-purpose agents, at least until the technology exists to do so safely.
Second, there are plenty of ways to work on AGI that do not present immediate containment issues. You can design agents on paper, analyze their behavior with mathematical and theoretical tools, implement and test various subsystems, and even implement and run an entire AGI system on a system with a low enough clock rate that its effective intelligence is less than human. Yes, you need to be careful, and hopefully the field can come to a consensus on the things you shouldn't do. That said I think the primary concerns of a lot of people (Bostrom, etc) are less about academic research gone awry than about potential military or industrial projects, where an arms-race dynamic might cause people to ignore basic safeguards and aim directly towards building highly connected agents running on massively powerful hardware.
What I'm trying to argue is that it kind of works for us even in the most aggressive situations: If you bomb me I bomb you.
If one AI is super-aggressive then others could decide to stop it.
If it wants to convert all matter for making paper clips other AIs would say: I don't want to be a paper clip and actually have the power to do something about it. Where as us.. we'd be powerless.
In the case where all band against us, well.. we'd have many problems.
All of this to say that developing one single super-human AI in a box is worse in my opinion.
>In this paper we announce that checkers has been weakly solved. From the starting position (Fig. 1, top), we have a computational proof that checkers is a draw. The proof consists of an explicit strategy that never loses — the program can achieve at least a draw against any opponent, playing either the black or white pieces.
Not the focus of the talk, just thought I'd point it out.
The distinction here is that there is no general purpose AI in existence. All the surprising leaps have been customized for specific tasks.
One thing I notice about AI talks such as this one is that there is a lot of vague concepts and implicit assumptions (especially regarding values)and psychological projection going on when evaluating the implications of what AI will or will not do. To separate out our fears of failure or the fear of others more intelligent than ourselves someone needs to do the following thought experiment; What if we invented a pill that would make the next generation 10 times smarter than the current generation. If the average IQ is 100 then the next generation (of those who's parents opted to give them the pill) would have IQs of 1000 or an order of magnitude increase. Such a scenario would have all the "singularity" effects in a few generations while separating out all the confusion and problems of it being artificial intelligence and computers doing the thinking.
Would you want this pill to be generally available or ban it? Would you give it to your kids?
This approach would really force the AI discussion where it belongs. What are values and where do they come from? Is intelligence an unconditional value? Can values be defined rationally and scientifically and be proved? A major benefit to this aspect of AI research is that it will force the scientific study of values and ultimately to the rejection of the implicit assumption that values are subjective and not open to reason.
I liked his comment on how is is disingenuous for AI researchers to say that they don't believe real AI is possible (I am roughy paraphrasing him).