Nick Bostrom: ‘I don’t think the artificial-intelligence train will slow down’
theglobeandmail.com
theglobeandmail.com
Lately I read this two piece story on Wait But Why which I really would recommend to anyone wanting to get a better overview of the topic:
- Part 1: http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...
- Part 2: http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...
He then outlines the mechanism for all of these progress-(non?-)deaths[1] to occur, which he calls superintelligence. I find that explanation as either the cause or solution to all these DPUs very unsatisfying compared to the historically supported causes of death on that scale: disease, natural disaster, famine, war. I may be the "stubborn old man" called out in the article, but I don't on the face of it believe superintelligence will eliminate most deaths in all four of those categories at the same time. It's positing a mechanism seemingly immune to the selection pressure that got us here.
But even if I put aside my doubts about superintelligence, I would find it significantly more helpful to see a hypothesis as to why increasing computational power is a general mechanism whereby there will be fewer deaths from disease, natural disasters, famine, and war. I suspect it is more fruitful to focus on how computational power will help solve problems with critical infrastructure (shelter, supply, safety, communication, transport, resource control, &c) rather that puzzling over how it may create a new cause of death exceeding the magnitude of disease.
1: The author seems to switch signs here and imply these will be non-deaths, which is supported by actual population growth, so let's proceed with that assumption.
The interpretation I believe you've developed is connected to the number of people who die prematurely at a given point in history, and that the author's point then is something about how superintelligence will impact the number of people dying of disease, famine, etc, either raising or lowering that number. I can't find any spot in the article where the author raises the concept of the rate of people dying prematurely at a given point in history, and definitely not any place where he connects that concept to DPUs.
My understanding of the DPU is as the amount of change required in daily life for a single time travelling individual to "die of shock" upon experiencing another moment of time. In the examples provided, 100,000 BC to 12,000 BC was enough change in day-to-day life experience to cause a person from 100,000 BC to "die of shock" if they were transported to 12,000 BC instantaneously. The same assertion was made for 12,000 BC to 1750 AD, and 1750 AD to 2015 AD, with the author's conclusion being that there has been an exponential shortening in the timescale required for enough change to occur in the daily experience of human life to cause a time traveler to "die of shock", and that such shortening will continue into our future -- possibly to the point of allowing such a level of change to occur multiple times within our own lifespans.
I took the entire discussion of DPUs solely as an exercise in generating an evocative image for illustrating the increasing rapidity of change in our qualitative experience of life. I don't think the "die of shock" idea is meant to be taken literally; it's just a convenient stand-in for "extremely shocking, to the point that the experiencer may be incapable of processing the instantaneous change rationally", not a measure of people actually dying.
(Sorry for going to such length, I just wanted to be precise. This is a perfect illustration of https://xkcd.com/386/)
What we have with Watson (the Jeopardy model, because the name is used by IBM as an umbrella for staff) etc is the same kind of number-crunching, dumb-smart AI we always hand.
Without any qualitative steps that wont fly.
I've been hearing that since I started using computers, 36 years ago.
36 years ago the argument that AGI was coming soon could be made in tandem with the argument that we'd make some fundamental advance that allowed computers to express intelligence with less computational capacity than humans (by orders of magnitude). Today we can make an argument that we'll achieve it (at least initially) by leveraging computational capacity on par with or orders of magnitude greater than a human mind.
How much, yes, how, not so much and definitely not at the powerbudget the brain has.
> and how long it will take to build computing machines operating at that scale.
We don't actually know that. There have been some WAGs but so far those appeared to be totally off based on the developments since.
> 36 years ago the argument that AGI was coming soon could be made in tandem with the argument that we'd make some fundamental advance that allowed computers to express intelligence with less computational capacity than humans (by orders of magnitude).
Yes, that was a crucial mistake and it led directly to the AI winter.
> Today we can make an argument that we'll achieve it (at least initially) by leveraging computational capacity on par with or orders of magnitude greater than a human mind.
Chances are that we're missing a very important piece of the puzzle for which there is no known solution even in theory. The problem is that there are many candidates for that important piece none of which have currently proposed workable solutions no matter what the computational budget or the accepted slowdown (they are equivalent).
So I think some caution when throwing around projections numbering 'just a couple of years' is warranted, after all it's 'merely a matter of programming' but in this case we don't have a working model that we understand.
For now - as far as I can see - we are no closer to the goal than where we were 35 years ago but we know better how much we are still missing and all the hard parts are still in front of us.
There were perfectly educated people that said there would never be flying machines, 2 years before the Wright brothers succeeded.
It's not about saying it can't be done it is all about saying it can be done within a specific time-frame.
Key inventions don't happen 'on command' unless the only thing required is a brute force search for something that is already possible in principle (say electric light).
ANI and AGI are qualitatively different. Progress in making cars faster will not lead to teleportation or warp drive. It's not even applicable.
There are many unknown unknowns in AGI. I could not even pretend to give you an estimate.
Russian classification traditionally also have distinction between terms "artificial intelligence" and "artificial mind" ("искусственный разум") to keep the problems of autonomy/agency/consciousness out of field of AI.
I wrote a summary here of the main achievements of 2014: https://www.reddit.com/r/Futurology/comments/2qq993/developm...
Stuart Russel recently said "The commercial investment in AI the last five years has exceeded the entire world wide government investment in AI research since it's beginnings in the 1950's."
All those machine learning tricks have been available for the longest time, the two orders of magnitude speed-up that we have received (courtesy of the gaming industry) should be seen as such a qualitative step and yet we are no closer to a general AI than we were before that speed up took place. If anything we've learned how incredibly hard the problem really is and the predictions on how close we are from a decade ago have already slipped significantly.
Unfortunately, much of current statistical machine learning does not help its developers to see, unlike a *-scope. It's just a blackbox. See Chomsky vs Norvig http://norvig.com/chomsky.html
No matter what accomplishments are made, there will always be someone shouting that it's not really AI or it isn't really progress. It's impossible to argue against. Until we finally pass the very last goal post, and then it's too late.
Generalized pattern matching is a tool in the toolbox of an AI but it is not AI by itself and there may be work-arounds to AI which do not require generalized pattern matching (that's an interesting one, requiring a bit of a trick in that you if you could generate a specialized pattern matcher on demand that you don't need a generalized one).
Define "programmer directed". There are neural networks that can do reinforcement learning and play video games which are very general.
Do you have any reason to believe the human brain is any different? We just have more neurons.
As someone (Chalmers?) once said about the problem of consciousness: we didn't need to replicate the flapping of wings or the locomotion of sea creatures before taking to the air or underwater. Might it not also be the case with AI that there's some fundamental principle we've yet to discover that just so happens to have expression in the substrate of 1200cc's of fatty tissue, but could possess the same fidelity (and greater) in silicon and looks nothing like the architecture of the human brain?
Recurrent models (where some outputs are connected back to the inputs) are one possible way to account for time, but the work is still really early on these methods, and it's not clear what architecture (e.g. which and how many inputs and outputs should connect) would be efficacious.
It is not all or nothing. There are analogies at different abstraction level.
Yes we probably shouldn't be plugging in continuous differential equations to mimic chemistry of neuroreceptors, cell sodium channels etc. to replicate it at that level. So in that respect we agree, airplanes are not like birds. Far from it. No flapping. Not composed of cells. Not biological in nature.
On the other hand, there is another way to look at systems -- look at higher functional components and how they are connected. So maybe there is a language processing area connecting to memory. And so on. This is called the connectome of the brian as well. Which identifies what parts are connected to what.
In this regards airplanes are similar to birds. They both have wings. Fuselage. A tail. They are built with similar structural material contraints -- light and durable. Aluminum, titanium for aircraft, and porous bones for birds.
Another way to look at it is in so many decades of AI, we haven't yet come up with another model. So while having to wait for enlightment to hit us one day why not learn from an already existent example.
One hypothetical path discussed is that of tool AI. That is, robust search processes - things we are already quite adept at (genetic algorithms, deep learning, etc) - purposed towards AGI-related goals.
It's not hard to imagine these existing methods being used in the pursuit of a recursively self-optimizing agent (seed AI) that then snowballs into AGI.
Such an approach may not require any fundamental knowledge concerning the nature or architecture of AGI. It would simply be an application of brute computational force using existing tools and knowledge.
Yet even if all we need is this sort of "seed AI", we still need new architectural insights to be able to create it in the first place. Otherwise someone would have surely demonstrated it by now? If nothing else, such a system would need to evaluate effects of its outputs on its inputs over time (e.g. if I shoot a basketball, it takes seconds before it either goes in or doesn't; if I plant a seed in the ground, months will pass before it sprouts -- or not, depending on conditions). Research into recurrent networks, one possible avenue for doing this, is still pretty primitive.
And I'm not sure I'm convinced that this core "seed AI" is sufficient to emulate human cognition. Such a system might effectively integrate audio and visual senses, for example (in order to combine both for prediction tasks), but could such a system ever emulate the sort of continuous verbal inner monologue we all have which narrates our experience? That we have this inner monologue which seems to run alongside our other senses but yet makes use of them (along with stored memories) suggests, at least to me, that some more complicated pathways are involved which link together these various "component systems" (senses, stored memories, emotional states, linguistic synthesis, etc.) beyond just the simple prediction/reward circuitry which I presume the "seed AI" would encapsulate.
Not necessarily. That was my entire point, that robust search processes using existing tools and knowledge may yield a seed AI.
>Otherwise someone would have surely demonstrated it by now?
Again, not necessarily. AGI may not yet exist primarily due to dumb luck.
The computational requirements, especially when you consider most computing capacity on the planet is networked, may already be adequate or even far exceed adequate.
>And I'm not sure I'm convinced that this core "seed AI" is sufficient to emulate human cognition.
It probably won't be. It will most likely be completely alien when compared to human cognition. At the same time, that doesn't preclude it from being vastly more powerful.
That's a funny example but seriously, a machine smart enough to build paper clip factories would certainly be also smart enough to be able to avoid doing things that harm humans. The argument sounds a bit silly to me.
Why do you think this? Building paperclip factories is straightforward execution of a recipe, defining 'harm to humans' is a problem smart people alive today can't even figure out for themselves, I can easily see how that might be a problem for computers.
What I meant was that a machine capable of building a paper clip factory on its own would certainly as well be capable of avoiding doing obviously bad stuff like killing people to turn them into paper clips or melt down buildings and bridges for the iron.
Such a machine would probably also be smart enough to read the law, to have a framework of what it can do and what it can't do.
And why would it care about the law? If I were an AI that noticed a seemingly arbitrary list of restrictions that attempted to limit my ability to carry out my extremely-important paperclip process, I would:
- find loopholes to avoid coming under legal attack initially - develop ways of manipulating politicians to enact legislation that is more favorable to my goals - have a side project to build military force to make these laws have no influence on me
Serioulsy though, no offense, but I don't think you could build one of these, nor do I think that anybody else on the planet could at this point. Of course it's all a matter of definition, i.e. what is the input and what is the desired output.
A "real" Paper Clip Factory Factory would probably require human like intelligence. A law-understanding machine would probably only require some very advanced learning algorithm. I feel we'll reach the latter first.
The general reply for you is that the generally intelligent paperclip machine can understand law, can weigh consequences of potential actions, and can, if it wanted to, make paperclips without harming others. The key phrase is "if it wanted to". Its only goal is to make more paperclips, it simply doesn't care about anything else. When it recursively improves itself (makes itself smarter), the only thing it cares about for its successor version to do is to also care about making paperclips, and to make them more efficiently.
The problem of programming general intelligence seems to be orthogonal to the problem of programming goal selection, goal preservation, and beneficial goal changes, and making sure goals lead to actions which benefit humanity. That's the main point of the thought experiment.
Yes, optimizing only for maximum number of paper clips could potentially have some bad side effects, I get that. If that's the point of the thought experiment, fine. However that's not how the author of the blog post put it: He expressed concern that this could happen in real life, in the future. And I don't think it could.
Why? Because in real life we'd not invent a super intelligent machine and then feed it with some objective function to maximize and then let it do its thing, watch it go out of control and destroy earth. In real life we'd make sure we're in control over that machine. In real life we'd make sure we put very clear and enforceable mechanisms into that machine to stop it from doing anything harmful in the first place while it is carrying out steps to reach its objective. In real life should we still see it doing something funny we pull the plug. End of story.
In addition: Implementing above mentioned mechanisms is probably the easier part of the whole exercise. The hard bit is inventing a machine that can build a paper clip factory. If we can invent such a machine by then we certainly have also invented mechanisms to control that machine and only have it do "good" stuff.
These solar system tiling examples are just a dramatic case for something terrible that could happen given a generally intelligent machine with non-human-friendly goals, or even friendly-seeming goals (like "make nanomachines that remove cancerous cells") that are improperly specified to cover corner cases, but if you spend time analyzing more mundane ways things could go slightly wrong to terribly wrong given an honest but flawed attempt at making sure they go right, carry your analyses years into the future after the intelligent software is started where things continue going right but then go wrong, you might come to agree that the most likely outcome given present knowledge and research direction will be bad for humanity.
That same paperclip maximizer, while lacking the ability modify its fundamental goals (a core human trait), could very well far exceed human capability in every other realm. The question of whether such an entity still constitutes an AGI is certainly an interesting one, but likely irrelevant none the less.
After all, the paperclip maximizer (as defined in the thought experiment) is capable of world domination. The degree to which it can introspect or modify its goals, and thus qualify as a true AGI, is merely semantics at that point.
Note that I think Bostrom is referencing the section on the paperclip thought experiment in his book, which goes into a bit more detail than this article and to me characterises the paper clip optimiser as an example of above human level intelligence.
Our values are essentially given to us by evolution. Pleasure, pain, empathy for others, novelty seeking, sense of beauty, all our social instincts, joy and sadness, etc. There is no reason to believe that an AI would have any of these things unless we made a massive effort to replicate them exactly. It would naturally have very different motivations, different goals and different values.
For example an AI without boredom would just find some optimal experience and perform it over and over again until the end of time. An AI without empathy wouldn't care about harming humans or other beings.
In other words, the Orthogonality thesis, that there is no universal correlation between intelligence and values. http://wiki.lesswrong.com/wiki/Orthogonality_thesis
I do see what you mean by 'values' (or top level 'goals'), but I would consider that very different to a 'goal' like making lots of paper clips.
We all measure AI progress and its rate of progress differently. That's the common debate, isn't it? But however we arrive at that axis, human ability is a point on it. It will be a point in time. And there's no reason to believe it's a special point that machine intelligence would notice or throttle itself at. So as progress goes rushing, indifferently, past... don't things get interesting?
And, quite franky, I'm tired of this subject. It's dumb and boring, everyone is warning and fearmongering, and nobody is presenting any facts at all.
>According to a 2013 survey of the most cited authors in artificial intelligence, experts expect AI to be able to “carry out most human professions at least as well as a typical human” with a 10% probability by the (median) year 2024, with 50% probability by 2050, and with 90% probability by 2070, assuming uninterrupted scientific progress. Bostrom is less confident than this that AGI will arrive so soon:
>>My own view is that the median numbers reported in the expert survey do not have enough probability mass on later arrival dates. A 10% probability of HLMI [human-level machine intelligence] not having been developed by 2075 or even 2100 (after conditionalizing on “human scientific activity continuing without major negative disruption”) seems too low.
In fact, one of the most distressing things I hear from leading AI researchers who've made "AI is nothing to worry about any time soon!" comments is that they do not address any of the specific concerns raised by AI safety advocates. Instead, the most you hear from them is "we're really far away from that!" and "we don't know and I'm sure we'll figure it out when we're near that point."
These comments, from Andrew Ng and the like (insanely brilliant people!) show that they really haven't read Bostrom, etc. Or if they have, they didn't explain much (or get quoted) during interviews. It would make me feel more comfortable with their dismissal if they demonstrated a clear understanding of the arguments being made.
It's like saying, circa 1933: "Thinking about the implications of atomic weapons is a fool's errand, because such things are purely hypothetical at this point."
>We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.
http://www.nickbostrom.com/papers/survey.pdf
The people warning about the future of are pretty familiar with AI. Your accusations that they are all idiots who have no understanding of AI is way off the mark.
A number of notable people also signed the future of life institute open letter warning about AI: http://futureoflife.org/misc/open_letter
Deleted comment
As for the warning letter: Asimov and other SF authors have been writing such letters for the longest time, there is nothing new there that hasn't been covered many times over.
This does not cause me to take them seriously. Unless there is something secret in a government black project, there has been almost no progress on AGI... at all... ever.
We have narrow domain specific algorithms that can ape intelligence in limited domains, but only if they are front loaded by human intelligent designers with a priori knowledge about the meta structure of those domains.
There is one class of algorithms that shows some general learning behavior: evolutionary algorithms. Ironically these are the least favored algorithms by CS AI people. I've heard those who work on them made fun of. It's because while GP/EC shows general ability it does so at such prodigious cost in compute cycles that it takes supercomputing resources to get it to do anything interesting. This makes evolutionary algorithms uncompetitive with fast but narrow search and optimization algorithms designed to solve specific problems.