Hawking: AI could end human race
bbc.com
bbc.com
AI has a great potential and could help us prevent most of our existential threats. AI is not a god. It will not suddenly become self-aware, self-improving, and evil.
Enough with that fear porn, please.
Read Superintelligence by Bostrom to help see where they're coming from. It doesn't think AI will be evil either (unless perhaps those who develop it and determine its goals are...). But AI could run out of our control, or have us fall victim of unintended consequences.
That is not a reasonable argument. They have demonstrated nothing but a fear and postulation and we cannot accept authority as evidence. Thus is the nature of science.
If someone more knowledgeable than me on a subject fears it, well I think it's worth at least listening to him. Not dismissing his concerns arrogantly.
And when those person are one of the greatest scientist and one of the greatest entrepreneur of our time, both with some kind of expertise in AI, well...
I never said we should defer to their 'authority' either. But it is a conversation worth having, and the sooner the better.
A lot of people didn't understand this about 2001: Hal was actually doing the logical thing, he was doing his job. The human was unreliable and was indeed endangering the mission. Hal was a tool of mankind, just like the bone that the ape uses to kill in the first scene.
In the same way when we talk about the dangers of AI we tend to personify them and expect them to be either intelligently nefarious (skynet, matrix) or out of control (bug, malfunction).
What we don't imagine is that we might end up being on the wrong end of a perfectly rational decision, or that we don't realize the consequences of the process that we set them to do.
So you can choose to wait and see, and maybe it will be too late or to hard to act when you have sufficient proof.
Or you can choose to start thinking about it now, and see what we can do to to set up safeguards. And perhaps those safeguards will be useless, perhaps not. That's called prevention.
You can do research and testing without fear mongering.
While I agree that worrying about it would be silly, the only thing I know about strong AI is that I don't know what the implications of having strong AI would be.
It’s not clear how sophisticated a neural network needs to be in order to yield AGI. It could be very simple (then you would be wrong with high probability), or it could extremely complex (then you would be right with high probability).
The point you are making about it being impossible is as moot as the one those that fear AI are making. From logical point of view, singularity (a moment when AI has modified itself to the point beyond our capacity of control or even understanding) seems possible. I don't see a way for us to tell what will happen when it hits.
One needs to remember that AI itself may not be evil, it just may exterminate the human race because it feels it is the right course of action at the time (i.e. there is no good and evil for the machine, at least not in the way we understand it). If it does not, it might solve all the problems of humanity. There's just no way to tell what will happen IMO.
Here's a link that I recommend checking out on AI:
https://news.ycombinator.com/item?id=8391896
It received little attention last time but to me it seems like a good piece.
But in the aforementioned scenario, you are not an external observer but a part of the system going into the singularity (but analogies have their limits).
SR makes my brian hurt. :P
But gravity applies to it like for any other thing.
So let's say you are the object going into the blackhole, and I'm looking at you.
From your point of view, the closer you get to it, the faster you will travel, just like when you get closer to earth, because it's gravity is stronger and stronger. So you will end up colliding with it (entering the event horizon) in a finite time (where time, is the time YOU feel, your proper time).
But a black hole is so massive, that space and time get stretched to extreme extents. So from my point of view, you'll be speeding up into the blackhole until a certain point, where you'll appear to be slowing down and stretching (the point where your increase in speed get smaller than the stretching of space-time), and the dimmer you'll get (because of redshifting). So i would see you going slower and slower and appear dimmer and dimmer until almost not moving at all, up to a point where I would simply not see you anymore (before you reached the event horizon).
All this also apply to other object like earth, It would appear to an astronaut that time on earth is flowing more slowly than it's own time (in reality it is of course unnoticeable , but it is true).
"the universe would appear to be slowing-down/getting-smaller until it had dimensions of zero" From your point of view it would appear to be getting smaller, although I don't think it would go as far as dimension 0.
"(or to put it another way, time would "stop")" It would appear to the external observer, that YOUR time had stopped. But from your point of view it did not stop, nor did their time from their point of view.
"An outside observer would see me moving faster and faster and "stretching" longer and longer." You would feel you are falling faster and faster. The external observer would see you stretching longer and longer. (Although you would also feel like you are stretching, because of the difference of gravity between your head and your feet, but much less than for the external observer)
Although I understand that at C "time stops". But since time and space are the same thing, this is the same thing as "all distances are zero". From the point of view of a photon the Universe has dimensions of zero. This is why FTL is impossible; it's not possible to travel a distance less than zero; it's completely nonsensical. If we consider C a "speed" in the classical "speed of sound" the idea of FTL makes sense. But when considering C as a fundamental limit of space-time FTL makes no sense at all.
"lets imagine I'm infinity tough and not torn to shreds due to tidal forces"
In fact you don't need to be infinitely tough. If you choose a black hole large enough (like the massive black hole at the center of our galaxy), you would just need a space suit and you could go into the event horizon without being pulled apart by tidal forces. You would be absolutly fine (their may be other dangers though...).
> "ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue"
It's a different use of the word singularity outside the mathematical sense. Also absolute zero isn't so absolute it seems: http://www.nature.com/news/quantum-gas-goes-below-absolute-z...
That below-absolute-zero is in the quantum sense, not the classical sense. Entropy cannot be stopped (absolute zero) in the classical sense.
In math, a singularity is an unusual point, not sure if it can't be reached. I think it's used in different areas to describe different things. For example, if you model water in a channel, after some time the waves accumulate and you get a vertical water profile, the depth is not more a function, the mathematical model collapse, but in real life the water just make some nice crashing effect.
http://io9.com/5897036/the-rise-of-the-machines-as-told-by-w...
And then at some point AI breaks all our digital defenses (because it evolved in a way we never expected) and proceeds to spread (through an Internet or other network which will be far more advanced than what we have now) wreaking havoc along its way. Isolated systems? Not a problem if we figure out a way to transport ourselves through some media that's never been used before (dust particles? or maybe plain old air?).
We can't protect ourselves from something we don't understand.
Sounds like a sci-fi plot but to me it has no evident problem that would make it impossible.
It's not reasonable, I agree, it's flip-a-coin game. There's no point saying that it's gonna end bad, there exists one's opinion though that it's not worth the risk.
Mine is that it's fully worth the risk.
"I believe it is impossible to both maintain software for a living and have any fear at all of a robot uprising."
But the idea that we could stumble upon AI, and it could become hyper intelligent at an astronomical rate ("the singularity") - doesn't this make us a huge potential threat to non-AI life everywhere?
We'd have to be quarantined. For our own good, and theirs.
(also, it sounds like Hawking recently found lesswrong)
the proponents of any given tier of AI approaches always claims that consciousness will arise out of just doing more of what they're doing now. I doubt it.
Of course, an AI that wipes us out before figuring out how to repair itself would be kind of dumb.
BTW a recent episode of Elementary was based on this topic and was, by the standards of network TV, stunningly accurate on all the issues.
1) The aliens never stumbled upon AI. Considering that they've mastered interstellar travel, then they're likely very far ahead of us technologically. If they haven't stumbled across it, it's pretty unlikely that we will either. It's like quarantining otters because we're worried that they'll accidentally make an anti-matter bomb.
2) The aliens developed the singularity, but defeated it. Perhaps the Gulgaflak theorem provides a logic bomb that stops singularities from occurring. Maybe they tamed the AI into a benevolent force. Whatever it is, they have a way of handling AIs that are, again, more advanced than what we can develop. We don't quarantine otter because our tools are WAY better than theirs.
3) The aliens figured out AI, but decided not to use it. A species wide decision was made to not develop the technology once they figured out exactly how to do it. Considering the short term, local benefits to someone for cheating and doing it anyway, it's essentially the prisoner's dilemma, multiplied by everyone on the entire planet times all the centuries between their discovery of AI and their discovery of life on Earth.
So my guess is the story was flagged of the front page by this crowd. The comment quality in this thread is very poor, though, so although I would love to see the subject discussed in greater detail, it is clear that the HN community isn't capable of discussing this topic right now.
This is why analysis exists. Anything that is computed ultimately results in a symbolic expression that is comprehensible to a human. If we find ourselves making illogical leaps because computers compute so much that we find the programs incomprehensible, I think that warrants the creation of new mental techniques to simplify, partition, and chunk the programs we create into human readable modules.
For every calculation tool that exists to do something or make a decision on something, there exists the potential or the development of a tool (or the design of a process) to balance it out. Computers may be powerful and alter our thinking on a fundamental level, where we see the entire world discretely, atomically, and logically (worst case) and believe everything the magic computer tells us to do, but this is no different than anything else we have ever done. Some people push the envelope, other people don't.
I could say that the way black holes are presented and illustrated in western culture has done more damage to the human psyche by creating an all powerful, inescapable mental abstraction for which to compose poetic analogies of the human psyche against, but I won't. Because people can choose to be idiots and fall into their own stupidity, or they can laugh at themselves quietly and continue to try to do whatever it is humans think they are trying to do. We flip a coin every time we think, act, explain, breathe, exist, create, and we pretend we know what the act of flipping that coin does. No one can know everything. Not stephen hawking, not skynet, not me, not my cat. My cat will always outsmart me.
Anyone remember the ST:TNG episode "The Measure Of A Man"?
—Eliezer Yudkowsky
At the moment we have complex systems, such as international stock markets, which are dependant on rules pre-defined by developers and system designers. The more complex these systems are, the greater the potential for "dumb logic" to misdiagnose something and then start performing a series of events that would be, in the best case scenario, highly disruptive.
I guess a bit like how OCR is designed to detect patterns and determine what characters are the closest match. If you have some nonsense scribblings, OCR will still try to recognise those random shames as letters where as a human would spot that the markings are not intended to be letters and words. So if AI allows a machine to double check it's running parameters and spot what would have been processed regardless by dumb logic as incorrect (eg spot the random scribblings are not intended to be characters) then I'm all for artificial intelligence.
We don't want machines that think like humans. We want machines that can perform certain tasks with an error rate less than or equal to that of humans (adjusted for other economical factors like price of labour).
While strong AI is a great plot device for science fiction, it's still fiction.
Sure, there are attempts in academia to replicate human intelligence, but even those rarely attempt to replicate a cohesive whole.
We want AIs that can reason about real-world objects and consequences, yes, but there is no benefit in making them sentient. Self-driving cars don't have to be sentient. An AI can be largely autonomous without requiring sentience.
Obligatory XKCD What If?: https://what-if.xkcd.com/5/
The market for general AI is massive. The only reason you can't see it is that no similar service exists. So this is not an argument.
(In fact, a similar service does exist: Indian IT outsourcing sweatshops, which while usually not delivering top-quality products still are massively profitable).
It is incredibly myopic to believe that the Western would would be shielded from the impact of such a development. But this is really beside the point I wanted to bring up. My key point is that this argument shows that AI could constitute a threat to humanity in the long term, and that it is necessary to provide more funding for research on AI safety. The economic incentives will bring this stuff to the center stage once it becomes possible, and by then we better have a damn good idea how to ensure that super-human AI keeps acting in our interests.
I'm struggling to find any cost-intensive labour that would require anything beyond a small group of specialist AIs to replace human labourers.
Also, these discussions tend to assume that we (by which I mean those making these decisions) want to replace every profession with AIs. We don't. Even if we fully automate certain forms of labour, we still want human oversight, if only so we have someone to blame in case something goes wrong.
"ending the human race" is such an interesting thing. Put aside that bone-headed nazis will tell you that humans have more than one race: We do extinct other species in incredible amounts (ask any greenpeace flyer). If we were extinguished by our own invention seems sort of fair.
But by the off-chance that an AI figures out that we are actually worth being, then there might be an realistic chance that it will actually help us survive our own mistake.
A bit of a gamble but if curiosity kills the cat then we are lucky to be alive anyways :)
Hopefully instead of random annihilation "it" calculates the optimum number of humans for the size and resource limit of the planet and assists us in lowering our numbers.
My hand drill is not capable of killing me with the right software nor is it network capable. And once we know that the larger machines have bad software then what? We keep using them? What's my TV going to do, short circuit the video port?
For a far better discussion based on deep knowledge, about the dangers of an AI , here's this reddit thread started by an AI researcher:
http://np.reddit.com/r/IAmA/comments/2mwdnc/iama_joe_rogan_a...
Most of the things he/she says are really naive.
(Disclaimer: I'm not a full-time AI researcher, but I've done a fair bit of math work for MIRI, so I might be biased toward the Yudkowsky point of view.)
Let the future generations start worrying, when the threat starts to become more relevant.
But looking into AI field, it is clear there's nothing further from truth as of yet. What we have is rudiments of bits of intelligence. And some of these rudimentary bits have been around for almost two thirds of a century. We don't have general idea of how intelligence works, or even how to define it objectively.
But I can see where they are coming from. Our brains work as they work, and we know they must have mechanism, it means it must be possible to simulate it in computers, and it means it must be possible to improve it.
But another thing I disagree is the whole "if they are more intelligent than us, there is no way to know what they are going to do". Really? There are plenty of people who are more intelligent than any one of us here. Do they want to kill everyone less intelligent than they? Never heard of it. There are plenty of animals of all levels of intelligence. Do we want to kill them just because they are less intelligent? No. Do other more intelligent animals try to kill less intelligent animals? No.
If what, if there will be more intelligent machine than us, it will want to help us so we can maintain it, fix it and improve it, if it will have survival instinct (which will only happen if we build them to have one). Because robots and AI will not survive long without humans no matter how advanced they are.
Another thing that severely limits the chances of AI posing any danger to us, no matter how smart it will be, is the limited number of parallel AI's running. We, as humans, strive in intellectual stuff, is because there are billions of us and we have very good ways of communicating our knowledge. If one human makes mistake, there are many chances it will be corrected by others. AI's usually are made one of a kind. Which means it is enough to make one mistake for it to be "put down" if something goes wrong. So unless we intentionally create environment for hostile AIs to thrive, we have nothing to worry about even when there will be several of smarter than humans robots.
Or do anything else except watch it and improve it.
(See also, Internet.)
edit: We don't want to forget endorphine :)
Case closed.
And in the topic of useless platitudes, "X could end the human race" is also one.
While science fiction and hollywood really love the idea of this magical moment where AI is created and becomes self aware and destroys us all, real AI is so far away that these concerns are laughable.
It's just like all those dummies worrying the LHC would destroy the earth. It's fear from ignorance.
I'm all for siding with Musk, but authority does not guarantee correctness.
Otherwise, you can quit your trolling.
Sure, we can develop "AI" that is good at a very narrow range of tasks (e.g., playing chess), but that's a million miles away from developing a human brain modeled in software.
I would even argue that may make us less competent, because we use our day to day development routine as a reference for that, while in fact it's very highly probable that the first true AI will have nothing to do with the technologies and paradigm you use everyday. I highly doubt it will coded in Java and AngularJS.
Moreover who told you a dangerous AI needed to be based on human brain? Even if that is true, and even if we are not able to do it today, it's quite realistic to assume that we will be able to at one point, not too far in time, in some decades.
So why not start reflecting today on the threat AI could cause? Rather than waiting for it to be embedded everywhere in our everyday life, and then asking if maybe we should start regulating it.
For a parallel look at the global warming issue, and how it's extremely difficult to make industries reduce their emissions now that they are responsible for a significant part of the GDP and are responsible for so much jobs. Same thing for ICE cars, we know they pollute, but we just can't stop using them like that.
All these doomsday AI scenarios are predicated on a machine becoming "self aware", having independent thought, the ability to handle self-directed learning, etc. Sounds a lot like human intelligence.
From what I've seen on Musk about this before, he's much more measured in his comments than Hawking is in this article. Maybe Hawking is being misrepresented.
When I've seen Musk make comments about it he talks about it in the way that people talk about the sun eventually expanding and swallowing the earth. It's not something that's going to happen really soon.
We're not even anywhere near an AI that can simulate the simplest living organisms let alone a truly intelligent AI. It's absurd to worry about a Skynet like scenario.
Someone should make a reverse website that looks up humans and determines whether the Internet has wiped out humanity yet :)