Comment by karpathy | Hacker News Reader

Comment by karpathy | Hacker News Reader

Parent Full thread

I could make a lot of comments here but I'd just be reiterating what people much more qualified than I have already said. People who are at the forefront of AI and have actually developed state of the art AI technologies for many years, in some cases decades:

Andrew Ng: http://www.wired.com/2015/02/ai-wont-end-world-might-take-jo... "I think it’s a distraction from the conversation about…serious issues,"

Yoshua Bengio: http://www.popsci.com/why-artificial-intelligence-will-not-o... "What people in my field do worry about is the fear-mongering that is happening,"

Yann LeCun: http://spectrum.ieee.org/automaton/robotics/artificial-intel... "there are things that are worth worrying about today, and there are things that are so far out that we can write science fiction about it, but there’s no reason to worry about it just now."

Ryan Adams: https://twitter.com/ryan_p_adams/status/563384710781734913 "The current "AI scare" going on feels a bit like kids playing with Legos and worrying about accidentally creating a nuclear bomb."

Rodney Brooks: http://www.rethinkrobotics.com/artificial-intelligence-tool-... "Recently there has been a spate of articles in the mainstream press, and a spate of high profile people who are in tech but not AI, speculating about the dangers of malevolent AI being developed, and how we should be worried about that possibility. I say relax. Chill. This all comes from some fundamental misunderstandings of the nature of the undeniable progress that is being made in AI, and from a misunderstanding of how far we really are from having volitional or intentional artificially intelligent beings, whether they be deeply benevolent or malevolent."

The fear that the whole AI community is asleep at the wheel, and that we're unable to adequately extrapolate our algorithms is difficult to falsify. How can we possibly prove that AI is not some kind of magical emergent property, "one simple algorithm away"? We can't. All we can do is make our best educated guesses, and we're consistently seeing the same ones over and over from the people who are in the best position to make them.

Another in the same vein: http://blogs.wsj.com/economics/2015/02/25/be-calm-robots-are...

I'll add Ramez Naam's

The Singularity Is Further Than It Appears http://www.antipope.org/charlie/blog-static/2014/02/the-sing...

Why AIs Won't Ascend in the Blink of an Eye - Some Math http://www.antipope.org/charlie/blog-static/2014/02/why-ais-...

"Many people seem to believe that SMI would be very dangerous if it were developed, but think that it’s either never going to happen or definitely very far off. This is sloppy, dangerous thinking."

Or as the above posts demonstrate maybe it's really well thought out

SMI might enter this world as a defensive measure in the fight against terrorism and/or cybercrime. It seem obvious that the task facing intelligence agencies today is overwhelming given the vast volumes of data they would like to monitor. This abundance of trainingdata paired with nearly unlimited resources seems a plausible incubator for an intensionally paranoid and possibly hostile SMI. As others have pointed out: self awareness and creativity might very well be emerging behaviors. This would not be first time in history that weapons-research crosses a point of no return.

It's kind of unfortunate that the spectre of "AI doom" has blown up in the popular press the way it has, in that it's forced a lot of top people to push back against caricatured, sensationalist arguments in a way that distracts from the more serious conversations that ought to be happening.

Most of these people you cited actually have more nuanced views on the future of AI, e.g., LeCun and Bengio are (along with many others) signatories to the "Research priorities for robust and beneficial artificial intelligence" document (http://futureoflife.org/misc/open_letter, http://futureoflife.org/static/data/documents/research_prior...), which encourages research into AI's social and economic effects as well as research into technical questions of designing provably valid, secure, and controllable AI systems. Unfortunately when the conversation becomes "Stephen Hawking says we shut down AI research now before it dooms humanity", it becomes harder to express the belief that there are potential consequences of increasing machine intelligence that are worth understanding and planning for, even if we are not currently anywhere close to building superhuman general intelligences.

For what it's worth, expert opinion is not unified on AI risk in the way that it is on, say, climate change. First of all, most AI researchers don't spend their days thinking about long-term risks, so their main reaction is "the stuff I'm working on right now is such a toy that it has no chance of ever becoming a threat to humanity", which is probably true, but doesn't really make them an expert on the topic. Of the researchers who actually do think about this stuff, Stuart Russell is at least one example of someone who believes that there are serious questions to be asked (http://www.cs.berkeley.edu/~russell/research/future/, http://edge.org/conversation/the-myth-of-ai#26015):

"None of this proves that AI, or gray goo, or strangelets, will be the end of the world. But there is no need for a proof, just a convincing argument pointing to a more-than-infinitesimal possibility. ... No one in the field is calling for regulation of basic research; given the potential benefits of AI for humanity, that seems both infeasible and misdirected. The right response seems to be to change the goals of the field itself; instead of pure intelligence, we need to build intelligence that is provably aligned with human values."

Hopefully once the popular fearmongering dies down a bit, the field can move towards more of a sane, thoughtful conversation regarding what if anything we should be doing to prepare for the potential arrival of very intelligent (specialized or general) machines in our lives.

> How can we possibly prove that AI is not some kind of magical emergent property, "one simple algorithm away"? We can't.

I hope you aren't a researcher, because this is a ludicrous statement. We haven't studied it enough to know whether we can or not.

> I hope you aren't a researcher

Andrej is a brilliant researcher, currently doing his PhD. He has a bright career ahead.

Maybe you should actually read his comment instead of dismissing it crudely, and likewise for the thoughts of the likes of LeCun, Ng, Bengio, etc. These are the people I would listen to, not the Nostradamus pundits.

I admit, my first sentence was in poor taste. Apologies to Andrej.

At the same time, saying that something isn't possible to prove when we literally have no idea about its provability isn't a good stance for a researcher to take.

This is similar to my problem with the list of AI researchers you've provided. Saying that there is nothing to fear and waving your hands isn't exactly scientific.

Also, it isn't like these people (Sam, Musk, etc) literally think it could happen any day. The point is that we should be aware of the risk and prepare accordingly -- why is that unreasonable?

I don't think anybody would posit that it's unreasonable to entertain the idea as kind of far-fetched long-term possibility, much like encounters with alien life or faster-than-light travel.

It's the fear-mongering that's the issue. It's as if these same pundits were warning us about the dangers of space travel because it could hypothetically cause us (1000 years from now?) to draw the attention of a dangerous alien civilization (does that even exist?) that could destroy the Earth. It's the same level of ridiculous speculation. And that has no place in the scientific discourse.

Write sci-fi novels if you care about this issue, but don't pretend it's science, much less a pressing technological issue.

Reply on news.ycombinator.com