Google scares me more than other companies because of their handle on AI. I wish they at least appeared to take existential risk around AI seriously.
Having Kurzweil near the top isn't exactly comforting, either.
Having Kurzweil near the top isn't exactly comforting, either.
That's just multiplying through by -1 the typical nerd wet dream about the Singularity.
I can't even keep Rails, Apache, or git from occasionally shitting the bed; I am not worried about AI just yet.
You can look at it that way, but I think it is currently far more likely than the "typical nerd wet dream about the Singularity".
Your last sentence is a complete non sequitur to me. You are comparing the output of a large, AI focused, for profit organization to an individual of unknown skill using some very specific open source tools.
I am not losing sleep over it, but I definitely think about it from time to time.
also not being able to keep git etc. running is some evidence about skill level
I don't think it unreasonable to brace for the case where some Skynet wannabe launches all nukes not because of a carefully considered decision motivated by hatred for humanity, but simply because it shat the bed.
Well said. What is it that AI is supposed to do that's so scary, anyway?
You would think an intelligent person would know this.
The site I linked to has some good introductory material to the related concepts. (http://yudkowsky.net/singularity/intro is short and to the point.) http://singularity.org/research/ lists some important texts to read, and following up on the authors will lead to more. (An author I will also recommend following for his practical pursuits is Ben Goertzel. Lastly, I think this is a good criticism of the Singularity Institute's general mode of operating: http://kruel.co/2012/11/03/what-i-would-like-the-singularity... But you should probably read that one last, if you intend to read anything at all beyond the simple introduction.)
That doesn't mean that they aren't taking it seriously, but it is at least a strong signal to me.
SPENCER MICHELS: Sergay Brin thinks the ultimate search engine would be something like the computer named Hal in the movie 2001: A Space Odyssey.
SERGEY BRIN: Hal could... had a lot of information, could piece it together, could rationalize it. Now, hopefully, it would never... it would never have a bug like Hal did where he killed the occupants of the space ship. But that's what we're striving for, and I think we've made it a part of the way there.
Which is kinda the point about AI safety. We don't really know what we want, we sure as hell don't know how to formalize it, and we're lucky that we don't know how to create a real AI because otherwise it could surprise us in a very ugly way, just lawfully following the plan we gave it.
Oh. Not rules then? Well, what rules. How about giving it a rule "children must not die". Apparently we're big on that one. Except, wait, we mean "US children must not die", so we can still bomb foreign suburbs where children are known to exist. But then we'll need "children must not die on US soil" because it is still possible that the union of (isa US minor citizen ?x) and (isa-resident baghdad ?x) is non empty and the AI would be unable to act. Of course then we need, "US children must not die except in a car", because otherwise the AI would have to destroy cars.
Ok, so lets abandon rule based AI altogether and model our AI on biological processes like humans. Well humans would never cause mass suffering of other humans. We have laws against it. Oh.
"hopefully"
"Let's play global thermonuclear war!"
Here's an interesting discussion about AI, rules, morality, etc. http://www.reddit.com/r/Futurology/comments/y9lm0/i_am_luke_...
All it would take is for the AI to just be smarter than you. Human psychology is hackable too. Cf. AI box experiment.
And I was, in fact, considering "AI in a box"-style scenarios. My point remains: it would be a very impressive AI that could break out. Presumably it at least needs some significant basic resources, memory at least, and CPU to do it in reasonable time, to do deep human-hacking. How can it convince us to give them to it if it doesn't have them to begin with?
If we're assuming we can control its goals at all (if we can't we're already doomed by virtue of trying), will it even occur to the thing to act beyond giving direct instructions to humans? Will it have enough concept of self to conceive of escaping, to say "I think therefore I am"? Can it even realize what kinds of computational resources it needs and how to ask for them?
There may be good answers to these questions, but I don't believe they're trivial.
More people, almost by definition, currently die from every other cause than x-risks.
Google is in the best position of anyone/any company to cause an AI disaster. That they don't _seem_ to care is what bothers me.
We're not perfect as humans, but I don't think we're stupid enough to build something we can't control. We don't even know what consciousness is, much less whether artificial consciousness is possible.
I do know that it would save lives to liberate humanity from drudgery and stupidity.
As for human stupidity, maybe you just have more faith in humans than I do (but I am not entirely pessimistic). Let me ask a related question, to try and gauge this more deeply: do you think anyone will ever release an engineered pathogen into the wild?
Are you serious ? Let's not overstate Google's position in this world.
They have a search engine, mobile platform, advertising platform and a bunch of popular websites. That's it. All of which have been done before for a decade now. Nothing they do directly influences whether people live or die.
I would be far more concerned about something like IBM Watson used in health care situations.
It's all about Risk Management 101: Risk = Impact x Probability.
Also best and brightest of engineers and AI researchers, shitload of money, culture of ambition and a clear intent to go after AI tech. I haven't heard of any other company like that.
> I would be far more concerned about something like IBM Watson used in health care situations.
I'd actually welcome it with open hands. It's about time for some automated diagnosis.
EDIT I'm sorry but I do have to address this point.
> Nothing they do directly influences whether people live or die.
Shut down Google and see what happens. Huge sectors of economy depend directly on search ability they provide, not to mention how many people are now using GMail. Moreover: do you know the addresses of nearest hospitals? Phones to medical specialists? How do you navigate around places? Without Google we'd be all back to Yellow Pages.
If there's one company humanity really grew dependent on, it's Google. Yes, you could probably replace most of the services if needed, given enough time, but the fact is they're the best out there right now, we're all using them and they definitely influence our lives and deaths.
IBM ? Who also have been doing it for decades and are far ahead of Google from all indictions.
>I'd actually welcome it with open hands. It's about time for some automated diagnosis.
My point was that IBM Watson if it made wrong decisions could directly influence health care outcomes i.e. life or death. Nothing Google does is comparable.
> Shut down Google and see what happens. Huge sectors of economy depend directly on search ability they provide, not to mention how many people are now using GMail.
Google is not the first or last search or email company. We would simply switch to Bing and Yahoo and the world moves on. Or have you never heard of Altavista, Excite, Lycos ?
> Without Google we'd be all back to Yellow Pages.
Hilarious you mention that since that's where Google gets its worldwide Local/Places search content from. So in fact we are already using Yellow Pages.
>If there's one company humanity really grew dependent on, it's Google.
Humanity doesn't depend on Google. Get a grip would you.
I imagine the CIA and/or defense contractors are in a better position- they have a ton of money, they work in AI... oh, and they also tend to equip their projects with a ton of destructive weaponry. Which gets deployed in foreign countries, sometimes without permission.
The key quote is at the very top:
"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." --Eliezer Yudkowsky
I wonder how EY feels when he reads threads like this. Maybe he just ignores them these days?