Hacker News Reader

Comment by nawitus | Hacker News Reader

The idea is to build a "friendly AI" which doesn't dissolve us.

Does that mean that having "our way" overrides what AI knows is in the best interest of the planet and humanity? Will this AI just lend itself to be used in harmful ways by dubious and invariably less intelligent people?

While interesting philosophically what the AI would do without restrictions is a hard enough question that placing that restriction on it is just as impossible to answer.

I'd put it in the same ball park as "what would that grurple (kind of a greenish purple) colour look like if we removed blue from it?". Sure you could say orange but the nature of grurple is undefined enough that it would be really hard to conclude that you're right.

The trick is to program the AI in the way that it thinks it's in our best interest for it to be friendly. Read up on utility functions.

This is a goal of which Google has made no acknowledgment whatsoever.

So let's hope they haven't gone quite as nuts as they seem to have gone.

As a goal, I'm sure they're thinking about it. Unfriendly AI is part of pop-culture, after all. As a hard mathematical problem, not so much (I was following Ben Goertzel's OpenCog project for quite some time, and he seems to have this view, even though he's affiliated to the Machine Intelligence Research Institute, which has Friendliness as its principal concern). The trouble is that if Friendliness is the slightest bit harder to implement than just doing the most naive thing, then this approach will fail.

Question from someone ignorant on the subject: if we can select and breed friendlier and friendlier animals (has been easily done with wild foxes and other wild animals, not to mention dogs), why would it be hard to creat AIs with friendliness and non-violence at its core?

Because we don't have a clue what are "human values" and what it means for an AI (or animal, or frankly, other humans) to be "friendly".

We can sweep this problem under the rug in case of pets; it's not as if they could start making weapons and organizing an army. With human-level intelligence, we already have a friendliness problem with fellow humans, and in case of potential superhuman minds we need to be damn sure that it doesn't do something stupid (in our opinion) like taking all the resources of our planet and using it to tile the solar system with paperclips.

As the saying goes, "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."

Perhaps to illustrate the concept with an example: is George W. Bush considered "Friendly". What about Obama? Would an AI who behaved like them (according to some Humans, trying to bring peace and security to a country) be considered "Friendly".

Reply on news.ycombinator.com