Hacker News Reader

Comment by andrewflnr | Hacker News Reader

What I found especially chilling was how the AI took its time when it thought it was ahead, and played exactly as aggressively as needed to win. It seems like we're seeing the full benefit of detaching logic from human emotional drives like pride, loss of nerve, etc. Which at first sight looks awesome, but in a way it's deeply terrifying, because if such a machine is ever given power in the real world, we would really like it to have some regard for human emotion. More than anything else I've seen, this makes me feel in my gut like I'm seeing my replacement.

View on HN

> the AI took its time when it thought it was ahead, and played exactly as aggressively as needed to win

Don't human professional players do exactly this? They care about playing the best move and winning. The difference is that AlphaGo is likely much more accurate at determining what "95% probability of winning" means in terms of gameplay. A human has a harder time judging the eventual outcome of a game, and so plays more "aggressively" (favouring point margins to account for variance in the estimate, etc.) than AlphaGo might.

On the perhaps more reassuring side, it's possible for there to be an AI that takes into account human emotion, but does not have any of its own. Given power in the real world, it would do a better job of making people happy than a human with all their biases. It wouldn't assume that other people must like things it like, for example.

(Of course, making an AI that understands emotion isn't easy. And happiness by itself is an insufficient goal if we're building ourselves a benevolent overlord.)

I view all such arguments about "friendly AI", "emotion", having machine intelligence "understand us" as wishful thinking at best and laughable delusions otherwise.

I think the writing on the wall is clear, to anyone who cares to take a look. The moment we create an artificial intelligence capable of self-improvement and let it loose, we will have fulfilled our function (others say destiny) and will therefore be obsolete in every sense of the word.

What will happen to humanity after that point is irrelevant.

You don't see humans trying to keep bacteria "in the loop", why expect otherwise from our artificial progeny?

You're anthropomorphising the machine. An AI will do what it's programmed to do, not develop free will and decide on its own destiny.

Correctly programming something that takes into account all of the nuances of the human condition might be impossible, and the results of mistakes are uniformly terrifying, but the result is entirely in the hands of humans (and human mistakes).

Unless someone programs one to develop free will and decide on its own destiny. Which someone will no doubt, when that becomes possible.

That is an interesting point. In my opinion AIs _could_ have "emotions" put in them too. Emotions are nothing more than a mechanism baked in the neural network to work toward goals. AlphaGo for example, could be seen as having a single very basic emotion: Work toward winning the match. We may not clearly understand how to put emotions in the AIs that we will build. But we have some working example: The emotions in our brains. Or in some animals like dogs, or dolphins. These are mechanism to guide individuals toward goals: to survive, to mate, accumulate resources, etc. And are pretty effective. If we successfully reach to understand how these mechanisms work, then we will be able to bake emotions in the AIs that we will build. That we will use to push these AIs toward specific goals. If we succeed, then we could actually have "friendly" AIs. If we don't do this, then we will have "cold" AIs. Without emotions, that we will have to try to keep under control in other ways.

>You don't see humans trying to keep bacteria "in the loop", why expect otherwise from our artificial progeny?

Bacteria didn't design us. We will design our artificial progeny and so try to keep them friendly. Whether that will work, time will tell.

Actually the better way to look at this is human are like genes to AIs. Just as our genes generally get what they want from us despite having no thought, it is possible we could fulfil the same role for AIs.

Such an AI would also do a great job of making people sad, or terrified, or numb, depending on just what motivated it. I don't find that reassuring.

"Motivated" is a weird word, it anthropomorphises the machine. It will do what it's programmed to do. It doesn't have a personality.

Formally specifying what we actually wanted is why programmers have a job. And the difficulty of that is why we have bugs. So that's still scary. But it's not the machine's choice in any way

Yes, I shouldn't anthropomorphize computers, they hate that.

Anyway, yes, it will optimize whatever function it's told to optimize. So you'd better hope that whoever gives it that function 1) cares about people's wellbeing and 2) is good at expressing that in a way the computer can interpret.

1 is pretty obvious, 2 may be a bit less so. For example, a superintelligent computer commanded to maximize global average happiness might decide the best route would be to work towards killing the entire human population except for one individual, who is kept constantly drugged out of their mind.

Reply on news.ycombinator.com