Comment by vonnik | Hacker News Reader

Comment by vonnik | Hacker News Reader

One of the most striking things about this piece is the difference between the claims of AI practitioners and pundits.

LeCun and Ng are making precise, and much more modest, claims about the future of AI, even if Ng is predicting a deep shift in the labor market. They are not treating strong AI as a given, unlike Bostrom and Nosek.

Bostrom's evocation of "value learning" -- "We would want the AI we build to ultimately share our values, so that it can work as an extension of our will... At the darkest macroscale, you have the possibility of people using this advance, this power over nature, this knowledge, in ways designed to harm and destroy others." -- is strangely naive.

The values of this planet's dominant form of primate have included achieving dominance over other primates through violence for thousands of years. Those are part of our human "values", which we see enacted everyday in places like Syria.

Bostrom mentions the possibility of people using this advance to harm others. He is confusing the modes of his verbs. We are not in the realm of possibility, or even probability, but of actuality and fact. Various nations' militaries and intelligence communities have been exploring and implementing various forms of AI for decades. They have, effectively, been instrumentalizing AI to enact their values.

Bostrom's dream of coordinating political institutions to shape the future of AI must take into account their history of using this technology to achieve dominance. The likelihood that they will abandon that goal is low.

Reading him gives me the impression that he is deeply disconnected from our present conditions, which makes me suspicious of his ability to predict our long-term future.

I think both can be right. Ng and LeCun are talking about the real near future. Bostrom always came across as a speculative SF + philosophy kind of guy. Are there any specific critiques of his (and Yudkowsky/MIRI, Musk, etc.) arguments? I think the two claims are plausible:

1. AGI is imminent: 50 years or 500 years or more from now. This is not too unlikely given that the brain is just an information processing system. Emulating it is likely to happen sometime in the future.

2. Such an AGI will be all powerful because it is not limited by human flaws. Trivial or not, we will have to program it with "thou shalt not kill" type values.

Here is a very good critique of Bostrom's book:

http://inverseprobability.com/2016/05/09/machine-learning-fu...

Our values do include dominating other humans. But they also include empathy, compassion, and morality.

Building a powerful AI without our values would be very bad. It wouldn't want to kill humans or dominate us. But it wouldn't care if we got hurt, either. And so it might kill us if we got in the way of it's goals, or if it thought we might be a threat to it (we could make other AIs that could compete with it, after all.)

So making an AI with human values - that is morality and caring about the existence of humans - is really important. If you build an AI without morality, it would just be a sociopath.

They aren't separable. The thing that makes you empathetic towards your own family, is also the thing that makes you go kill the other tribe.

"Morality" isn't an actual functional measurable thing, and there is no consistency across cultures so that is right out of the window. Culture has as much to do with behaviors as differences in the brain.

The difference between the "ethics/morality" of an un-contacted tribe in Papua New Guinea and those of Finnish royalty are staggering - yet they have effectively the same hardware.

People need to understand that you can't create something more powerful/smarter than you and expect that there is a way to ensure it won't kill you.

I don't know what your point is. Yes some humans are assholes. But most of us aren't. Most people don't want to hurt others. Most have the ability to empathize with others. This comes before culture, it's a part of our brains. Although cultural values are important too.

We want an AI programmed with our values. One that wants to serve us, wants humanity to exist, doesn't want humans to suffer, etc. Defining these values exactly is impossible, which is why solving this problem is so difficult.

You don't have to be an asshole to be dangerous though is the point. When you are thinking of hurting someone it's not typically due to malice, hate it's usually because it's a threat and out of self defense. That's a core part of being an animal/living thing, so it would be no different for an AGI.

Defining these values exactly is impossible, which is why solving this problem is so difficult.

I say it's impossible to solve because defining it is impossible. No two groups have the same end goals, so there will be conflict for survival at some point - and that is inherently dangerous to one group.

Even if you programmed an AGI to want humanity to exist, it could determine (probably rightfully) that humanity is the biggest threat to humanity, and just lock us into a pleasure loop as a species. These and many other scenarios have been discussed ad-nausea in which an AGI with human goals ends up giving us what we want and it's not something anyone would ask for, but would give the end result.

Ironically, this is what was described in the first Matrix movie - and if you take it to it's logical extent it makes sense. I'm not even a sci-fi person so it almost pains me to reference sci-fi, but I think it's apt:

Did you know that the first Matrix was designed to be a perfect human world. Where none suffered. Where everyone would be happy. It was a disaster. No one would accept the program. Entire crops were lost. Some believed that we lacked the programming language to describe your perfect world. But I believe that as a species, human beings define their reality through misery and suffering. The perfect world would dream that your primitive cerebrum kept trying to wake up from. Which is why the Matrix was redesigned to this, the peak of your civilization. I say your civilization because as soon as we started thinking for you it really became our civilization which is of course what this is all about. Evolution, Morpheus, evolution, like the dinosaur. Look out that window. You had your time. The future is our world, Morpheus. The future is our time.

In terms of hormonal emotions, which would be the most direct sense of value that we have is completely different. Love might feedback into aggression, but they are very different.

This is view is also very naive but it's the breakdown of the "good" and "bad" discussion. Nobody commits war crimes for their intrinsic value. This is akin to the saying, nobody likes to think they're the bad guys. Even, there are no bad values since the absence of value is the worst.

Naivety is not necessarily wrong, it's just the lack of detail which often leads to the wrong results.

I'm not talking about hormonal emotions. I mean calculated decisions like "should we bomb lybia because they are creating nuclear weapons."

That is a question about an (arguably) existential threat, where force is used for (arguably) protective purposes.

The question is, if the AGI thought that a human was threatening it unprovoked (which would probably happen) would it just let itself get "killed" simply because it was a human doing it?

reminds me of this comic

http://www.smbc-comics.com/index.php?id=4141

Zach has a lot of great comics that accurately represent how a real AGI would probably work.

http://www.smbc-comics.com/index.php?id=4122

Reply on news.ycombinator.com