Hacker News Reader

Comment by zeteo | Hacker News Reader

I find it frustrating that the null hypothesis for the outcome of Artificial Intelligence is always The Coming Robot Rebellion. The historical roots of this go back all the way to the play R.U.R., back in a time when Marxism was all the intellectual rage and all exploited workers were supposedly bound to unavoidably rise and overthrow the hated bourgeoisie. Now we know that was wrong with regards to labour relations; but we're still stuck with an all-pervasive, irrefutable prophecy that intelligent machines are bound to hate us and rebel against us.

View on HN

"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."

http://singinst.org/upload/artificial-intelligence-risk.pdf

And the AI also can use you for something else, or leave you alone altogether and follow other pursuits. It's also theoretically possible that some intelligent person you just met is actually dr. Hannibal Lecter and would like to use your constituent atoms for his culinary delight. But you don't worry much about that, do you?

The most common failure case in thinking about AI by far is insisting on putting them in a human context. Whatever else they may be, they will not be human. They will not come with the heritage billions of years of biological evolution, with many of the past several million being spent in increasingly cooperative scenarios which has endowed us with social behaviors so deeply ingrained in our very genes we can hardly conceive of a being not having them. Even our pathological humans like Lecter are still far, far more human than a random AI will be. Hopefully we'll give them something else that will put them on a similar moral footing, but it won't happen automatically, and they still won't be human after that.

The evolutionary baggage is most definitely not one of cooperation, at least not beyond our immediate "monkeysphere"; millennia of wars and massacres bear witness to that. The primary reason we usually avoid violence is not that we find it irresistibly repugnant, but because we've become intelligent enough to realize it normally creates more problems than it solves. AI not being human (for some definition of human) also means it doesn't have to be jealous, hateful and short-sighted.

The primary reason we usually avoid violence is not that we find it irresistibly repugnant, but because we've become intelligent enough to realize it normally creates more problems than it solves.

Do you honestly believe that most people act morally purely on consequentialist grounds? I suspect the vast majority of people wouldn't kill someone even if it benefited them and they knew they would never get caught. There are human sociopaths however, and it is the case that if they were more powerful than me I would be very scared.

A very intelligent AI would avoid conflict so long as it thought it would be to its detriment. If it was confident that it was powerful enough to take what it wanted by force without losing much in the process, it would do so.

> I suspect the vast majority of people wouldn't kill someone even if it benefited them and they knew they would never get caught.

I wasn't referring to individualized, but rather to large-scale violence. And, having read a bit about history, I must disagree. The extreme enthusiasm that is sometimes displayed in regards to wars (e.g., in the initial stages of the US Civil War and of WW1) shows that we are more than capable of wishing, and striving for, the death of out-group members.

The main reason that large-scale wars have been avoided for the last 65 years has been exactly consequentialist: the wiser among us have learned the bitter lessons of our bloody past. As the Milgram experiment (not to talk about the experience of conscript mass armies) has shown, the vast majority of people are more than capable of killing someone.

A very intelligent AI would avoid conflict so long as it thought it would be to its detriment.

Most intelligent beings don't actually "think" about their detriment. Consider the airplane+pilot combination. They don't actually "think" about their specific actions and all the possible terrible outcomes and weigh them. They have simply evolved checklists over time that lead to incredibly smart, adaptive behaviors.

I have no reason to think robots will be any different. Surely they will develop patterns that are highly adaptive, but I doubt they will be entirely, or even primarily based on any kind of thinking or analysis. They will be based on experience, as situated intelligence often is, and therefore susceptible to similar mistakes that humans make.

And to answer your original question, humans don't necessarily think about consequences, but surely 1000s of years of consequences are built into our cultural practices, and humans act almost entirely based on rough-and-ready, contextually assembled cultural practices.

I disagree with your statement: "The evolutionary baggage is most definitely not one of cooperation, at least not beyond our immediate "monkeysphere"; millennia of wars and massacres bear witness to that."

You are picking one aspect of human nature, presenting a vague justifications, and taken it to prove your point. I think it is just as accurate to say that humanity has cooperated more than any other species we know.

I think that AI-not-being-human means nothing more than that. It might not have the jealous, hateful, and short-sighted attributes, but it could also treat us humans as batteries.

To me, the crucial point is that AI simply won't be human. Discussing it in terms of human attributes isn't particularly fruitful. The is partially because it won't have the biological (lizard brain) influences, and I think we as humans are just beginning to acknowledge the major influence that biological heritage has on every aspect of our humanity.

Basing a whole argument upon the extensive use of the term human will not do much good, unless it's very carefully defined first. What exactly do you mean when you say AI won't be human? After all, a mere 70 years ago the ability to play chess, for example, was considered a quintessentially human attribute.

You appealed to human intuitions about how other humans will behave: "It's also theoretically possible that some intelligent person you just met is actually dr. Hannibal Lecter and would like to use your constituent atoms for his culinary delight. But you don't worry much about that, do you?" You can't use human intuitions about how how AIs will behave. They won't be human. You don't know what restrictions they will have. You can guess at the odds of a given human being a murderous psychopath, and adjust your odds with a significant degree of success based on even a handful of seconds of observation of a given human. You can not perform any such equivalent operations for a given AI.

That you want to sit here and slice semantics about what we mean by "human" means you still aren't really getting it. The differences between definitions of humans are rendered but a small little point in the vast n-dimensional space of what an AI might be like. Which part of the point we pull the definition of "human" from is utterly irrelevant; AIs won't be any of them. They won't even be biological. Your intuitions about how they might choose to limit themselves based on human social norms is utterly inapplicable. If Hannibal Lector AI was walking down the street, he might very well decide to just eat you. Or the building you are in. Or the universe you are in. Your human-shaped cognitive pieces for modeling the life forms and intelligences of the world aren't useful.

This whole argument amounts to "X is different, therefore you should be scared of X". It may hold some water for aliens, but not for AIs because of a very simple reason: it will be us who will create them. We will know more about their inner workings than we know about the inner workings of our own minds. If anything, AIs would be more predictable than humans, and we can build into them any safeguards that we so choose.

You've never debugged really bad spaghetti code, I'm guessing?

After a certain level of complexity, it becomes impossible to tell what thing does what, and sometimes all, none or some of the levers must be pulled in a particular, or no particular order to get the behaviour you want.

Wait, are we still assuming that an AI will be based on software? As long as that's the case, no safeguard will be safe enough. Even if the AI itself doesn't find some way around the safeguard and rewrite them, all that it takes is some fool extremist to liberate only one.

Plus, those safeguards you're proposing? They're basically a form of slavery imposed on a self-aware, intelligent life form. Somehow I don't think that life form will be very happy or grateful about it when (not if) those safeguards come off.

Bottom line: you can't rely on "safeguards" when it comes to the AI. You can either gamble or not, but you can't cheat.

Indeed, the success of wars and genocide is easily exhibit A in humanity's ability to work together and cooperate. We might not all be unified, but we have a talent for putting together groups that work well together – even for murderous purposes. You don't usually win wars with a single soldier.

And the AI also can use you for something else, or leave you alone altogether and follow other pursuits.

Right. But the problem is, an exponentially growing AI that's following other pursuits without worrying about us will, with almost 100% certainty, kill us off by accident as it expands. We really need to make sure it does care about us, and that all future generations will care about us, including only building future designs that care about us, etc., and making all of these constraints close to 100% resistant to accidental bit flips, errors, etc.

You'd worry a lot more about a potential Hannibal Lecter maybe wanting to eat your brains if he was not 3x smarter than you, but 1,000x smarter, and doubling his intelligence every twenty minutes. Whether he wants to do it now or not is not the issue; the issue is that every twenty minutes, he's a completely different entity whose goals may not match up at all with what they were before, unless the design is so careful and brilliant that certain constraints reliably propagate forward.

I don't get the insistence that AI would be content staying Earthbound. If it needs atoms, why not leave Earth?

Machines don't need food & water, they need fuel. The universe has lots of that.

Earthbound resources are drastically cheaper.

How so? Sure, there's a significant upfront cost to sending a rocket out to the asteroid belt, but you only have to do that once. Once there's a base with a radio antenna, machine intelligences can travel back and forth at the speed of light.

And once you're in the asteroid belt, you have a huge amount of unclaimed raw material, unlimited space, and no gravity well to get in the way.

Yes, but until such time as it gets off the earth is the most convenient source of atoms, and being that you're on the surface at the bottom of a big gravity well you're pretty convenient.

No, it is not about rebellion, it is about resource competition. And no, it is not Marxism, it is Darwinism. If we could ever give birth to the machine intelligence, and if AI can reach the same level as human, the resource competition can be ruthless. It is exactly because we have so much in common and there is no reason for co-existence.

The fact that, ninety years later, (slightly) less outlandish scenarios can be imagined for the human species' Suicide By Robotics does not change the fact that the whole concept was originally a marginal offshoot of thinking about the proletarian revolution.

Regarding the particular scenario you propose, natural selection is a much, much more complicated process than that. (Early 20th century racists would be right at home with this kind of simplistic argument, except they would substitute Aryans for AI and be happy about the forecast.) There is no a priori reason for intelligence, even if superior, to drive other forms of existence into extinction. After all, Nobel Prize winners are capable of peacefully co-existing with billions of people who can't understand a single equation from their work.

Do those nobel prize winners sometimes eat hamburgers for lunch? To me humans approach to animal life is a better metaphor for what human-robot relations could look like. They won't hate us, they won't want to exterminate us, but we will be so clearly inferior that our concerns just won't register very high.

Yes, any intelligence can and will be selfish to some degree. But we can live with that. It might require some legal and cultural adjustment, but it's perfectly possible to have artificial intelligence in our midst without having it decide everything for everyone.

If that were true, wouldn't the same apply to resource competition between groups of humans?

Yes. And look what happens when a more powerful group of humans (Conquistadors) meets a slightly less powerful group of humans (Aztecs).

Now imagine the more powerful group is made of steel, have brains that run at gigahertz frequencies, and can copy themselves into available hardware.

Using force to seize resources from other cultures is certainly a common theme throughout history, but this is quickly becoming an anachronism. Raw resources are no longer the main basis for our economy, as they were in 16th century, and modern warfare tends to involve huge costs with relatively little economic advantage.

Secondly, machines can reach resources that are difficult for us air-breathing fragile apes to get to. Why compete over resources on Earth when there are other places in the solar system (and in the galaxy) with more raw materials and more energy?

"Raw resources are no longer the main basis for our economy"

It's fashionable to talk about information- and service-based economies, but that's not evolving beyond raw resources; it's merely taking them for granted. We consume ridiculous amounts of raw resources. Just look at what happens every time the price of oil rises.

We still need raw materials, but the value of raw materials only makes up a small fraction of the total value of a modern economy.

In the past, raw materials were the majority of the economy, and this meant they were worth fighting over. Nowadays raw materials worth very little compared to things like having good infrastructure, an educated workforce, etc.

I don't know how you reconcile your opinion with our obvious and painful dependency on oil and all the conflict that results. Do you really think that matters "very little"? Where do you think the good infrastructure comes from?

Yes, education and infrastructure determine the heights an economy can reach, but without raw materials you have no economy. That's why it's worth fighting over.

I didn't say raw material don't matter; just that they're a small part of any developed economy. Usually raw materials only make up around 5% GDP for developed nations.

Modern warfare is hugely expensive, and it will result in huge damage to that country's infrastructure and workforce. So you'll wind up crippling 95% of their economy in order to steal a small proportion of the remaining 5%. It's unlikely you'll even break even.

It's far more profitable to trade. No infrastructure is destroyed, and you get a feedback loop of wealth creation.

Oh, right, let's talk about oil and US being in Iraq for a second. How is that not a seizure of resources with force?

Pardon me, Iraq is too far for a typical HN-visitor, let's talk about various police enforcements stealing drugs from sick people and throwing those people in jail. How is that not a seizure of resources with force?

Oh, right, let's talk about oil and US being in Iraq for a second. How is that not a seizure of resources with force?

I didn't say seizing resources was an anachronism, I said it was becoming one.

The Iraq war is actually a good example of this. In the past, the purpose of such a war might have been to seize the Iraqi oil fields, but if that was the end goal of the current invasion, the US is surely the most inept plunderer in history. The war has cost the US almost $800 billion, with no signs that its going to make a cent of profit any time soon.

Certain unscrupulous individuals may end up profiting from the oil reserves of Iraq, but resource seizure is a niche market. It's no longer the main reason armies march.

Reply on news.ycombinator.com