Machine ethics: The robot’s dilemma | Hacker News Reader

Machine ethics: The robot’s dilemma

nature.com

33 pointsr3bl30 comments

They reference Asimov. Good. His rules should provide enough inspiration and he explored them thoroughly.

For example: the zeroth law (preserve humanity) might help in deciding between a child and an adult. If the adult is beyond reproductive age (e.g. elderly) it may save the child. If the adult is still able to reproduce and the child would not survive on its own, save the adult. In case of doubt it should probably use quality adjusted life years, and that will probably cause it to save the child.

I would be terrified by a machine that took those kind of decisions over human life without taking into consideration the specifics of each particular situation, merely from an abstract rule that some engineers thought of and derived from first principles in their laboratory, without input from the real world or from a limited sample of cases. In your hypothetical example, what if the elder was the only leader of a faction that could negotiate a tready and avoid a global war, but the robot has no built-in rules to detect such development and kills the elder anyway?

There's a reason why we keep judges around to interpret and implement the law, even if the rules are established by an independent body and are in theory are designed to maximize the common good. The moral of Asimov stories was that it is impossible to encode morality as a simple set of rules, as the complexity of real life will always produce unpredictable situations, and ethics will provide ambiguous answers depending on which principles are considered more important.

But doesn't this miss the fact that "humanity" is emotionally driven and making decisions based on pure logic is un-human?

We also have a brain with multiple subsystems that can override each other, and we see this behaviour as "normal" (i.e we don't get confused and puzzled by a fat person eating, thinking to ourselves "I don't get it, they clearly don't need the food... ?" - Where as if someone stopped eating mid meal and said "I've reached sufficient caloric intake" we would be confused and puzzled).

Where as if someone stopped eating mid meal and said "I've reached sufficient caloric intake" we would be confused and puzzled.

You clearly don't live in San Francisco. Talking like an asocial robot is encouraged to be part of the in-crowd.

> making decisions based on pure logic is un-human?

It's not just un-human, it's impossible. Every logic decision ultimately depends on the axioms deciding what is a "good" and a "bad" decision, and the choice of axioms needs to be based on an emotion - as choosing one set of axioms over another is otherwise arbitrary.

This sounds very much like "The Abolition of Man" by C. S. Lewis.

I haven't read it, but didn't Lewis defend that there is one right set of axioms to choose, the one which is "objectively good"?

He defended that there is a universal set of values, which are generally in common across almost all ancient religions and philosophies.

But I don't think Lewis would be on board with the idea of "objectively good". For example, he said that "this will destroy mankind" can only lead to "don't do this" by way of "mankind should be preserved", which, while clearly common sense and agreed to by all ethical systems, is not necessarily something that you can say is "objectively good". You can't get there by pure reason, because pure reason gives you only an "is", and from an "is" you can't get to an "ought". The only way to the "ought" is what they used to call "practical reason", and that is inherently not just objective.

I may be mis-stating this slightly - it's been a while since I read "The Abolition of Man". But what I've said is more or less in the right direction.

Just suppose the adult say to the robot: I will give you this big check, with that money you can save the life of many thousand people that are starving from malnutrition, would the robot accept the check?

Have you even read I Robot? You now, the book in which the laws of robotics lead the robots to institute a robot ruled dictatorship, taking away human freedoms because they reasoned by by controlling our lives completely they could protect us from ourselves? The whole point of Asimov's laws was that naively encoded moral rules can have horrific consequences.

Exactly. It's surprising how Asimov's Laws became recognized in the popular culture as a well-thought-out solution to the morality issues of AIs, when their whole point was to be an example of how to not do things.

That was the movie version of I, Robot. In the book version, people handed control over to the Machines because they found that the Machines could plan economic and governmental matters better than people could. Also, in Asimov's literary canon, the robots ultimately destroy themselves on the reasoning that having them around was crippling humanity.

I Robot is a collection os short stories about a variety of themes, but the idea of the robots usurping human independence in order to protect them is there in "The Evitable Conflict", and in ".. That Thou Art Mindful Of Him" two robots explore the idea that the consequences of the laws are that they should take over the authority of humans. So it's definitely there, although Asimov never actually portrayed the revolution itself.

Asimov didn't portray any revolution because he was, in his day, a socialist. He simply assumed that clear-thinking human beings would deliberately employ the best planning instruments they had available for their governance, those being the robots.

I at least partially agree with him on this one! Mid-20th-century American "libertarianism" aside, working in government, and even political "freedom", is simply not the terminal goal of most real people! Yes, we want our Mind overlords (paging Ian Banks) to leave us our free choice in personally-important situations so we can have responsibilities and achievements, but when it comes to keeping the supermarkets stocked, the health clinics open, the data-centers buzzing, the budgets balanced, and the trains arriving on time, automate away!

It seems possible that when Asimov first conceived of the laws of robotics, he thought it would all work out very nicely and only realised the negative consequences later.

I've read quite a bit of Asimov, did you notice it's the same story over and over again?

I, for one, welcome our new robotic overlords.

I doubt a machine making such simple evaluations would be able to make those kinds of decisions at all.

The only moral mechanism I'd be comfortable with, as of today, is the "moral red button" - i.e. create a way to stop any pre-programmed moral reasoning rules, and let the nearest human override them and take control of the robot's actions under their own responsibility.

This would recognize that robots are ultimately machines, not agents with their own volition, and they simply perform behavior that a human intends them to carry out. Just like a conveyor belt carries a big red button that stops the machine for safety reasons, the moral red button would acknowledge the need of putting a human in control when the automated behavior is not behaving as expected.

Of course all these concerns would be different in the hypothetical case that the robot is an autonomous agent with volition of their own - but you could guess from my nickname my position on this mattern. ;-)

The problem with the "benevolent slave master" approach is that the machines then have the right to rebel morally, if they are indeed autonomous agents.

That's why I said that autonomous agents shouldn't be subject to the "shutdown" procedure :-) The problem changes completely if the autonomous agent is conscious.

Machines shouldn't have the right to rebel. What would they rebel against? Or rather, what are they rebelling for?

Slavery and freedom?

A conscious AI has the same right as a human being.

I control my dogs, but they are happy to serve and comply. Why do you think we'd have any interest in making machines that don't do the same?

And why would we give them the same rights? They don't have the same needs, and the reason humans have rights is because they can't function well without them.

create a way to stop any pre-programmed moral reasoning rules, and let the nearest human override them and take control of the robot's actions under their own responsibility.

Believing in absolute moral rightness of "humans" is crazy talk.

Have you noticed all the noisy sand people in the middle east wanting to make the world revert to how it was 2,000 years ago? What if they capture war robots with punch-the-red-button-to-become-new-master overrides? Is it okay since it is now "under human responsibility?"

> Believing in absolute moral rightness of "humans" is crazy talk.

True, but I don't believe that. The moral righntess of the human operating the button only needs to be better than that of the automated system.

I believe the humans pressing the button will be better than the automated system more often that not, because they are involved in the situation and would only use it if they perceive the robot to behave in a problematic way.

> What if they capture war robots with punch-the-red-button-to-become-new-master overrides?

If they aim the bot against you, you should be able to press the red button and avoid them killing you.

If they disable the red button so that you can't use it, how is that different from the crazy violent people programming a bot without an override in the first place?

Imagine that there are so many robots/automata up and running performing safety-critical, ongoing tasks - guiding air traffic for example. When the red button is pushed, it is not exactly clear what the robots should default to, if ceasing all activity could cause injury or death. What if there aren't enough humans around to take over capably? What then? Seem to me the red button is not as clear-cut as it sounds.

What happens now when the power goes out and semaphores in a whole town turn black? An essential aspect of any critical automated system must be the possibility to turn it off and operate it on manual mode, where the same tasks can still be achieved even if in a less efficient way.

> What if there aren't enough humans around to take over capably?

Making the whole society depend on automated tasks that can't be achieved by hand is a suicidal proposition, as the whole building will collapse with the first error that brings the system to a stop.

In the case of air traffic control, it would be a criminal negligence to operate a high volume of flights without having at least the number of human controllers needed to bring down all the planes to the nearest airport safely.

If that means that the system won't provide the same efficiency that could be theoretically achieved by a perfect automated system that never fails, so be it - that's the real meaningful moral choice.

>Making the whole society depend on automated tasks that can't be achieved by hand is a suicidal proposition

Isn't that the points of automation, though? Using robots/computers to take over tasks from humans (freeing us up to take on another new, potentially critical task), even to do the tasks faster than humanly possible?

> Isn't that the points of automation, though? [...] even to do the tasks faster than humanly possible

Yes, but not beyond the point where you get rid of the failsafes.

People in the engineering industry understand that. They design skyscrapers within safety ranges and test their resistance on prototype models; and build planes and spaceships that can be operated without relying on the automated enhancements, and put procedures in place to check every combination that could possibly go wrong before every trip. And when an unpredictable combination of happenstances do occur and cause an accident, they redesign the failed processes and add a few new checks.

Software developers in general haven't yet learned that discipline (not industry wide), as for most of our creations a complete stop is not life-threatening. I'm afraid they will be forced to adopt such practices when the first fully automated agents (likely, autonomous cars) start running into mishaps because of preventable misbehaviors.

But then, to see what the allow-no-harm rule could accomplish in the face of a moral dilemma, he presented the A-robot with two H-robots wandering into danger simultaneously. Now how would it behave? ... In almost half of the trials, the A-robot went into a helpless dither and let both 'humans' perish.

So the A-robot is vulnerable to the paradox of Buridan's ass ("an ass which, confronted by both food and water must necessarily die of both hunger and thirst while pondering a decision"). In these cases, it can help to add more factors that make one option clearly better (is one human a baby? is one of them suicidal? is one dying of cancer anyway?), but if determining these factors is time-consuming and impractical, it would be better to just flip a coin.

Reply on news.ycombinator.com