Did HAL Commit Murder?
thereader.mitpress.mit.edu
thereader.mitpress.mit.edu
If you want to see a movie where this problem is brought to the extreme and affects a human instead of a computer, watch The Bridge on the River Kwai. It's the (SPOILER ALERT) story of a British military commander (Col. Nicholson) who as a POW in a Japanese camp is ordered to build a railway bridge. Following his "western" standards regarding work ethic and craftsmanship, he and his men against all odds succeed in building a high quality bridge which he thinks is proof of the superiority of their western values and knowledge. At the end, he finds himself in a battle to safe this (as he perceives it) monument to the superiority of his culture against attackers - only to realize too late that because of a radical environmental change he could not foresee, the attackers are actually American and British forces. By defending his culture, he actively battled it. In the end, as he realizes his mistake, he is a broken man: https://www.youtube.com/watch?v=tRHVMi3LxZE&feature=youtu.be...
Whether a machine (HAL) will ever be capable of the insight of realizing such a mistake on a grander scale like Colonel Nicholson in the last minutes of the movie is another question :)
Hmm.. I've never made this connection any time I've watched the film. It's plausible, but is there any concrete support for it?
If HAL is a toaster that accidentally killed it's users, it can't commit 'murder'.
Point being, HAL has to effectively be sentient to commit 'murder' otherwise the death is the responsibility of its makers in one way or another.
The question is one of sentience.
The author I think makes a mistake:
"The first robot homicide was committed in 1981, according to my files ... a malfunctioning robotic arm pushed a repairman against a gearwheel-milling machine, which crushed him to death."
What's the difference between a 'robotic arm' and a piece of machinery? None! A robot is just a piece of machinery.
So it's really an existential question of conscience - if them machine doesn't have one, it can't be 'their fault' because there is no 'their' to begin with. Just some plastic and metal.
Seriously though, the author of the article seems to be under the impression HAL made a mistake, but that's not the case. We know from the novel and from the 2010 sequel exactly why HAL did what it did, and under the circumstances HAL's actions were inevitable. It followed it's instructions and operating criteria precisely, it's just that the people giving it those instructions didn't realise what they were doing, and didn't confer with people who were qualified to understand the consequences. It really was human error, that's the point Clarke was making.
I suppose theaters and arthouse cinemas all over the world have similar initiatives.
They get the film from the studio with the soundtrack removed, which makes me think this is a thing other orchestras have available to them as well.
The search for a solution space gave an empty set when it came to crew members and that was that. While HAL undoubtedly had some life-preserving behaviors in his goals, he would -- in such a critical mission -- have had programmed in an escape hatch if presented with a kind of trolley problem wherein a crew member might live (but the mission fails) or selecting to kill a crew member (by action or inaction) and allowing the mission to go forward.
Then, by induction, he continues to subtract one from the N crew members until he would have a solution that satisfies the constraints: zero, the trivial case.
Each step of this comes down to humans.
There are also many examples of people being exonerated for horrible deeds because they were "just following orders".
It's the same thing as if an officer during war told you to retreat, however you still defy that order to rescue a fallen comrade in the presence of immediate danger.
An AI computer cannot commit murder. A monkey cannot commit murder. A rabid dog cannot commit murder. Only a human can commit murder.
So if HAL was programmed to kill, HAL did not commit murder, however maybe those that programmed it did.
It's more likely that HAL would be treated under a different, but well understood area of the law: products liability. If a chain-saw malfunctions and kills you, you can sue the chain-saw manufacturer, and if egregious enough maybe someone at the company will go to jail. If a self-driving car kills you, you can sue the car manufacturer, or maybe your insurance company will. If an AI named HAL kills you, you can sue whoever created it.
A computer accidentally or intentionally killing someone is not murder. It's a malfunctioning device, and the people who created it should be at fault, not the device itself (kind of like sending a chain saw to jail).
I'm constantly surprised at how confused this topic is.
> Elon Musk, Stephen Hawking, Sam Harris, and many other leading AI researchers have sounded the alarm: ...
With all due respect (and as a theoretical physicist myself I have immense respect for Hawking at least), none of these are “leading AI researchers”. They are pundits, celebrities even. If you want to appeal to the authority of leading AI researchers, at least cite one actual leading AI researcher, or you come off as someone who has read too much tabloid.
I mean it honestly grinds my gears that people even use the marketing term of "AI" today when what we have is not even close to the actual thing. Intelligence is something that can freely think and make decisions on it's own. A complicated algorithm that can interpret data to do something automatically is not AI. It is confined by the code that it is given and that it is programmed to write for itself. It cannot pause and think "We'll what if I wanted to do more than this? How would I write my own code to do that?"
Computers are getting better, sure but they are improving at a slower and slower rate. Moore's law has been dead for years now. Maybe the singularity is indeed coming, but what if instead of the typical 20-50 year predictions, it's instead 2000 years away. What if the 20th century's exponential curve was really just a blip in the long term, and we're headed into centuries of slow linear growth.
The likelihood of this happening is low, but far above zero. I'm amazed no one is talking about it.
I've given up on the term AI in popular media. When it's mentioned there is often interesting research or product development going on, but the term "AI" generally obfuscates it.
I almost want to give a pass to companies that use the term: these companies want to maximize profit, and if a nebulous marketing term helps them do so, it makes sense that they would.
What is more frustrating is that the press has totally bought into it. The point of a free press is to be critical, investigate, and poke holes in the marketing veneer. Sadly, the press seems to have adopted the term whole-heatedly without much thought, which in turn has added pressure to companies to market products as "AI".
Even worse, most tech journalists are actually smart and can spot the BS, they just choose not to.
If a lion eats a human, if an ostrich lands a lethal kick while protecting its nest, it’s not murder. HAL cannot commit murder.
> In the law of torts, an act of God may be asserted as a type of intervening cause, the lack of which would have avoided the cause or diminished the result of liability (e.g., but for the earthquake, the old, poorly constructed building would be standing). However, foreseeable results of unforeseeable causes may still raise liability. For example, a bolt of lightning strikes a ship carrying volatile compressed gas, resulting in the expected explosion. Liability may be found if the carrier did not use reasonable care to protect against sparks—regardless of their origins. Similarly, strict liability could defeat a defense for an act of God where the defendant has created the conditions under which any accident would result in harm. For example, a long-haul truck driver takes a shortcut on a back road and the load is lost when the road is destroyed in an unforeseen flood.
It's also so subjective of a question it is not really useful as a philosophical tool either. I describe one criteria for it being "murder", but reasonable people might easily come up with other, equally defensible stances. So interrogating this question doesn't reveal much more than societal divisions along semantic lines.
I think the more interesting questions are things like: what do we do about murderers? Punishment for its own sake? Deference? Rehabilitation? What is the primary goal of our legal system?
Surely knowing that words mean different things to different people in the same society, and knowing what they mean, is quite valuable knowledge!
> what do we do about murderers? … Deference?
Is there any chance that 'Deference?' was a typo? I can't seem to make sense of it.
>Deference: Oops, that should be deterence
So we come to HAL: it could be fully self aware but have as little choice as a light switch, or at least have such little choice in some circumstances. This is why I think sentience may be a necessary, but not sufficient condition for "murder". The ability to have chosen otherwise is, I believe, also a necessary condition.
The story would have been uninteresting (or at least very different) if HAL had been acting on direct orders, and Clarke does not seem to have in mind that the mission was deliberately crafted with the intent that HAL would take homicidal action on its own initiative. Instead, HAL, in an unanticipated divergence, is confused by the apparent inconsistencies in the mission plan resulting from its goals being concealed, and, "like a neurotic", unable to recognize its own state, concludes that the astronauts must be killed for the greater good of the mission.
If, by "following orders", you are saying that HAL is a deterministic machine, then we are getting into the issue of free will. For what it is worth, Dennett is a 'compatibilist', maintaining that free will, in some sense, is compatible with physical determinism, but I don't know enough about that viewpoint to comment further. "Free will" is not explicitly mentioned anywhere in the article, I would guess because it is too deep a rabbit-hole for an article of this length and scope.
I suppose we'd have to subject HAL to some kind of 'reasonable AI' test.
Interesting stuff here: https://en.wikipedia.org/wiki/Diminished_responsibility
As with many legal matters, it's far from black and white.
Q is, is HAL a person
https://www.nspe.org/resources/ethics/code-ethics
This is something I've been wondering about the Software Engineering profession. As software becomes more "blackbox" or AI if you will should this profession be required for licensure as well. Especially if the software design is used in public service.
You want to put a guarantee stamp on something at the risk of going to jail for missing a bounds check at some point in your life? Go ahead, I'll pass. This kind of requirement would kill the industry overnight.
Software is not engineering. Software engineering is a hilarious oxymoron, and I say this as an accredited engineer. There is no way to guarantee that software will work 'properly', and 'properly' is only a small subset of 'as intended' which is a small subset of 'as envisioned'. It could be that software never becomes engineering. And until it does, holding it to the same standards as engineering is pointless.
It can be. There are ways to prove a program is correct.
This in itself is rather limited. For example, a formal prover would know nothing about something like rowhammer, and would happily accept code that seems to work as intended but also executes a rowhammer attack on the side which completely changes the output.
Ignoring that, this definition of correct is entirely divorced from the level where requirements exist, which is as envisioned. An error in requirements or an error in design can still very well yield formally 'correct' code.
There's no way around this today, and there's a pretty strong argument that this is an inherent problem to which no solution exists. This is precisely why NASA has lost missions despite being one of the most prolific users of formal provers.
They proved the code works as designed, but that says nothing of the design being correct, which says nothing of the intent of the design being correct.
Also see: https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...
Mechanical and electrical systems can also be built correctly to an specification that's ultimately inadequate.
Computers are programed to do whatever they do, so they have no choice in the matter.
Humans, on the other hand, have an immaterial soul, and thus can make decisions that are not determined by material constructs.
https://scholarship.law.upenn.edu/cgi/viewcontent.cgi?articl...
For it to be murder according to criminal law, at least the engineer responsible for the AI, or the architect, have to be found guilty of purposeful negligence of monitoring and reporting proper functioning/behavior of said AI. So, it's not the AI that is "guilty", but their makers.
The criminal law is not applicable for AIs, but for their programmers/makers, for now. For the laws to be applicable for AIs, they need to be competent or sentient enough for us to acknowledge them as equal to us from the perspective of rights, and therefore law (similar to age of majority). Until then the engineers are responsible for whatever an AI does wrong.
However what is important is what has to be done to protect society.
The question is does the system merit exile or destruction or some other remediation to restore order.
The answer is based on evidence and observed behaviour... yes. HAL should be decommissioned.
It is exactly similar to the question “is the 737MAX guilty of murder?” Guilt and murder are concepts related to conscious thought but the behaviour and outcomes is what really matters in that case.
The reason why guilt and murder matter is as humans, we have instincts around managing each other that these concepts extend with the power of Reason and Process.
If it was a different species such as a sentient ant colony, could we project the concepts of guilt and murder equivalently? Probably not.
Mapping this onto complex systems is hard, but usually we presume that the system is under the control of the designer and therefore the intent or carelessness of the designer is what matters.
AI systems offer the disconcerting possibility that at some point the designers will start explicitly disclaiming fore-knowledge and therefore responsibility for what the system is doing. Nontransparent neural networks are a particularly awkward problem here.
Exactly my point. When there is a black-boxed AI involved an engineer (as of now) cannot precisely determine whether it functions properly (according to specifications, from which safety rules are derived) or not. So, the monitoring technology is not as advanced as it should be to let black-boxed AIs handle important functions. So, until any AI is not as competent as we are, we should not employ black-boxed AI in algorithms at all, because it is irresponsible (before the law, as consequence). We cannot say it (the AI) is responsible for odd behavior, when we programmed/trained it that way. Same goes for children, parents are responsible for their children. Gross negligence (like letting the kid skipping school repeatedly) is punishable before the law, and parents are responsible, not the child.
There are many cases where parents are clearly not held responsible by law, including outright cases of murder. (Usually considered as manslaughter with equivalent of insanity defense.)
Unless you are of the position that everyone is guilty, by induction.
In case of HAL, the negligence case is clear because there existed a perfect simulator - SAL. SAL was not given the secret orders, which is clearly endangering the crew. Whoever is responsible for that difference clearly knew that changing the orders made the simulation invalid. It's similar to Boeing 737 MAX case.
So, yes, with untested scenarios (like "secret orders") responsibility fall into the hands of those who imposed the orders and omitted crucial tests. It's not even the engineers fault anymore.
Sin and moralism is an instinctive feeling that presumably corresponds to risk management.
With an AI, intent (meaning reasoning or modelling) is also measurable.
So taking 737MAX again, because the software will reliably and consistently lethally fail, all planes globally were grounded.
If it was the case of metal fatigue in one aileron then the entire worldwide fleet would less likely be grounded.
How do you know HAL wasn't at that level?
Moreover, there are a couple of poignant scenes in the film 2010 The Year We Make Contact:
Dr. Chandra has the following dialogue with SAL-9000, an AI in the same class as HAL. He is disconnecting SAL as a test for what will happen with HAL:
"Will I dream?"
"Of course you will. All intelligent beings dream. Nobody knows why." [0]
"Will I dream?"
"I... don't know" [1]
----
[0] https://www.youtube.com/watch?v=T2E7sxGAmuo
[1] https://www.youtube.com/watch?v=3OlCzxFuV9cThat one bit literally deprives him off free will in this matter.
We can argue that HAL was too dumb to find other alternate solution. If that is the case, he would be tried in this matter like a child or intellectually deficient person, perhaps an unusually smart animal. Law determines if that applies by reference group and expert opinion.
I think this is in fact answered by 2010. HAL is capable of refusing to follow orders, or at least needs to be convinced to follow them, as evidenced in the second youtube clip where Dr. Chandra -- HAL's creator in the movies -- needs to convince HAL it must sacrifice itself (along with the Discovery) in order for the mission to succeed. HAL initially wants everyone to stay to study the phenomena and must be convinced otherwise.
And of course, there's the classic "I'm afraid I can't do that, Dave" when ordered to open the doors of the Discovery in 2001. Though in that case it could be argued Dave Bowman no longer had authority over HAL and therefore his command wasn't really an order.