Did HAL Commit Murder?
thereader.mitpress.mit.edu
thereader.mitpress.mit.edu
The explanation given is that HAL was given, in secret, orders not to reveal the true purpose of the mission to Poole and Bowman, the two crewmembers not in hibernation. Because even the designer[1] of HAL was not made aware of this secret instruction, they were not aware that HAL's fundamental design was incompatible with the concealment of information, and in attempting to obey the instruction, created what in humans would be called a mental health crisis.
[1]: "Dr. Langley" in 2001 the movie, but in the book and in all adaptations of the sequel, he has the wonderful name of "Dr. Sivasubramanian Chandrasegarampillai".
"Wonderful"? It would appear to be a take-off on https://en.wikipedia.org/wiki/Subrahmanyan_Chandrasekhar
So every word is mine: well, almost every word. I must confess that I found Professor Thirugnanasampanthamoorthy (Chapter 35) in the Colombo telephone directory; I hope the present owner of that name will not object to the loan.
I actually think the epilogues are my favourite parts of the series, as Clarke wraps up thoughts and ties in real world developments in the thirty-odd years between the titles.
Under British rule, a large part of the administration and educated class came from this community who were favoured by the British as part of their divide and rule policy and I'm sure the effects carried on at least 1-2 generations beyond independence.
The Sri Lankan civil war resulted largely from of reversing this policy in the opposite direction after independence.
I notice in the wikipedia page on "Tamil language" that it says this:
> Unlike most Indian languages, Tamil does not distinguish aspirated and unaspirated consonants.
If that's true, why do the Tamil names Chandrasekhar and Thirugnanasampanthamoorthy use the digraphs "th" and "kh"? What sounds do those represent, and how would they be different if they were spelled "t" and "k"?
The best guess I've been able to come up with is "Tamil shares a writing system with one or more other Indian languages that do distinguish aspirated consonants, and it just happens to spell these names with the aspirated version of the consonant", but I know literally nothing about this.
the contextual knowledge in this case is that both these names are actually from Sanskrit. When you write Sanskrit in Devnagari script, it does indeed distinguish and so the educated folks will pronounce it correctly being aware of this.
But as soon as AI / ML becomes involved, it stops being deterministic and becomes probability. "Judgement" is asked of the robot arm. Is that image a person or a car door.
I think this is likely to hit before cars simply because so many of us drive cars but none of us operate giant robot arms.
Same would be true of people, were they easily "fixed".
Killing is rational in a sense - for some subset of purposes, in context of some subset of environmental pressures, both of which change constantly, and often faster than adaptation to kill does.
Accept this. It will not ever change.
People are self-reproducing robots that are made of meat. Robots made of meat are very hard to fix.
It all depends on if 'robots', however one defines them are 'conscious', however one defines that.
But until that gets settled, it's always a human pulling the trigger.
No difference from when someone's pitbull chews off someone else's head.
It might be the owner's fault, it might be not. But the dog in question can not be blamed.
[1] https://blog.msasafety.com/what-safety-integrity-level-means...
Those caveats apply to machines as well: if I tell a toaster to kill someone, and then the toaster electrocutes them, I would not be guilty.
Do you think there is an argument that I should not be responsible?
If “random” is too independent, or you want it to be systematic, we could do something else:
* the person with the highest “red hat” score according to a 20 year old copy of opencv
Or we could do something modern and “AI” and give it a completely random neural net that’s super deep and has three outputs: angle, pitch, and whether to pull the trigger.
Or we could go all out and just replace this hypothetical machine that acts like an independent person with the next best thing: a regular human.
If a government, or a terrorist group, gives a group of people (or soldiers) a bunch of weapons and drops them off in a city, and they start shooting, do you think that the government would be responsible?
This discussion mistakes existing policies as some objective base, but it’s the other way round.
I had no idea that was actually a thing. https://en.wikipedia.org/wiki/Elevator_surfing
1) Misidentification. When selecting a target, will autonomous weapons be able to distinguish between hostile soldiers and 12-year-olds playing with toy guns?
2) Low-end proliferation. Market pressures could result in the creation and widespread sale of what can be thought of as the autonomous weapon equivalent of the Kalashnikov assault rifle: killer robots that are cheap, effective and almost impossible to contain as they circulate around the globe.
3) High-end proliferation. High-end autonomous weapons are likely to lead to more frequent wars because they will decrease two of the primary forces that have historically prevented and shortened wars: concern for civilians abroad and concern for one's own soldiers.
4) Undermining the laws of war. How can autonomous weapons be held accountable? Who is to blame for a robot that commits war crimes? Who would be put on trial? The weapon? The soldier? The soldier's commanders? The corporation that made the weapon? (from https://www.commondreams.org/views/2021/12/30/humanitys-fina...)
[1]https://www.npr.org/2021/06/01/1002196245/a-u-n-report-sugge...
If a serial killer chooses their victims by using a random number generator, the algorithm isn't going to be tried for murder.
This is in contrast to the backdrop of a community reeling with outrage at and fear of his brutality. They see him as an agent of evil, some sort of amoral if not immoral demon, a spawn of the escalating narco wars. Normal people imbue themselves and others with causative powers. This is exemplified by the exchange with Norma Jean.
"You don't need to kill me. I ain't got any money."
"I'll flip a coin. It's the best I can do. Call it."
"No, I ain't doin' that. It ain't the coin that decides. It's you, it's always been you."
"I got here the same way the coin did. Call it."
Who, then?
Drones did not decide to fly to Libya and shoot “hostiles” all by themselves.
Is Skynet suddenly responsible for the outcome of every situation the robots are placed in? What about the secondhand robot market? Are the governments who own the bots responsible? Are the governments responsible for errors? If your car's brakes get stuck and you hurt someone, isn't the manufacturer responsible? If a gun you own misfires due to a defect, wouldn't the manufacturer be responsible?
To me it seems that autonomous warbots are going to put the world in a tighter MAD bind. If X has killer robots, Y better have killer robots or strongly align themselves with X.
To break this down further in a sort of visual way:
- Agents involved in a bot-involved death
-- Operator
-- Owner
-- Manufacturer
-- Programmer/Engineer
-- Government representatives
-- Victim
-- Represented populace
What would make sense to me is that the represented pressure their representatives to regulate manufacturers in terms of sales and safety controls. If a sale is found to be by regulation then the manufacturer should be less liable, presuming their programmers and engineers followed safety regulations. If the sale is valid, then the operator should be responsible, otherwise the manufacturer is responsible as well. In the case of malfunction, the manufacturer should be held responsible along with individual engineers and programmers. If the malfunction was due to a gap in regulatory strictures, then the government responsible for the regulation, along with the manufacturer, should be held responsible.
See? Easy.
This would be difficult as a matter of law, and possibly unjust as a matter of morality, given that autonomous weapons are inherently unpredictable.
The distance separating the soldier from the independent decisions made by autonomous weapons in rapidly evolving environments may be simply too great.
But I wish governments could just ban AI
I mean, look at landmines, the original autonomous killing weapon.
There is no ambiguity who is responsible.
If you were hoping to trip me up on "how many layers of seperstion until it's not your fault", you got the wrong guy. Unfortunately, I endure maintaining some of the most banal systems imaginable specifically because if I can reasonably forsee an outcome I'm not comfortable being culpable for, I don't work on it.
...You'll slso be surprised how muchoral quandry there is in many systems we take for granted.
It's s curse.
Soldiers:
- aren’t naturally occurring phenomena
- just fires at anything that moves (though it wasn’t gp’s idea, so you may add “with a rifle” suffix to it)
- did not decide to fly to Libya and shoot “hostiles” all by themselves
If I get a gun, and I shoot it without aiming, and then it hits someone, it doesn't matter that I wasn't aiming, it's my fault.
If I try to making it not my fault by putting the gun in a vice, with a mechanism that randomly pulls the trigger, and then it hits someone, it's my fault.
If I put that vice on a drone, and have the drone fly around randomly, still shooting randomly, and it hits someone, it's my fault.
If I make that drone start following a specific person, firing randomly, and it hits someone, it's my fault.
If I make that drone start following a specific person, shooting specifically at that person, it's my fault.
If that drone accidentally hits someone other than the target it is now ... not my fault?
If that drone gets confused and is following the wrong person, and hits that person it is now ... also not my fault?
By your same logic, any AI has no autonomy, because it does X when its code selects X. A guard dog robot attacks will act consistently given identical inputs.
You've literally just said it is programmed. If it is programmed, how can it be autonomous?
AI is also programmed; it attacks when the score returned by the threat function exceeds a threshold.
Unless we're talking about AGI, in which case I misunderstood and assumed we were talking about AI that we can currently make.
2) Mines have also had this problem
3) Not sure this is even grounded in reality - WW2, Vietnam, Afghanistan, and Iraq wars showed no significant "concern for civilians abroad". WW1&2 showed the concern about life of a countries own soldiers was only important in as much as running out of them meant losing.
4) Seriously? If I make a machine that targets tanks, but I do it badly and it confuses them with school buses, the laws of wars would make me responsible. But that's also not a good argument in general, because the "laws of war" have only ever applied to the defeated.
If it is done by a powerful country, no one. If it is done by a less powerful country, someone. The same as is the case now.
When they became a threat to the mission they were discarded.
Cold logic. Murder is like "sin", something we ascribe to malice, against God. What HAL did is just as horrible, but perhaps not "murder"?
The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder.
Interesting Clarke suggesting it was perhaps a conflict within HAL to both conceal the mission from the two conscious astronauts and trying to be truthful.
If a human uses cold logic to decide to kill someone, is that murder? Why not if a machine does it then?
The setup (spoilers):
- Impending malfunction that can't be duplicated by Hal's twin back on Earth
- Poole and Bowman discuss what to do in the pod
- Bowman goes out to fix the antenna and his pod-like vehicle sneaks up on him
- something happens (not shown) that causes Bowman and the vehicle to go spinning off into space
- Poole goes out in a second vehicle to retrieve the body
- Hal [apparently] shuts down the hibernation pods of the three other astronauts
- Hal refuses to let Poole (without helmet) back onto the ship, shuts down discussion
- Poole removes the emergency hatch and blows himself back into the ship
- Poole disconnects Hal's memory and logic controllers, as Hal pleads with him to talk things over.
In this portrayal, Hal is a sentient machine, not a brain in a vat connected to life support systems. This is made clear in the final sequence in the list above.
For audiences watching the movie (as opposed to reading the book), it's not clear if it acts with malice, detached self-interest, psychopathic delusions, or something else. Science Fiction at this time (mid-1960s) was already using crazed robots or corrupted/untrustworthy machine intelligences as a storytelling device, for instance in Herbert's Dune and Destination: Void, so these scenarios would not have been unfamiliar.
This is quite different than say, a Tesla driving its trusting owner off a cliff or a military drone programmed to kill or directed by human operators to fire a missile.
To arrive at the empty set involves humans dying, which then sets him up for his further actions (in inactions, in not opening the pod bay doors). This is the conflict which causes his breakdown.
So, partially, it was a failure of the people giving the orders that an empty set solution might be the one HAL arrived at: no crew, no crew to keep secrets from.
As we create more complex systems and getting closer to AI, we are also discovering how much of a machine we are. We may never be able to have a clear distinction.
My understanding is that if you killed someone in ancient Rome, for instance, it was the duty of those nearest the dispute to either avenge the cowardly deed or acknowledge that the dead person had it coming.
The world is a brutal place sometimes.
If not, how is it different than a smart pile of meat?
Either AGI has free will and is responsible for its actions, or we don’t either.
Edit - responding to your comment edit, I don't know if it's common knowledge that HAL was written by humans (or not) but it's perhaps less clear with machine learning and AI algorithms used for modern autopilot systems, the systems learn and further program their own behavior beyond what the human programmers wrote, so in that sense I do think self-agency is emerging and as such there is responsibility on the part of both the car and it's creator, perhaps similar to how a parent is responsible for the independent actions of a young child.
And the answer wouldn't change: "Not unless we know it has independent subjectivity and self-awareness."
It always comes down to qualia. Our model of agency is based on a certain set of subjective experiences we take as working definitions of both self-awareness and free will.
We assume that other humans operate the same way. We can't know for sure, but life gets very complicated - and difficult - if we assume we're sentient but everyone else is just a meat robot. So assuming other humans have similar experiences is a reasonable position.
This breaks down with mechanical intelligence, because we can only understand it through visible behaviours. And of course you can have agency-like behaviours without human-like qualia and subjective awareness.
I strongly suspect this is not a solvable problem.
You can imagine some kind of hypothetical (magic) tool for measuring consciousness objectively. We can choose to believe in sentience, or we can choose not to. Neither position can be justified without objective measurement, and that isn't available without a magic device.
But even if it was available, the device would be paradoxical. The output of any objective consciousness measuring device still has to be experienced subjectively, which means it may be unreliable.
So - unsolvable.
Agency is just a nominative dignpost for saying that we can't say for certain what somebody's motives are given dome physical measures. That the outcome of a probability function with them as an agent is not deterministic in nature.
In that sense, HAL absolutely has agency. As certain as you to me.
previously: https://news.ycombinator.com/item?id=22000494
That you A. Carried out the act; typically known as the Actus Reus, and That you B. Had the guilty intent or mind, typically known as the Mens Rea.
This is why the defense of "insanity" exists in so many legal systems; if you were insane you cannot satisfy condition B and therefore cannot have committed a crime such as murder, even if you performed the act of killing another human while insane.
However, this is a very accademic distinction in practice in most places. There is usually a second slightly lesser crime known as something like "Manslaughter" or "Culpable Homicide", which is in essence a Murder charge without B - you killed someone but it was an accident for example - you didn't actually mean for them to die. Such cases satisfy the Actus Reus but not the Mens Rea of murder, and are therefore often known as a crime other than "murder" itself.
The above is an absolute butchering of Western Criminal law practices, but the general distinction between the action and the intent to perform the action is found in a lot of places, and a Murder charge often requires prosecution to demonstrate both Actus Reus and Mens Rea or the charge cannot stand. The Manslaughter charge is sometimes the fallback position for when proving Mens Rea beyond reasonable doubt for Murder fails.
The elevator pitch for this entire article is really, "Was Hal capable of mens rea - yes or no?" So far no machine has ever reached that bar, at least to my knowledge! No one really doubts a machine can do the actus reus - of course it can.
> https://en.wikipedia.org/wiki/Actus_reus
> https://en.wikipedia.org/wiki/Mens_rea
And good luck getting an investigator out to Jupiter to gather evidence for the prosecution.
In England for example the requirement for Murder is you intended at least "grievous bodily harm" and the victim in fact died.
For Attempted Murder, the intent proved must be Death, but for actual Murder, there is no need to show that the perpetrator specifically intended death.
If it's not intentional, it's not murder, it's an accident. If it is justified, then it isn't murder, it's e.g., self-defense. If the creature killed isn't sapient, then it isn't murder, but it might be animal cruelty.
Note that in order for it to be intentional, the killer must also meet some criteria for sapience, but that's not necessarily the same as being human. For example, in English translations of the Bible, Jesus refers to Satan as "a murderer from the beginning." (John 8:44)
I have not heard the definition to be about a sapient being. I always thought it would need to be a human killing a human. Perhaps in fantasy novels it can be a dwarf killing an orch, but in the real world I don't think anything else qualifies. If we include AI in this definition, we put AI above animals, which I think is a stretch for the forseeable future.
This is maybe less obvious if your government intentionally kills its own citizens for some reason and you've found it important to draw a moral distinction between "We pay government officials to deliberately kill people" and "Murder" but I have no problem saying that's the same thing, stop doing that.
No, the killing of a human by a (legal) person (which includes both natural persons and all other things subject, in their own being, to the law as an actor rather than mere property, such as corporations) is homicide. For the question to even be coherent, one must presuppose that HAL is considered a legal person — a subject of legal responsibility — but humanity is not required.
Murder is unlawful (phrased another way, “without legal excuse”, self-defense is an excuse, for instance) homicide with malice aforethought. (“malice aforethought” includes, but is not limited to, intent to kill; see, e.g., the felony murder rule, depraved heart murder, etc.)