When the orders come from a human then there is someone who is ultimatly responsible, someone to put through the courts, someone to throw in jail.
When the orders come from a computer. Will any of that happen, will there be someone responsible?
When the orders come from a computer. Will any of that happen, will there be someone responsible?
Humans are very quick to abdicate their own moral authority, and it is the most repugnant human impulse that exists. If someone puts a gun to your head, and tells you to kill someone or die, you are still responsible for whoever you kill. Others might consider your actions understandable, even forgivable, but you're still a murderer and still one who valued their own life above anothers.
If you do something because you were "just doing your job", that's even worse. That's exactly identical to doing something for money. Which is usually looked very poorly on by society.
The danger I see is the stifling of potential human subjective decisions. That broad decisions will be made and carried out without a sanity/humanity check by humans.
I think measures should be put in place for a guarantee of human intervention in the case of situational anomaly or system error as well as an emphasis on redress.
Since automation scales and a human workforce doesn't, a guarantee of human intervention might not be practically possible.
Anyone have any ideas on how this kind of thing might be mitigated?