True.
> ... AI can't be allowed to be a decision maker
Non sequitur.
Part 2 only follows from the Part 1 if you can demonstrate that AI makes more errors, or more severe errors, than humans in the identical situation.
And for those situations, I agree with you.
But there are already many situations where the AI is making fewer errors than humans, and for those situations, it would be unethical to exclude AI (see Part 1).
Ah, they need to. Thanks for clearing this up.
If I pay a company to do something, and that company uses an AI to do the thing, and the AI messes it up, why isn't the company liable?
Let's assume you get paid 0$ for AI mistakes -- no liability at all. Then your EV is roughly:
(AI accuracy - human accuracy) * (cost of accident) - (1 - human accuracy) * (human payout)
Now I don't know about you, but even with pretty big numbers on the human payout scale and relatively small differences, I would not accept 10 million (presumably paid to my next of kin) to die. Does lead to weird situations where I may prefer humans for minor medical procedures where I can take the risk -- but only in edge cases.
Or is there some weird punishment thing going on here?
Ethically, we can easily justify refusing to delegate responsibility and accountability to a system that has no responsibility or accountability.