Loved that section about "meat shields". LLMs cannot be held accountable. Someone needs to be involved in decision making, with real stakes if those decisions are bad.
"Meat shields" has a nice physicality to it, though
What I had considered is that in the case of self-driving cars, nobody is criminally accountable, even though the rest of us may be criminally negligent should we make some horrific error. Philosophically, there is some kind of reason that criminal acts require punishment beyond mere financial liability (e.g., prison time) and self-driving cars are exempted from this. Currently, self-driving cars are also exempt from the actual laws of the road because the police are dis-empowered to enforce anything on the self-driving car.
Whether the tool is too powerful or ethical to use is an orthogonal discussion, in my opinion. Taken to the extreme, nuclear weapons still need someone fire or drop them. (We should still have discussions on safety and ethics always!)
Why though? I imagine things could also go the "google way" as well. The automated system makes the decision; and you just.. deal with the consequences if that decision is bad. We could just have an extended version of this dynamic as well: 3-4 entities with enough compute owning everything with no responsibilities; and when something goes bad, oh well. It's not like you can go to their competition, who also works the same way.
I think we saw many times that after a certain scale (after becoming, say, too big to fail), there is no bad decision you can't afford.