Should I be allowed to own C4 explosives and machine guns? Because I can use C4 explosives in a way that doesn't harm other people by simply detonating it on my private property. I am confused about what the limiting principle is supposed to be here. Do we just allow people to have access to technology of arbitrary power as long as there exists >= 1 non-nefarious use-case of that power, and then hope for the best?
> There's also the question of wether you're challenbging the state's monopoly on violence (i.e., national security) which will never apply to AI.
This misses my point about offense vs defense asymmetry (although really it's Connor Leahy's point). I'm not saying that AGI+person can overtake a government. I'm saying that AGI+person may end up like machine gun+person in the set of nefarious asymmetric capabilities it enables.