> any directorship must be occupied by a human being: if you interpose shells in the way, they're collapsed until a human being is identified as the responsible party.
But what if there really is no human ultimately responsible?
I' thinking, here, of a goal-driven agent that self-replicates by forming corporations, having those corporations purchase machine time on cloud hosting services, and then running algorithms (e.g. Bitcoin mining, e-commerce) which are more profitable on those hosts than the cost of said cloud resources.
The first such entity might be "blamed" on the person who wrote it—but what if its codebase includes code to detect beneficial open-source patches to itself (on Github et al), and spawn its descendants with those patches in place? Eventually there could be a whole ecosystem of these things doing different things, coded effectively "by the world" but running for nobody's benefit in particular but their own. They'd be "AIs that compete with us" in the literal sense. (And if you think about what a real autonomous AI agent would have to do to survive fault-tolerantly on the Internet, it'd probably look very similar; this is sort of a porto-AI-rights thing.)