I agree that we probably won't see human extinction before robotics gets much better, and that robot factories will require lots of infrastructure. But I claim that robotics + automated infrastructure will eventually get good enough that they don't need humans in the loop. In the meantime, humans can still become mostly disempowered in the same way that e.g. North Koreans citizens are.
Again I agree that this all might be a ways away, but I'm trying to reason about what the stable equilibria of the future are, not about what current capabilities are.