Thinking about it now, if one would deliberately add much more latency (a few seconds), it might be possible to use real-world simulation as aid. At least for operations which can be decomposed into sequences of transitions between stable/safe states. Say moving dishes from dishwasher to cupboard. Picking up is critical, but holding in hand is (presumably) safe, placing in cupboard critical, once placed it is safe. Then one could let teleoperator do the entire critical move virtually, act it out in simulator only. See what the outcome is. If high risk of failure, deny operation. If good chance of success (per simulation) can allow to execute in the real world. More autonomous operation will need ability to simulate actions, project alternative approaches into the future, and a world model strong enough that can also plan and execute based on it. So there are potential synergies in a full-teleop, to hybrid teleop to autonomy transition. Note, this approach would also assume the relevant environment to be static. So it would not help handle the pet or toddler...