If it's like the system Waymo's talked about, there will be remote operators who can help the system disambiguate unusual circumstances but they won't have direct control of the vehicle. It'll be more like the car asking "I'm stopped behind this van, should I wait for it or try to go around?"

Once the physical driving is done by the system there's nothing wrong with letting a human help out with the corner cases. And obviously as each case occurs it generates more targeted data so that the system can learn to cover those cases itself in future.

As with most automation, 'perfect' is the enemy of 'good', and the most pragmatic and cost effective solution is to let the system handle the low hanging fruit while humans do the tricky stuff.

If that's really how it's working I don't see it being good enough. Latency would be too high, and the infrastructures not there for a reliable enough internet connection from (somewhere in the USA) to (somewhere in asia).

I mean that would actually be pretty cool! Taxi driver as a desk job with multiple offices geographically so drivers never have to work outside of 9-5.

Has anyone done this successfully so far? I would have thought that the round trip time for the video feed + returning inputs would be prohibitive. I've done a similar thing with vehicles in videogames, and anything over around 150ms was pretty much undriveable.

Read the blog entry on disengagement. This is exactly it: there are remote operator that can disengage the AI and take over in some situation for caution.