> a remote one to handle unusual circumstances

How could that ever work? The "unusual circumstances" that matter will usually require either action in less than a second, like an improvised detour sign, or situational awareness, like a human directing traffic around some random obstacle. At best, Google will be making cars that pull over to the shoulder and call for help in a few years.