So you have to spend the resources to develop self-driving technology and to maintain a stable of remote drivers. And all that- for what? Human brains are still in control of cars and car AI is still dumb as bricks often enough to make them less safe than humans. If you had problems with humans forced to jump in and take control at a moment's notice where the car failed, before, now you're making these problems even worse by having someone take control remotely so that they're even less aware of the unfolding situation than the human in the car.
It's just ...pointless. It makes even less sense than a car that drives itself safely only when you're paying attention (hey, I can kill myself when I'm not paying attention just fine on my own, I don't need an AI car to do that).