I'm pretty sure this idea has been kicked around, and is generally seen as not-super-useful because it requires a perfectly reliable low-latency connection.
Phantom Auto did this. They had cars ferrying people around CES 2018 in Vegas driven by remote drivers in LA. Apparently the company folded just a few weeks ago.
To solve this issue you can integrate the person into the car.
Conjoiner Drives come to mind...
Now I'm imagining hiding drivers under the hood, or in a car following ...