I've worked on a videogame where vehicles are controlled from the client but have their physics done on the server. Server sends car position, and client sends inputs. So I can give some insight into how driveable it might actually be:
- <40ms (round trip) is not really noticeable
- 40-~80ms is driveable
- 80-150ms is getting difficult
- 150-250ms you're overcorrecting all the time
- 250ms+ is completely undriveable
That's not trying to avoid sudden obstacles, that's just making turns etc that you already intended to make. Avoiding anything sudden would be impossible much earlier. I agree remote control over Internet will never happen.