Personally, I'd feel unsafe. (I'm just trying to imagine getting in the back seat of a car, moving at high speed, with nobody in the front). That's hard to get past.
Probably going to be a generational thing. After some tipping point, people won't give it a second thought.