Not everybody wants to talk to their devices? I think they are fine to have, but there always need to be a fallback to a manual hands on input.
Buttons are the way. Or those cool switches from fighter jets... I'd like to see those in a car.
I can't speak to all cars, but this isn't the case for Tesla. It's a very good user experience. It may be inconsistent across manufacturers right now, but as that evens out I don't see a barrier to more adoption.