Very unlikely I will ever talk to my computer irrespective of how good the speed technology gets. If I have to talk to my computer how will I work in crowded places ? How does it work ?
I can open Microsoft Word faster than I can say "open microsoft word". It would have to be smart enough to short circuit the entire process of doing something useful with Word.
It might pass in a coffee shop, it probably won't in an airplane, it will almost certainly never pass in a library. People will try it regardless of the appropriateness of the location, because some people aren't aware or are assholes, and as a result the entire technology will get a bad rap. See google glass as an example.
I agree if it was already the social norm it wouldn't be a problem (same with Google glass), but it turns out that the technology being ready isn't always enough to make the social norms change.