I've loved the idea of a verbal commandline, too, especially since watching the Play For Tomorrow episode "Shades[*]." It's a teen brave new worldish sort of thing, but all of the characters are constantly interacting with databases, networks and media (including VR) with this very stylized language, not how you would talk to a person.
That's the problem, really. You have to use a stylized language on a commandline. A GUI is really just giving you a graphical diagram and guide to a really stylized language (if you reduce a GUI to its clickable/typeable areas.) It's very hard to draw an arrow pointing at a picture representing a particular predicate in the very restricted bandwidth of serialized sound.
The reason why "Shades" was so interesting to me was that its future teens clearly all knew the stylized language well, and obviously would have learned it in school. To have that kind of universal training in a certain form of computer interaction would require a universal OS, like what UNIX quickly became.
So maybe this is a conclusion coming off the above at an odd angle, but what I'm actually saying is that commandlines are hard and people don't like them. Visual, and even tactile stuff have the ability to offer so much feedback and so many affordances for amateur operators in comparison to audio that audio can't compete. Audio feedback is all exposition and nothing to hold onto or see. And just like film exposition, it often comes too slowly to keep people's attention, or too quickly for people to understand. It's always going to be easier to grab the knob and turn it, or to find the button and click it.
The major advantage of audio interfaces is that they're cheap both materially (you don't need a screen or a control panel) and for the user (you don't have to stop walking, or sit down, or grab a thing, etc. you just speak.) For me these advantages are actually dominant over innate usability, which is why I also believe with you in the huge potential of audio interfaces as a platform.
I just think that the only way to get there is to build a universal and open verbal protocol, with standardized interfaces for accessing databases, media, and networks. Something that children can (will) learn, and will have a lifetime of use. An opinionated and efficient design moonshot like the web (or the Bourne Shell or Gnu Corutils) turned out to be. A platform whose innovation comes from addition of new ways and not from subtraction of the old ones. A platform that holds onto backwards compatibility as long as is practical, thinking in terms of how many decades to keep things around, not how many releases, or possibly even a closed kernel that will never change built on top of some mathy orthogonal basis system.
* I wish this hadn't been deleted from youtube.