Those are all techbical limitations that will be solved with time. In 5 years I would be surprised if we do NOT have an AI as smart as today's frontier models, that runs completely locally, and has a real time voice interface.

I'm not a Treky but I think Star Trek has the right interface here. For basic operations you just use voice and tell it your intent or what you need to know "Computer, what's the fastest route for me to get home?". "Madison is fastest right now, it's 3 minutes faster than Lake, it traffic is building up".

This isn't just whatever Waze gives you, it knows about your constraints and preferences. So you need to program that somehow. Imagine taking it one step further "Computer, what's the fastest route to pickup my daughter and get home?" or even it just responding to the first prompt "Take Roosevelt, and today you need to pickup your daughter from volleyball".

In Star Trek they often do this when they create "simulations". They tell the computer constraints and then it refines it. No writing programs, but effectively that's what you are doing but with the natural language voice interface.

"Computer let's create a simulation to figure out the fastest way for me to get home. I want to take a route along the expressway and know whether to get off at Roosevelt, Madison, or Lake. I need to pickup my daughter when she has volleyball." You don't need to repeat the whole instruction each time, and behind the scenes it is probably turning it into some deterministic code, so next time you trigger that program it's not going to hallucinate some nonsense.