That's insightful.
Yes, I had moving-the-chair-in-physical-space class of capabilities in mind: robots guided by multimodal intelligence, cars 'surprising me' on a day I'm idle, etc. The challenge here may be in what can be achieved at the edge, the feedback control system for correction of successive prompts.