It seems to me like there's sufficient value even if the assistant can't complete any actions fully (e.g., can compose but not send an email; or maybe not even compose). There's so much potential simply aiding with executive function: what to do, when to do it, acquiring any dependencies, handling partial work, helping break down tasks, and a great deal of potential with perception if the assistant is highly available and perceives what the human perceives.
(Rayban Stories could be pretty awesome for this, if they were hackable enough to actually prototype things on: https://hachyderm.io/@ianbicking/110833737363686936)
Imagine a personal assistant that was always ready to respond to the question "what should I do now?" – and of course enter dialog, not just dictate an action. That you could tell about all your tasks, but not just the tasks but also the _why_ of the tasks, giving it the chance to set or change something like a deadline on its own, or even simply discuss those deadlines.
Imagine you could co-develop a process with that assistant. Maybe there's times you like to do certain kinds of work... what are those? What features distinguish different kinds of work? If you have to do a certain kind of work, what do you need (time/place/mindset) to be successful? It doesn't need to be some magic algorithm, it can be a deliberative process that you engage in with your assistant, something conscious and explicit.
Maybe it helps both move through and construct to-do lists. You have an item on your list: either the item is very easy or the question is "what's the first thing you have to do to achieve that item?" – and the assistant has some idea (and can learn more) about what a good size of a task is for you personally. And now it's keeping this list of tasks and dependencies. It should be able to understand enough to mark subtasks complete if you complete the parent task. It can probably suggest items. If it has access to enough information – even if you have to put the information in explicitly – it can probably help you resume tasks by reestablishing all the context you need.
Like maybe all your assistant needs or should have is access to your clipboard (in and out), photos and screenshots, mic and speaker access (with a wake word), a library of notes and observations, and task initiation that isn't any more sophisticated than what you can do from a link (mailto:person?subject=...)