That's the hill that almost every new UI toolkit effort dies on.

If I were going to start one, I would actually start with accessibility. If everyone keeps tripping over the same thing, start by tackling that thing first.

Edit: it's occurred to me that an approach of starting with accessibility could also lead to an approach that degrades gracefully into text UIs (TUI) and command line usage and then scales all the way up to a GUI. That would be super interesting. Not sure it's ever been done.