Me three.
Ages ago, I implemented an Illustrator style direct manipulation user interface. We had spent so much time analyzing, documenting, specifying in prep.
My initial implementation, per our "perfect" spec, really sucked.
Turns out there's a significant amount of fit and finish to a direct manipulation interface that I wasn't able to handle formally with charts and machines. Ended up with a bunch of special cases. Unanticipated knock on effects were unsatisfying negotiations with QA/Test (how to test) and tech writers (how to describe) our product.
I've always wondered about that gap. Was I doing it wrong? Are there better formalisms? How'd Illustrator (and others) do their direct manipulations?
With the rise of touch UI, I anticipate the problem space (complexity of the states) has gotten smaller. For instance, no modifier keys. But that's just a guess.