But isnt this where PureData, Max and Supercollider shine? It is possible to create very exotic interfaces with them and many people use it for that purpose. The user communities of the 3 tools is small but very knowledgable.