There's a realtime programming language called 'Supercollider' designed for use with audio, and they have a facebook group. I'd ask this question there. Amongst all the devs involved, someone ought to know of something like what you're after.
That is really a fascinating project. Never heard of Supercollider before. If a "blender-for-audio" does not already exist, it looks like the sort of language that could be used to build an implementation.