Computational Music Synthesis
cs.gmu.edu
cs.gmu.edu
I spent five years working with audio/video codecs and APIs on the Xbox 360. Had to support various DSPs and help games improve their performance (curse you, denormals!). But I never got a chance to go full synthesis.
Thanks for publishing this!!
Just curious: on your list of publications[1], music shows up only once. How does this course relate to your other work, if at all? Or is this more of a side interest for you?
But given that a fair amount of post-war contemporary classical was written using aleatoric techniques - basically constrained randomisation - and no one had a problem thinking of it as music, and modern generative music can be far more sophisticated both culturally and structurally, I think your prof was just plain biased and wrong.
See also: digital photography isn't real photography, vinyl is better than FLAC, computers will never play chess/go, etc.
I mean you still could argue it's not music, but I bet the audience on the dance floor disagrees.
Pretty much any generative art is still shaped by the artists input and choices.
I studies some signal processing stuff in college but didn't really connect it to music unfortunately so struggled. Since then I've gotten very interested in electronic music production so it's a 2020 goal to learn more about the underlying software and math techniques, so I really appreciate this resource.
In no universe could my little offering compete with Julius Smith. He is the man.
It would be better for the author's information collection, and friendlier to readers, to indicate that email is not a required field – otherwise people will assume giving email is necessary, as in a zillion similar forms online. And/or explain why email is wanted.
If you put in fake information, my little collector is none the wiser. Or just click on the "without honor" link, which is really there so Google can grab it.
http://www.plastaq.com/scythe_vst/
I'd strongly advise against trying to write music software for a living though. The market is just too competitive now and DAWs are increasingly bundling everything a user might need to make music.
The most accessible DAW that can handle spatial audio is Reaper, which doesn't even allow an odd number of channels.
But even without leaving stereo, DAWs are already really bad. Think about how a pan or balance control works, instead of representing the perceptual localization of a sound they ask you to control the gain feeding two channels, left/right. The semantics of those two channels change after rendering, which makes it difficult to convey the same intent of a mixer/producer to the audience using that interface. Compare a stereo mix on near field monitors positioned perfectly, vs reference headphones, vs a car, vs a club system... the pan/balance control you set in your DAW changes the localization of the sound depending on ultimate render target, whereas what you need for spatial audio is for the localization to be independent from those targets.
The solution is object based audio, which is not perfect and a misnomer 90% of the time, but even that is because of how terribly DAWs handle spatialization. And it's not something you can really control with a plugin without hacking the mixbus configurations inside a DAW, and you can't propagate position down to other plugins.
Waving your arms around explicitly conflicts with the very fine motor control required to play an instrument accurately and expressively.
You need between two and 12 or so (including feet) channels of very precise movement data to make a useful musical instrument. VR hardware isn't designed to have that level of precision.
Of course you can have virtual pads/buttons triggering samples, with a few faders for control of effects. But that's more like sample-based DJing than instrumental performance. It might well have commercial value, but it's a very limited subset of music production as a whole.
However it's more about thinking of musical creation very laterally. It could mean leveraging tilts of any place where one can put an accelerometer (heck, the OP1 does it in a gimmicky way to change any specific setting you want for a given track for example), your finger movements position in 3d space (mimu does some of the finger movement stuff), heck even smoothed-out eye "gestures" could be explored (google-glass-y devices would need to succeed since you're not going anywhere with current tech without invasive things like the EyeLink II). Pivoting around 360 degrees in a "musical cockpit" could also be a thing. Voice too, and NLP could be leveraged in creative ways (to bring forward different aspects of the tool or to give instructions at a high-level - "take the first four measures of the first guitar track and apply melodic inversion, shift down one octave and paste at the 2min mark on the bass track").
Different parts of the music-making process could take on different shapes in spaces that add multiple dimensions. As the mimu shows, a lot of what those dimensions do could be entirely configurable and fitting the creator. Conductors are a good example of someone leveraging rotations and depth (over half-sphere surfaces) to create expressions and rhythm and so on.
Plus nobody has to throw away the current tools either, once could think of the advantages of AR (Augmented Reality) which could leave perfectly clear the areas of your synths (or under a certain line of vision or even physically marked delimitations) during a performance, and all of the glory of ye olde knobs and faders can remain intact.
Current VR is out of reach for too many people for much experimentation (basically forces you to be on Windows, requires either heavy-duty hardware or has very little room for fine details), which means all the exploratory stuff to lead to even a subset of what I mentioned is only accessible to a handful of people so far. Same for multi-modal interfaces (the mimu is still a toy for rich artsy-techy people, the Tobiis are either too basic or too expensive, etc).
But we're still in the infancy of much of these kinds of tools. And while the usual engineer will talk about why everything's impossible, a designer or an artist will always be curious to see what just could be eventually.
However, I do have to make one slight joke:
> "The topics are short and light on theory."
This did not prepare me for full-page greek-letter and symbol walls haha.
A book on music production or an introduction to audio engineering would probably offer a higher level overview.