[0] https://mitpress.mit.edu/books/audio-programming-book
Edit: My goal is to make a mini-synth which takes input from the computer keyboard.
——
The founder, and sole developer of Mutable Instruments (a well known modular audio/synth maker) wrote about her recommendations on how to get started in audio programming here [0]. Much recommended for the cited books, and general/business advice on making a hardware audio product.
[0] https://modwiggler.com/forum/viewtopic.php?p=2858814#p285881...
——
My own post about this was from a mini thread on HN about programming sound that may also interest you:
https://news.ycombinator.com/item?id=20491766
——
Last note! Another synth maker personally recommended the same DSP book as noted in the comment above (on Modwiggler), so it’s probably worth checking out!
Udo Zolzer's "DAFX"
Fixed link (anchor is broken!): https://modwiggler.com/forum/viewtopic.php?p=2858814#p285881...
As an EE that only became a programmer later, I would like to add that those are amazing tips.
The Computer Music Tutorial by Curtis Roads is a good companion here if you want inner workings. You might be able to snag a PDF copy online to see if it's for you.
Audio Programming Book is entirely in C and can be a bit of a pain to get things working in C if you're new to programming (although highly worth the initial struggle if you're looking at it with a career in mind), although I'd still recommend stepping from Max/MSP into MATLAB/Python, which focuses on functionality rather than programming itself.
Python is also bit more beginner friendly, although C++ will probably end up being your long term bet.
I'd pick up the Pyo library for python, and/or a tutorial like this one, after dabbling with Max/MSP a bit: https://python.plainenglish.io/build-your-own-python-synthes...
Then if you want to get even deeper into the DSP, you could go with pretty much any of the other resources the others have mentioned here. My personal faves are the Steven Smith DSP book, Julius Smith's resources (with its runnable MATLAB scripts), and DAFX book by Udo Zolzer.
Idk maybe it’s just me but I’m an experienced programmer and I’ve spent hours learning max/msp and really don’t like the visual aspect. I like programming in vim normally and zipping around the file manipulating text very efficiently without ever taking my hands off the keyboard. Max/msp is the exact opposite, everything is drag and drop mouse movements, and then you can’t just insert new lines in a file to extend something you have to constantly be dragging your existing components out of the way to make room for new components. I find it way more tedious than any other programming environment I’ve used.
Max actually does have a Javascript API, it's a bit old fashioned and you couldn't use it for any DSP related stuff, but can be useful for some stuff. I used it to build a prototype of something that I'd have no idea how to build visually in Max (basically a macro parameter fader for Max for Live), but is fairly straightforward in code, and took some time to figure out a nice(ish) workflow for working with it with Typescript: https://github.com/tomduncalf/livefader
I wonder if you could use that plus the "gen~" object and/or dynamically generating Max objects from JS to do more advanced synthesis-type stuff. May be more trouble than it's worth though.
- "Introduction to Computer Music"[0] by Nick Collins: very focused on audio programming for musical applications but you can easily extrapolate for other domains. Lots of pseudo-code algorithms are provided.
- "The Scientist and Engineer's Guide to Digital Signal Processing"[1] by Steven W. Smith: the bread and butter for everything concept related to DSP. Everything is explained in a crystal-clear fashion, lots of sample code to get you started with digital filters and analysis (although written in BASIC but easily translatable to C/C++ or even Python).
Also there is this one article "Real-time audio programming 101: time waits for nothing"[2] by Ross Bencina (creator of AudioMulch, Portaudio, etc.) about the basics concepts of real-time audio programming
[0] https://www.wiley.com/en-gb/Introduction+to+Computer+Music-p...
[2] http://www.rossbencina.com/code/real-time-audio-programming-...
I wanted to help myself and other folks develop better intuitions around the material, particularly focusing on short animations to develop better visual intuition along with working code examples of the material in the books.
Books covered (with a notebook for each chapter):
* Musimathics volumes 1 & 2 by Gareth Loy
* Introduction to the DFT by Julius Smith
* Introduction to Digital Filters by Julius Smith
* Physical Audio Signal Processing by Julius Smith
also a couple not directly about audio but helpful for the domain:
* Coding the Matrix by Philip Klein
* Accelerated C++ by Andrew Koenig and Barbara Moo
Hope someone gets some value from these - have fun!
I am comfortable writing low level efficient real-time code in C (I'm an ex game dev). I understand most synthesis techniques in pretty good detail. I could probably write a book on subtractive synthesis, and I have implemented a complete functioning FM synth.
I get the fundamental concepts around DSP: Nyquist, sampling, Fourier analysis, etc.
But when it comes to sitting down and, say, designing and implementing a filter from scratch, I'm pretty well stuck. Part of it is that most of the material I've found either goes straight to heavy calculus (which I can kind of hack but am pretty rusty on) or to MATLAB (which I don't have).
Likewise, implementing a simple digital sawtooth or square oscillator analytically is trivial. Sample playback is also a piece of cake. But doing those without aliasing seems to jump from super-easy mode all the way to hardcore nightmare. All of a sudden I'm trying to understand what minBLEPs are and other stuff that feels over my head.
Any recommendations on how to get over this hump? Should I get MATLAB?
For exploratory/analysis/prototyping work, I have not found a better platform and ecosystem. These notebooks are really powerful feature-wise, you can mix Markdown notes/documentation with code blocks with data visualization natively, which is a no brainer when doing your research work.
Web Audio API and JUCE are the modern building blocks for mostly everything out there; if you're familiar with these, that's a really good place to start.
If you're looking for something accessible, SOUL is an excellent domain-specific scripting language that leverages JUCE underneath; see https://github.com/julianstorer for more context.
I'm not as sold on Max/MSP or Pd, though they're fun to play with for a brief moment.
https://filmora.wondershare.com/audio/best-free-daw-online.h...
https://m.youtube.com/watch?v=tgamhuQnOkM&t=77s
https://m.youtube.com/watch?v=OSCzKOqtgcA&t=2s
I asked myself too how I should get started with Audio Programming in 2019. I ended up building my own little synth in the progress and I ended up talking to people from Ableton and even gave a talk at Native Instrument's headquarters here in Berlin.
I blogged about my experience and my product here: https://timdaub.github.io/2020/02/19/wasm-synth/
* JUCE for making VSTs. https://juce.com/
* ELK Audio for hardware / audio. https://elk.audio/
* Find open source projects that meet your needs / desires.
It ccomes with many Xcode playgrounds for interactive exploration of the SDK.
This book teaches you DSP and also how to apply those DSP algorithms by employing them inside an audio plugin.
I started with Csound and Cabbage framework[1] which I used to create some VST plugins.
Then I moved to javascript lands (as everyone already has runtime called web browser) and I am creating sort of DAW using webaudio. I used Tone.js, but it was somewhat confusing, so I started my own library[2].
It's not finished yet but I have already nice and usable demo - comb organ playable with computer or MIDI keyboard with built-in "tape recorder"[3]. Good news is that is open source so you can actually stole some ideas from me.
[1]: https://cabbageaudio.com/
[2]: https://github.com/severak/cyber-music-studio/blob/main/js/h...
[3]: https://severak.github.io/cyber-music-studio/regenschori.htm...
> My music used to make children cry. But since using Cabbage the children laugh and dance with joy.
> Julie, Montreal
> No vegetables were harmed in the making of this software
Covered on HN late last year https://news.ycombinator.com/item?id=24940624 ("How to create minimal music with code in any programming language")
Also related to bytebeats: https://soundcloud.com/robertskmiles/bitshift-variations-in-...
I don't know if you're a musician yourself but I'd highly recommend downloading a DAW and playing around with the plugins to get a sense of common audio processing modules, you can perhaps make it a near-term goal to create an EQ or filter plugin that you can install and use in the DAW. IIRC I think Ableton Lite should have a free demo, Reaper is another well-regarded free DAW from what I've heard
Also, you can learn a ton by looking at various other higher level languages do it: Csound, SuperCollider, PureData are all open source.
Oli Larkin made a brilliant page of resources on github: https://github.com/olilarkin/awesome-musicdsp
In my head, I'm imagining a more programmatic approach to "arrangement of samples and loops," as in, instead of, or perhaps complementing, the classic "grid" or "buttons" view -- something like,e.g.:
- Specify 4/4 @ 95bpm for the piece
- Sample 1, a drum loop stretched/and or pitched
- Sample 2, a hit on the 2nd beat of each interval
- Sample 3, a simple electric piano melody, repeating over X intervals.
- Sample 4, a pad with an echo/reverb
etc. Now, what should be easy to do is to switch out, e.g. which drum loop, or change your piano to a trumpet. I get that MIDI is okay for this -- but is of course better for the latter situation and not great for effects and audio samples.
Your Abletons and so forth are not bad at this, of course, but - and maybe this what I'm describing -- Ableton's features -- but doable in vim, perhaps with the ability to create functions et al on the fly, which strikes me as a potentially powerful way to "group" musical sounds/ideas/intervals/melodies/chords/effects etc etc
Sample wrangling becomes pretty thorny quickly as if you are triggering samples you want to make sure they don't "clip"/"pop" when you change them (due to discontinuities between the abrupt stop and the new start). I wrote a SuperCollider tutorial that demo's how to make a really high-quality sample player that takes care of these kinds of things plus gives you lots of control over rates, looping, etc. [1] That was my main hurdle with SuperCollider as that it has surprisingly few tutorials for such a long-running and deep program. Hopefully that helps :)
The Cipriani & Giri books for Max cover working with samples and loops in great detail and are fantastic books.
I thought I might throw my hat in the ring of shameless self promotion since you might be interested:
I'm working on a project called Elementary Audio, which is a brand new javascript runtime and framework for writing native audio applications– meaning desktop apps, like perhaps the mini-synth you're hoping to make, as well as audio plugins (VST, AU, AAX, etc) and even embedded hardware, all in JavaScript.
The project is still in an early stage (and macOS only right now). I'm planning to share it more widely here on HN once it's ready for beta, but I thought this thread might be interested in it before then.
Website: https://www.elementary.audio/
And here's an example MIDI Synth: https://github.com/nick-thompson/elementary/blob/master/exam...
You could even spin up an Electron frontend to take input from your computer keyboard and send those over to Elementary to render the synth sounds. It's actually an example project I was considering doing myself but you might beat me to it at this point!
Lastly, I'll say briefly that there are a lot of tools out there (as we're seeing in these comments), different things work for different people, and there's no single "right" way to start audio programming– find the path that makes it fun for you, because it's such an awesome field to get into.
But if you're using an existing computer music programming environment like Csound (used in The Audio Programming Book), SuperCollider, or PureData, you can make a mini-synth which takes input from the computer keyboard without any of that stuff! (Stay away from Max/MSP; it's proprietary.)
The most fun I've had with audio programming has been bytebeat, though: http://canonical.org/~kragen/bytebeat. I didn't invent bytebeat but I did write that page about it.
Also, there are a lot of the hassles involved in audio latency on CTSS-derived OSes like MacOS, Linux (especially Android), or Microsoft Windows; it's easy to end up with 100+ ms of latency, and hard to do better than 10 ms, and either of those is a super shitty musician experience. If you do your stuff on an Arduino or similar microcontroller instead, those problems evaporate, and your computer music system will fit into an effects pedal. You need a DAC and, if you're taking audio input, an ADC; the built-in PWM and ADC in a standard AVR may not be sufficient. (And, beware, a 31.25 kHz PWM carrier can go right through a stereo system and burn out your venue's tweeters if it's turned up high enough. Filtering is required.)
Now that you've read a book, pick an extremely simple task and just do it. For example, maybe try writing a command-line program that will just play a square wave through the speakers. Then, build up your skills by making options for sawtooth, triangle, and sine waves. Finally, in the callback from the audio API, poll the keyboard to decide what tones to play. ;)
FWIW: A few years ago I wrote a functional keyboard and mouse synth program that just did sine, triangle, square waves and harmonics. It sounded a lot like an organ. (I later lost interest.) What helped was that I found a wrapper for MacOS CoreAudio that dramatically simplified the API so all I needed to do was provide a stream of floats. (I don't remember the name.)
Don't fall under the illusion that you can just read books and at the end you'll know how to do this. You'll need to work through simplified tasks to build your skills and understanding before you can write your mini-synth.
A book I thought was great has no code in it, but is FULL of very useful info about audio and computing it. Computer Music by Dodge and Jerse, ISBN 0028646827 (1997) ... a little pricey, might be hard to find, but very hard to beat.
[1] I haven't seen Richard's book (I'm sure it's great - and I will see it, thanks for the tip) ... he just might be pitching this idea in there!
You can absolutely make a nice synth that takes input from pretty much anything you can attach to a computer.
And as everyone else has said - theAudioProgrammer is good too.
(Edit) - Pure Data (PD) is an open source alternative to Max which is free. I've heard people recommend it but never used it myself.
Sound on sound has a series on synthesizer concepts. https://www.soundonsound.com/series/synth-secrets
Dsp guide is a great resource. http://dspguide.com/
Music dsp also has some good information and implementations. https://www.musicdsp.org/en/latest/index.html
Ross Bencina also has some good articles about the realtime aspect. Basically, you shouldn't use locks and should become intimately familiar with ring buffers :)
If you want to do anything useful you'll need a GUI which can control the parameters of the audio plugin. The GUI will communicate with the audio thread at control rate. The audio thread will usually pick up messages from the GUI via a ring buffer. And by the way, this can happen in reverse, e.g. for level meters, where you do a moving average of squared amplitude (RMS), and output this as a control rate. You can use ring buffers for this, but another approach is to use atomic variables. And in x86 architectures you can actually even use just regular variables because of its strong memory architecture.
- Musimathics Vol I & II: This will teach you the basics of Music and DSP in a very approachable way. I wish I had these when I started writing audio software.
- The Art of VA Filter Design: https://www.native-instruments.com/fileadmin/ni_media/downlo...
If you are programming in C, these libraries are very useful:
- PortAudio: Cross platform audio playback
- libsndfile: Easy and cross platform sound file io
- ladspa: Really simple plugin API, useful if you want to make or use audio plugins without all the boilerplate of vst or lv2
I'm using the Unity game engine, and am leaning heavily on already-implemented FFT algorithms available on the Unity AudioSource component, but the fun part is taking the spectrum data, dividing it up into bands, and doing visuals with those.
Here is a short video showing a simple audio visualization demo I created: https://www.youtube.com/watch?v=kYt0dcUY3i8
It led me down a bit of a rabbit hole with watching videos about Fast Fourier Transforms, and learning more about audio specific programming.
What is your background, and which parts of audio programming would you like to focus on? e.g. are you interested in writing your own low-level DSP code, or more interested in hooking together higher-level blocks?
If you're from a web programming background, you can have a lot of fun with the WebAudio API [1]. It provides high level building blocks like oscillators, filters, effects, etc., which you can hook together. It has some rough edges but it's a great starting point.
Tone.js [2] is a great way to get started with WebAudio, it provides some useful abstractions that let you build fun things quickly, and there are some great examples of WebAudio creativity out there e.g. Blokdust [3]. If you want to go lower level, all major browsers now support the AudioWorklet API [4], which lets you write your own custom DSP algorithms and have them run efficiently within WebAudio.
However, WebAudio has limitations in terms of performance, and you can't use a WebAudio synth as a plugin in your music production software of choice (well, you might be able to get a browser which runs as a plugin, but it's not going to be ideal!).
If you want to write professional audio code, you can't really avoid working with C++ - it's the industry standard approach. This is largely because C++ allows programmers to very carefully design the performance characteristics of their code - specifically, you can avoid doing anything which might take an unknown amount of time (e.g. allocating new memory, locking) while processing on your audio thread - audio is realtime and very low latency, so you need to be sure your code is going to run in the amount of time you think it will.
Personally, I think JUCE [5] is a great starting point. It abstracts away some of the complexities of both C++ and audio APIs, allowing you to build cross platform audio apps and plugins without having to worry about platform-specific quirks. They have a pretty good selection of tutorials too [6]. An alternative you could look into is iPlug2 [7]. There are a couple of books by Will Pirkle on the topic of programming audio effects and synths with C++ - the author uses his own framework, but the same principles apply to other frameworks. Make sure you get the new editions if you do get these.
You'll probably want to take time to learn some C++ basics first if you don't already know it, it's a hard language to master! I'm actually working on an integration right now to allow you to build web UIs for JUCE apps, which takes away the complexity of also building the UI in C++ - I'll hopefully have something to share in the next few weeks.
If you want something a bit more straightforward and you're happy just supporting MacOS and iOS, AudioKit [8] is worth a look. It's kind of in between WebAudio and JUCE in terms of complexity I'd say - you write your code in Swift and it provides a load of high level building blocks, but you can drop down to a lower level if you need to write your own stuff in C++. Also the CoreAudio/CoreMIDI APIs on Apple platforms are very powerful, though I've struggled to find great documentation.
One other option you could look at are specialist audio languages like Supercollider [9] and Faust [10], or visual programming tools like Max [11] or Pure Data [12]. I'm not very well versed in any of these, but I've heard good things about Supercollider especially.
Phew, hope that helps! As you can probably tell, I'm really interested in this stuff so please give me a shout with any questions!
Edit: one more resource I remembered, there's a Youtube channel dedicated to learning audio programming, mostly using JUCE, called The Audio Programmer [13]. I haven't actually watched much content so can't say too much about that, but I know the guy behind it and he's great, and also runs a great community for people interested in audio programming [14].
[1] https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...
[4] https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkle...
[5] https://juce.com
[6] https://juce.com/learn/tutorials
[7] https://github.com/iPlug2/iPlug2
[9] https://supercollider.github.io/
[13] https://www.youtube.com/channel/UCpKb02FsH4WH4X_2xhIoJ1A
I would add to that VCV Rack [1] is another great starting point.
Very low barrier to entry, see developer manual [2], hundreds open source modules to learn from. [3]
[2] https://vcvrack.com/manual/PluginDevelopmentTutorial
[3] https://github.com/search?o=desc&q=vcv&s=&type=Repositories
I saw this the other day that lets reuse gen~ code from Max as a VCV module: https://github.com/isabelgk/gen-rack
I don't know much about audio programming, but here's a question.
When emulating sounds from physical instruments, can you get close by using only code without using audio samples? If yes, is it harder/easier?
I believe Audio Modelling are one of the leaders in this field: https://audiomodeling.com/
In terms of harder/easier... I'd guess it's a lot harder to code, but easier in terms of not having to sample thousands of different articulations for each instrument! Judging by the huge selection of sampled physical instruments and the small amount of synthesised ones, I guess overall harder. Would be interesting to hear from someone who knows more though!
I assume you've seen the Blueprint project for building React UIs on top of JUCE? Very interested to hear more details about what you've been working on whenever you have something ready to share!
My primary motivations for doing something different was that Blueprint currently doesn't work on mobile (and I'm not sure it could ship on the iOS app store currently, with it using Duktape), and I wanted to get away from writing any C++ at all for the UI – and also that I love playing with this kind of stuff, heh. The app that I work on is built using React Native and JUCE (and Unity), so it's an area I have a lot of thoughts about!
TBH the web stuff started out as an experiment, but I'm actually pretty impressed by how practical web is for building UIs – I've got some nice integration going on to make it pretty natural to write plugin UIs. I'll post it on here when it's ready, but if you have a Twitter/email/whatever you'd like to share or can contact me via mine in my profile, I'd be happy to share with you in advance – I'd love to get feedback from people interested in this area.
My understanding of Rust audio programming is that it is possible, but still fairly early days... something like JUCE provides a huge amount out of the box that you might not initially appreciate, e.g. it abstracts away the details of all the different plugin formats and all the different platform's audio APIs, and it has been around for long enough that you can be confident it does it well.
I'm really curious about Rust as it feels like a nicer language to work with than C++, but if you are serious about building professional audio software I personally don't think you can avoid C++ so you might as well embrace it.
I have a feeling that with Rust, you'd spend half your time figuring out how to do stuff that JUCE does out of the box, which is fine if you are interested in the figuring out part, but ultimately it might not help you focus on the actual audio programming.
Also there are a tonne of resources on C++ and especially JUCE out there, never underestimate the power of being able to Google for your answers!
Edit: on the C++ side, I found Bjarne Stroustrup's "C++ Programming Language" book quite a good way to understand the fundamentals. To be honest, I usually learn new languages just by doing and Googling, but with C++ it quickly became apparent after the 100th mystery crash that I'd actually have to understand what was going on under the hood with memory etc!
One thing to watch out for is that the language has evolved a lot recently, but a lot of code samples are still written in an old style. You want to make sure you are using “modern” C++ (e.g. post C++11) features where possible e.g. smart pointers. It can be a bit confusing as some JUCE classes have been superseded by the C++ standard library, they date back to the days before “modern C++”... but I wouldn’t stress too much, using the JUCE versions is usually equally fine.
No, you can do with C just fine.
Learn C++ Real-Time Audio Programming with Bela - https://hackaday.io/course/171240-learn-c-real-time-audio-pr.... The course does recommend buying the Bela development board from https://shop.bela.io/ but I don't think it's a hard requirement.
[0] - https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...
https://news.ycombinator.com/item?id=27229836
My favorite was
If you want a really gentle start, try Sassy: https://sol-hsa.itch.io/sassy
If you want this to play nicely musically you want low latency and I'd look for programming languages made for audio. Just watch a few introductions and pick what you like best.
More professionally I'm not experienced, but maybe writing a VST MIDI instrument would make sense.
If it's just about the sound maybe you should start at the DAW end and use Bitwigs Grid or Ableton (possibly with max/msp).
If you are a Ruby programmer on Linux, you could use this rubygem I wrote: https://github.com/mike-bourgeous/mb-sound
A video about using that gem to make a synthesizer: https://m.youtube.com/watch?v=aS43s6TWnIY&feature=youtu.be
Part of a long-running experiment of mine to make educational videos about sound, which I hope might help you on your audio programming journey: https://www.youtube.com/playlist?list=PLpRqC8LaADXnwve3e8gI2...
There have been recent posts to HN about the difficulty of reading key-up events from the terminal. I used MIDI and a separate MIDI keyboard app for my video demo.
It might be worth studying the sources of some working soft synths such as Helm, Surge or ZynAddSubFx or mixed firmware-hardware synths such as TSynth. The latter in particular is a low cost polyphonic external module built around a Teensy microcontroller, so the code base isn't that big, and it sounds great.
https://surge-synthesizer.github.io/
https://github.com/zynaddsubfx/zynaddsubfx
[0] https://sonic-pi.net/ [1] https://github.com/sonic-pi-net/sonic-pi
It's not as powerful as something like PureData but it does give you a simple and intuitive introduction to DSP and audio synthesis.
Tons of great code to read and projects to understand ..
Also, the Basic Synth book is a dream:
- https://news.ycombinator.com/item?id=27269589 - Elk OS – Audio Operating System
I generally learn best by having a real thing to build; and then finding the resources I need when I get stuck.
I buy books aspirationally (I have bought the Audio Programming Book as well come to think of it) - so I often end up with many books on subjects that interests me, without learning much from them.
If you don't want to do realtime synthesis, you can also dive in the (uncompressed) WAV file format and make programs that generate sound files.
Another thing that might be nice (but I personally haven't tried) is to synthesize your own drums, 808-style. Google 'kick synthesis' for example.
https://online.stanford.edu/courses/sohs-ymusic0001-audio-si...
See also this book about computational music synthesis:
Also, I second the suggestion to join the Audio Programmer community.