It's worth noting that while you can ease your way into programming with the functions available within the Arduino IDE, there are libraries for many of the special functions of the chips, and all of the special function registers are exposed, so you can make full use of your hardware capabilities if desired.
As for choosing a microcontroller, I'm generally tempted to start by choosing a development tool, because I'll be spending more of my time in code development and testing than in any other activity.
I'm interested in synths (and teensies) and would be interested to see it.
I statically compile the loader. On BSD at least, no 3rd party libs are needed. Then all I need is avr-gcc. I can do everything from the command line. No closed source tools. No requisite graphics layer (e.g., Windows OS). No large interpreters (Python, etc.).
https://raw.githubusercontent.com/PaulStoffregen/teensy_load...
Can anyone recommend other boards where everything can be done like this, i.e., without needing closed sorce tools, graphics layer, or installation of interpreters and scripting libraries?
The MSP430-based TI Launchpads.
The various ESP-something WLAN boards have open toolchains and programming over serial as well as far as I know.
I got it to the point of booting (very slowly) up to the command line prompt and processing basic commands, but unfortunately I never figured out how to make the screen work, so that's where the project stalled. The hardware's since been broken up and repurposed.
All in C and hand-written assembly tied together with a makefile, using tools that are in Debian. The standard flash tool (also in Debian) is really easy to use and even supports JTAG debugging over USB. (Also the MSP430 has a lovely assembly instruction set if you want to work with that sort of thing.)
Can't comment about AVR or ESP; never used them.
And of course the RISC-V HiFive1, though it's still very new and in sampling quantities.