You might want to try Sonic Pi, which pairs Ruby with the SuperCollider synthesizer engine: https://sonic-pi.net/
Edit: I’m on the Sonic Pi core team. I mean that I’m looking to add these features to sonic pi soon
The question isn't can you do it - because you can, with varying degrees of difficulty.
The question is what specific user problem you're trying to solve.