Back when I was writing a software synth for fun (and also for getting hired in the audio industry), I found the book DAFX to be very helpful. It describes basic algorithms for doing digital audio, though not the state of the art.