jessejengel

Coincidentally, that's how I came around to it, from a physics undergrad, to music software, to cs.

Hi, yup that's true we keep the filter order fixed. For the experiments in the paper, the time-varying coefficients are generated by a neural network that is trained end-2-end to generate audio like the training set (conditioned on high-level controls such pitch and loudness).

I agree that IIRs are a great avenue of future study, also with time-varying coefficients. I've played around a bit with them, but they are harder to efficiently train with current autodiff software and GPUs/TPUs. I think they may require writing a custom cuda kernel, but I'm hopeful for things like JAX's scan operation.

DDSP modules are helpful in situations where you want to impose some level of interpretability and modularity. Most also don't have parameters themselves, but must have them provided by another network or variable. So you could imagine for instance feeding your data through a NN that then predicts filter coefficients, then running the same data through a filter with those coefficients (if you wanted to enforce time-varying linearity for interpretability let's say).

Sorry about that :), definitely an interesting coincidence. Funny enough we actually were thinking of naming dDSP (as in derivative of DSP) but had already submitted the ICLR paper so just stuck with all caps (also because python is all lowercase).

Hi I'm Jesse, one of the authors, thanks for the interesting questions!

- In terms fo the FIRs, I think you can think of this as a form of more general/nonlinear filter modeling. The difference being I think that you can have a filter as one of several components, and adapt them all jointly to achieve some task (which itself can be more flexibly defined (different losses, adversarial etc.). The filter itself is still just LTV-FIR, but it's being controlled nonlinearly. We only have examined synthesis so far, but other signal processing problems like denoising are definitely good directions. The "effects" processors are designed for this.

- It's true neural networks often learned correlated parameters but it usually is of less significance because they operate in an overparameterized "interpolative" regime, which has a lot of interesting ongoing research trying to understand it.

- We didn't do a quantitative comparison, but in general the tradeoffs will be different. Dereverberation by a modular generative model will only sound as good as the generative model itself, so artifacts will be from not modeling the source properly. However, if you learn a good model, the dereverberation should be essentially perfect (you can losslessly apply different reverb), although that's a big if.