How would this work with second, third, etc, derivatives?
I'm not sure where the term "Dual Numbers" came from, but in the numerical math community, this is known as Complex Step Differentiation. https://en.wikipedia.org/wiki/Numerical_differentiation#Comp...
Complex step is only straightforward for first-derivatives. For second-order and above, there are a number of competing techniques. One of them is multicomplex numbers.
From a performance perspective, Automatic Differentiation pretty much outperforms complex step methods in most situations. The only reason to use complex step methods is when you have a very fast language with a complex types (i.e. FORTRAN) and you need a quick-and-easy way of getting 1st order derivatives.
Otherwise, just use AD.