Seeing the symbolic derivative is the amazing thing to me. Really highlights the blurring of the lines between data and code.

You are correct that the analytic derivative isn't that hard to do in any language.

Edit: Should have said "many languages" not just "any"