Hacker News Reader

Comment by jiggawatts | Hacker News Reader

There is practically no scenario where a digital computer couldn't do better if what you wanted was a numerical result and not a symbolic one.

It's simply criminal how underutilised the technique of Automatic Differentiation is! It can solve incredibly complex differentials to machine precision, good numerical stability, and minimal tuning or hand-holding.

Lots of these "difficult" problems aren't, it's just that people aren't aware of the solution because it's not commonly taught in a University setting. Automatic Differentiation is one of those techniques that's "boring" and "numerical", so there are no courses that teach it.

View on HN

The parent comment is talking about solving differential equations. That's a very different thing from computing differentials. Automatic differentiation doesn't help with differential equations at all.

Yeah, this was a while back. The guy used to work for the South African military during apartheid and later worked on systems in Canada and Asia. I'm sure you're right that digital computers could do the job today, but back then the software and hardware wasn't close to being up to the task.

Doesn't literally every optimisation and ML class teaches it nowadays?

Actually, no! You would think so, but again, it's a boring numerical technique so it is all too often sidelined in favour of the more "flashy" symbolic techniques.

An awful lot of tertiary education has been caught up in the publish-or-perish mentality. Unfortunately for AD, it is not "impressive". Differentiating hideously difficult expressions symbolically is impressive. Publishing a paper packed full of enormous, scary looking expressions is impressive. Merely saying that "we used AD and it worked just fine" is unimpressive.

For decades now we've been incentivizing intellectual wankery over practical utility.

Here's an example from my field: "The derivatives of the finite-build coils are computed -- with almost no additional effort from the authors -- using automatic differentiation." Well, what kind of scientist are you, if you are too lazy to take derivatives? /sarcasm

http://meetings.aps.org/Meeting/DPP20/Session/BP14.32

Reply on news.ycombinator.com