Hacker News Reader

Comment by MrUssek | Hacker News Reader

To be clear this is a forward mode auto diff implementation, not reverse mode, as might be inferred by the reference to the SPJ talk, correct?

View on HN

Correct. Reverse mode seems much harder to express in Haskell..i was trying to understand AD some time ago and used Haskell for that (before running into Conal's paper). This way of writing forward AD was easy and it was awesome to see type inference and laziness help me with the understanding. At that time, I tried to code up reverse mode AD and failed to do it with comparable simplicity.

If I may ...

1. First attempt - https://sriku.org/blog/2019/03/08/automatic-differentiation/

2. Dual numbers and Taylor numbers - http://sriku.org/blog/2019/03/12/automatic-differentiation-d...

3. Higher ranked beings - http://sriku.org/blog/2019/03/13/automatic-differentiation-h...

The struggle I had with reverse mode felt comparable to my struggle with coding up union find (for which mutation is crucial for its efficiency).

Which paper by Conal?

https://arxiv.org/abs/1804.00746

Did this end up helping you with forward or reverse auto diff?

I'm yet to give reverse mode a second try, but my attempts at forward mode (linked in parent post) were on my own without the paper's help. There's a talk that Conal gives that makes the paper a bit more accessible - https://www.youtube.com/watch?v=ne99laPUxN4 - but only a bit more. I know it'll take effort to wrap my head around the implementation details ... but the paper certainly helped clarify things I was stumbling around about.

Reply on news.ycombinator.com