Hacker News Reader

Comment by nimish | Hacker News Reader

Normally reverse mode automatic differentiation is used and is called backpropagation since everything old has to have a new and shiny name in machine learning a/k/a continuous optimization ;)

View on HN

I concur. When I first started taking online coursework in machine learning, I was in almost every unit asking myself, didn't I cover this already in my non-linear optimization class at Uni.?

The main difference is that a lot of practitioners using ML have very little advanced mathematical background. Let's be honest, you can't understand much of optimization without at least multivariable calculus and linear algebra.

I think this is a very bad thing, but basically what happens when things become "commoditized".

I feel it's the same with web developers who have no formal background in Computer Science. All these frameworks reinventing "paradigms" that have been studied since the 60's.

I'm not sure what your point is. People use tools without understanding the underlying principles all the time, frequently in ways that are more (tangibly) practical and interesting than a person who does would use them.

Presumably his point is that you can be more effective at something if you understand it at a level one level below the one at which you are working.

I'd say that depends entirely on context. It also has diminishing value each "level" one goes. I mean, understanding quantum degeneracy pressure doesn't make physicists more effective at being solid.

I think his point is about reinventing the wheel due to insufficient theoretical background.

ML researchers have the math background.

I have had the same thoughts too. What also had me a bit puzzled in the ML community is the lack of discussion on local optimality, and constraint/Lagrangian loss functions. But there is probably a subtlety that I'm missing.

The engineers (aerospace, etc) also prefer the term "adjoint mode"

Reply on news.ycombinator.com