Hacker News Reader

Comment by tw1010 | Hacker News Reader

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theoretical dirt.

View on HN

Often the trick is not to make the new thing, but to make the old thing comprehensible so someone else can pick it up and run with it.

I agree. The real low hanging fruit is not new research, but to make the easy, basic first or second year graduate math understandable in the context of ML.

Or applying the old thing in a new way to a different audience, like Einstein using Lorentzian manifolds.

I guess that's why pure mathematicians have been at the forefront of all the major developments in neural networks and deep learning? This is the classic applied vs. theoretical debate. Let's not pretend like there's ever a clear winner.

I suspect you're being sarcastic. As a species, engineers are currently way more successful than pure mathematicians because of their higher "frequency dependent fitness" (to use terminology from biology), meaning that because of the huge number of engineers that do the small incremental improvements, looking at all of them as a single organism makes them look more well adapted to the innovation landscape than the (pure) mathematicians. But if you normalize for population size, if you were to pit an equal number of pure mathematicians against engineers, then I suspect the former would prove to be more innovative, at least on a medium to long term horizon, than the latter.

Listen, I respect and love pure mathematics as much as the next guy. However, where's the utility in establishing, with any more clarity, exactly which field should be understood as being responsible for the biggest innovations? This feels like a needless detour into a tired debate which has already been hashed over countless times.

I agree. I was mostly addressing your argument that the output of each field (engineering vs math) is a good marker of ability to innovate (which needn't be true because of their relative population size).

On a similar note, taking a look at a mathematician's coursebooks made me realize how puny our mathematical tools are. Most Computer Science topics only rely on (relatively) basic concepts from logic, algebra, calculus, and/or probability. The hardest course I ever attended was a quantum computing one, and even that does not require much, apart from a generalization of probability theory to complex numbers.

I bet there are lots of brilliant results that a smart mind could apply to our field.

I think the human brain is capable of only grasping so much abstraction. I'm sure every genius working in the deepest areas of advanced mathematics has heard of deep learning. If there was some amazing concept from their field directly applicable to deep learning we would have heard about it by now.

Differentiable learning is a new hot topic and this is research from an (applied) math phd from UCLA. This is valid research that may or may not advance the field - like any other research. What's the issue?

Nobody is discounting that someone could open up a completely different learning paradigm, and it would most likely come from a pure mathematician with computer skills. A few prominent researchers have said something to that effect already.

So according to you, why hasn't it happened yet? Not trying to flame, I'm genuinely curious.

I would think that the economic incentive to do exactly this would be pretty high.

What are some good results coming out from applied mathematics?

Reply on news.ycombinator.com