Hacker News Reader

Comment by grandempire | Hacker News Reader

If you want to contribute to ML and not just use existing techniques, math skills are the most important limiter.

Who do you know making contributions who isn’t fluent in linear algebra?

Also why are you summarizing the entire field as “LLMs”?

Are math skills really? Most aspects of deep learning don't require a deep understanding of mathematics to understand. Backprop, convolution, attention, recurrent networks, skip connections, GANs, RL, GNNs, etc. can all be stood with only simple calculus and linear algebra.

I understand that the theoretical motivation for models is often more math-heavy, but I'm skeptical that motivations need always be mathematical in nature.

I’m not saying you can’t use these existing techniques without understand all the theory, but you’re not going to be able to find new techniques.

For example, how would you know optimizing a convolution kernel is a good idea if you aren’t familiar with linear time invariant systems?

I think CNNs follow very naturally from the notion of shift/spatial invariance of visual processing. That doesn't require a mathematical understanding.

Image processing and shift invariance come from DSP.

Every MLE who didnt study Math really likes to downplay its importance. Yeah you dont need measure theoretic probability, but you need a grasp of Lin Alg to structure your computations better. Remember the normalization that we do in attention ? That has a math justification. So I guess yeah academics did have a role in building LLMs.

I mean computer scientists really do like to pretend like they invented the whole field. Whereas in reality the average OS, compilers, networks class has nothing to do with core ML. But of course are also important and these barbs dont get us anywhere.

I think you might've taken my point too strongly. Of course math is very useful, and certain contributions are purely mathematical. I just don't think it is as hard of a requirement for innovation as was claimed.

Forget actual CS and proper engineering without discrete math.

Also, without Shannon you wouldn't have neither Telecomms nor Computer Science.

Heck, Lisp it's just a formalisation and implementation of Lambda Calculus, which began as a paper... from a Mathematician.

Also: https://hakmem.org

Forget any serious reading without Math skills.

Reply on news.ycombinator.com