Hacker News Reader

Comment by adamsmith143 | Hacker News Reader

This is pure gatekeeping. The math behind LLMs, that is, the math behind Neural Nets, is undergrad freshman level Calculus and some linear algebra. Not really complex at all. Can you deal with derivatives, the chain rule and matrix multiplications? Great you know all the "math" behind Deep Learning.

View on HN

That's the beginning math, but definitely not "the math behind LLMs". That includes probability theory, metric theory, topology, and more. But most people don't even acknowledge this, but then again, unless you're deep in a subject you don't really know the complexities of that subject. Red flags should go off whenever anyone says "it's just <x>" or calls something simple. It's like the professor saying the proof is trivial, when that's the hardest part of the entire problem.

People like you are hilarious. You're sitting high in your ivory tower thinking that no one without a PhD in CS from Stanford/Berkeley/MIT can do what you do. Meanwhile people will take the Fast.ai course and be training full llm's from scratch in 6 months all the while you moan that "they don't even understand the REAL math". Yawn.

People like you are funny because you don't realize I'm actually also calling out a lot of ivory tower people.

Also, I'm not saying you need the math to train a model. I'm not sure you even need linear algebra to do that, mostly just programming. That's why I called it the beginning. I was directly responding to your claim that this is all you need __to understand__. Because let's be real, you don't need to know backprop (and thus derivatives and chain rule) to train models. The math is about how to analyze your models. You know, specifically what academia is supposed to be doing. Research and engineering overlap but they aren't necessarily the same thing.

Besides, math education is notorious for being essentially free. Who needs Stanford/Berkeley/MIT when textbooks exist widely. (Btw, CS doesn't typically produce mathematicians)

If you want, you can get all of that math at any Top500 university, even in undergrad. I agree that people will be having impact and deploying these models without that understanding, but you don't need to be at Stanford to gain that understanding and it's something desirable if you want to do research instead of deployment.

> Can you deal with derivatives

Linear Algebra is a huge ... space?

And then there's all the other algebras. I'm not sure what you are thinking, like, do you not know matrix multiplication you had better get a clue.

What does that have to do with ML/DL? You don't need Lie Algebra to understand how to do a matrix dot product...

Reply on news.ycombinator.com