Hacker News Reader

Comment by bertil | Hacker News Reader

You are describing those models as they are being applied (and as many introduction present them) not how they can be trained. To understand whether those models converge, how to calculate the solution to even a basic linear regression, you need graduate math: matrix algebra, differential calculus, topology.

View on HN

To me, your statement is like saying an airline mechanic needs to have a pilot's license because he needs to know how the rudder affects the flight.

There are three different roles that I think you are conflating: Designing, Building (and maintaining), and using.

Each has a different skillset. But to think that you need to determine the convergence of models makes no sense to me. Why can't an undergrad build a simple classifier?

And in finance, CAPM uses regression. Every single finance undergrad in the country learns it. Are you implying that they can't because they don't know how the model will converge with something else?

Seriously, it really frustrates me when people look down on undergrads and say "Oh, they can't do it". Baloney. I've met tons of really smart, focused undergrads. And some pretty f-ing stupid grad students (at a PhD level to boot!).

Two decades ago, you needed some pretty heavy CS to build a web page. Today, my mom can do it. With your line of thinking, that wouldn't be possible.

I like the metaphor, but I think it would be more appropriate to flip the roles: it’s more akin to asking the person using the models (or the airplane in your simile, the pilot) to understand how their work (i.e. have basic mechanical understanding). As it happens, that’s precisely what we ask pilot: they need to have enough mechanical engineering talent to inspect the plane in detail before take of as well as understand, diagnose and fix a problem in the air.

To come back to modelling: I see daily people with a cursory understanding of models misusing their tools and not noticing blatant problems. Just today:

- someone using random forest to predict a quantity (which you can do, it’s often nicknamed “random tree regression”) so the first thing their did was to turn quantities to predict to as many categorical variables;

- someone else, on the same problem didn’t see why having a massively higher mean square error on the out-of-bag sample compared to the test sample was suspicious.

I’m not saying that five years of university will solve that, but I’m noticed that taking the time to dig into the model, vs. assuming it’s like a car -- you only need to turn the key and go with it -- is strongly correlated with making those mistakes.

I don’t see how building a web page is any different now than it was in 1998, if anything it seems like everyone has made it harder to build correct pages.

If you stick to the basics of static html content + styling it's actually a lot easier because there are far fewer browser 'quirks' to work around nowadays (you rarely need browser specific code), and things like flexbox are a lot more intuitive than hacking around with floats.

I'm a front-end dev (amonst other things), so I'm pretty comfortable with React, Angular, etc. And therr are definitely cases where they make sense. But simple static or server side rendered sites are much simpler now.

Unless you mean some sort of differential topology, those are rigorously covered in and respectable math undergraduate.

I believe that’s the point of contention: I’ve you’ve gone through those programs, you are a graduate.

To be honest most of those are covered at a good-enough level in the sophomore and junior years of a math undergrad. You don't need measures, differential geometry, or even epsilon-delta analysis to do ML which pins the calculus requirements pretty much to whatever proper multivariable calc class one takes in their sophomore year

Edit: if my school wasn't so obsessed with teaching CS majors diffeq (probably just as a gpa filter...), they could already fit in the requisite math for a solid ML understanding

I would argue that you need measure and differential geometry to understand Support Vector Machine and the kernel trick properly.

I think my contention is less things like being formally introduced to ‘epsilon-delta analysis’ (not sure what that is) but more that people trying to cut corners by skipping a semester of differential calculus tend to also skip a big part of the explanation around how models really work. They tend to not grasp what is convergence, get very confused in higher dimensions, and assume ‘harmless’ short-hands like: ”you should aways normalise your data, in some cases, you need to, but why actually remember why, just do it”; “as long as it’s not overfitting, the model is fine” -- without really much recourse when things are not acting as expected.

You need differential calculus in R^n, but there's no need for the full force of differential geometry. Also, I don't think that measure gains much in terms of understanding, but it certainly is needed to do some proofs the proper way.

I agree that cutting corners is something I would be super skeptical of in this degree. It should really be an offshoot of a mathematics program, not a CS program with the bare minimal mathematics included. It's end goal is probably PR, money grab, and pumping out students that are really attractive for doing analytics grunt work.

Yeah, well those are not graduate courses, maybe the exception of topology (for a non math major). Those are 1st and 2nd year undergrad engineering/CS courses.

I had courses in each of those topics as part of my undergraduate math degree. There’s no reason an AI degree couldn’t include those as standard curriculum for undergraduates.

That's undergraduate math at most universities.

Reply on news.ycombinator.com