Hacker News Reader

"It boggles my mind to think that I used to know advanced calculus, group theory, linear algebra etc. I have forgotten all of it bar the very basic concepts behind them."

I thought it was just me!

edit: we should add confessions like DHH twitter where all these advanced programmers confessed they had to look up print() commands on google or whatever it was.

edit#2: The point might be that you don't remember linear algebra but you remember there is that tool to help. If multiple, parallel computations come up, you know to look up linear algebra. If a rate of change question comes up, you know to brush up on calculus. If the halting problem comes up, you know it isn't solved etc.

A long time ago I gave up the "manager track" and went back to be an individual contributor mostly because i noticed I was having to look up things I "ought" to have known

Slowly now I am settling into the idea (the confidence) that I am actually pretty good at this, and that occasionally looking up how to open a csv file is just an artifact of knowing why and what if not precisely how

(it's also a good indication that another abstraction that is easier to remember might be useful)

I’ve spent the last few months immersed in modern C++ after mainly doing C# at my job over the last few years. I opened up a C# REPL (can’t recall why exactly) and realised I wasn’t exactly sure how to print to the console....

I don’t think learning something and forgetting it should really be justified, it’s just not an efficient use of time.

If you wanted the benefit of knowing tools that help with linear algebra, memorize that, there’s not point cramming the entire thing just for a class.

I think a good remedy to this issue is spaced repetition software like Anki/SuperMemo. It’s a bit messy to use with math, especially with something like a proof, and they’re meant for cards that shouldn’t take you longer than say 10 seconds to solve. It is doable though if you can break things down into atomic components. And it can be worth more than you realize to have usable access to those memories for say decades rather than till the month after your class

I think it’s difficult though to make classes focus on such long-term memories because tests create such poor short-term incentive structures which aren’t easily replaceable. I think the only way to get around this is to just focus on learning what you want and if you’re missing something that seems useful to acquire it on your own rather than being required to study something and just forgetting it as soon as the test passes.

I've also found Anki a good remedy to the "I feel bad that I don't remember this from undergrad" problem. I'm still nowhere near as fast at actually doing maths as when I was taking or teaching it. But now when some basic, relevant mathematical fact comes up in life and I've forgotten it, it just means there's one more thing I need to put into Anki, not that my mathematical knowledge is on a steady, irreversible downtrend.

Likely unwanted commentary but how do you do leech handling/minimum information principle with your math cards? I'm a SuperMemo user and I think one of the biggest failures I see with Anki, in general, is that it doesn't handle leeches well (leeches are cards you repeatedly fail that end up taking most of your time) and failure to adhere to minimum information principle (only 1 thing to recall per card). I think with math this would become even more difficult if you make minute long cards to go over some proof you will definitely be setting yourself up to suffer.

I'm only one year into using Anki and to be honest mathematics hasn't been a huge focus for me, so I'm not a great person to ask. Have you read this article by Michael Nielsen? http://cognitivemedium.com/srs-mathematics

I've noticed that I spend a lot of time on leeches and sometimes I go back to break them down into smaller pieces, but I haven't found it to be a major problem. How does SuperMemo handle it?

Lol, I felt amazing the other day for being able to write an ajax request without looking it up after more than half a year of not coding in javascript. I was already looking it up while testing it and was actually surprised when I saw that it was working.

I tend to think of math as having two parts: a way of thinking along with all the things people thought up by thinking that way. To learn the first, you have to study the second.

Examples of the second are calculus, group theory, algebra, etc. Examples of the first are being able to define useful words with useful logical consequences, knowing when you can infer things beyond what can be directly tested, detecting patterns that might profitably be abstracted, etc. It's a bunch of force multipliers for your logical mind.

Specific subjects were developed with the goal of solving particular kinds of problems, so if you are faced with such a problem, it can be very useful to know what people have already figured out! However, if you're not, then it might not be so useful remembering these details.

>Honestly there have been very very few times where my knowledge of mathematics has directly helped me with coding . . .

> its not directly applicable unless you are doing graphics programming, AI, data science etc.

Well, those are pretty large fields!

If you ever get to touch machine learning, that's all about linear algebra, sparse methods, numerical methods, calculus in many dimensions (e.g. gradient descent).

Any kinds of image or signal processing, and you run into your old friend Fourier Transform, which is, again, analysis, linear algebra, complex numbers.

In my previous job, I got to have fun computing an orthonormal basis of polynomials on a disk, and doing some linear algebra with that (although to be fair, fields like computational lithography are probably more math-heavy than others).

The amount of programming jobs where math would be directly applicable is on the rise.

That's not why I studied mathematics. I do it because it is an art[1][2], and because I can. But it certainly comes with certain benefits in the workplace.

[1]https://archive.org/details/hardy_annotated/page/n1

[2]https://www.maa.org/external_archive/devlin/LockhartsLament....

I think you stumbled exactly into why people say that (including myself) - proofs.

With AI, ML, and Data Science becoming hotter and hotter topics, a lot of the material is still VERY mathematical proof heavy. This [1] is an example of understanding the Lagrange multipliers for SVMs. This is not a worked example, but a repetition of the mathematics. If you are shaky on what dL/db means, then you're lost. It doesn't help that the only other options are import sklearn tutorials, so you only get a slight understanding of the inner workings of something like SVM. You still can't do it by hand, if say for example, you were given a toy problem with like 10 items.

[1] https://towardsdatascience.com/understanding-support-vector-...

I don't think understanding proofs is essential to understanding AI papers.

A lot of papers are "dressed up" with proofs, many of them bad. But it doesn't matter if the technique can be effective.

Sure, but if I'm reading a paper that starts to show equations, I can start getting lost. There's no "knowning" their garbage because it gets over-math'd. I get that's a sign of a bad paper, but my only argument for "why" is because I can't understand it. Not exactly the strongest defense.

The same for stats - if you are uncomfortable with them, then seeing terms like Wilcoxon and Spearman are just confusing. Should the number be high? Low? Or simply, why do some papers have one and others have the other? If you are shaky on stats, its easier to just accept a particular formula was used even if it is not appropriate.

My point was you can understand a paper without understanding all of the proofs shown. Following the proof isn't essential to understanding what the technique is.

There are papers with bad proofs but valid techniques. Because NNs techniques are difficult if not impossible to prove mathematically. Empirical data is much more valuable.

Right, but my point is what if I want to implement the math? Without deep diving into my research, I disagree with your second sentence entirely. It is why I think we are also seeing the rise in Jupyter notebook still articles - because you can explicitly follow along. As a rule of thumb, I try to assume no one understands anything and so I must be very explicit in anything I teach. I find this to be a better baseline than assuming everyone already knows what I'm about to teach.

Back to my point - if I want to implement something I've read in a journal article on something like queuing theory that I'd like to work into my CS education research, having a weaker foundation on math has made it harder for me to implement someone's work.

What you said is true, lacking the mathematical intuition will limit you. But understanding the proof is not essential to implementing the approach.

Academic papers in computing don't tend to be written with lay practitioners in mind.

If I want to try out a technique in a business setting, I ideally want at least one of:

* A library or application I can run

* A code sample I can copy and modify

* A pseudocode block I can turn in to real code

What is less helpful is some mathematical equation in which one of the variables is the result of a technique described in another paper and a whole load of unfamiliar concepts named after people.

Agreed. In fact, code can be more expressive than mathematical notation, and providing code would do a better job of demonstrating a theory.

A lot of the proofs are just to legitimize papers. The cynic in me also thinks that by not providing code, the authors intentionally make it more difficult to reproduce results to hide tampered data. You see the same happen in science all the time with p-hacking, or with just plain fraudulent data that is purposely difficult to reproduce.

Luckily there are a good number of papers that provide a repo link, but not enough yet.

Let me shamelessly advertise a book for programmers I am writing for this exact case: Numerical Linear Algebra for Programmers: An Interactive Tutorial with GPU, CUDA, OpenCL, MKL, Java, and Clojure!

Drafts are available!

https://aiprobook.com/numerical-linear-algebra-for-programme...

Unless you continue to study mathematics, which you will likely only do if you stay in academia, you will forget what you've learned!

You've remembered more than you think, most notably you do (or would if appropriately prompted) remember that things are possible and probably a bit about the approach. You don't remember the details of how, but those are details you can look up if needed - you have the framework in which those details fit and that's the important thing.

Though for the most part I hardly use advanced mathematics in my day to day job, it is sometimes the case that math is like a super power. Where if I programmed the obvious idea to solve a combinatorics problem, the solution would never be tractable. But, then I learn the mathematics of combinatorics, and suddenly I can write a much simpler and much more powerful solution.

So, math doesn't help frequently (or perhaps not trying to use it enough), but when it does it can help tremendously.

It can also help deflate a lot of the hype in AI and ML, especially if you know the fundamental theory of ML.

> Honestly there have been very very few times where my knowledge of mathematics has directly helped me

And that is the intended use of math! Knowing math is like knowing judo or another powerful martial art. You take great effort to learn and master it, not for its "practical applications", but because you are honestly interested. One day, hopefully never, this knowledge will save the day, and you will be the only one around who has it.

I chose to study mathematics in university precisely because of the second reason: logical thinking. By extension that includes the ability to write proofs. Even if you aren't actually writing proofs on the job, getting into proof-writing mode will make your technical exposition much better.

>its not directly applicable unless you are doing graphics programming

You missed the point. It actually is directly applicable.

The difference with math and programming is that the axioms of a program constantly change. A reassignment, a mutation is an axiom mutated.

Otherwise they are the same.

You could invent a style of programming where all things are immutable. Then that style of programming will be virtually indistinguishable to math. I wonder what I would call such a style of programming?

You describes how mathematics can be used to create a new programming language/style.

The OP describes how using current programming languages requires very little knowledge of mathematics.

Both of those perspectives can be true and from my own personal experience the OP's perspective rings true.

No my argument is, if you program you are doing math. That's why the techniques from math are applicable to programming. If you know advanced math you can become a better programmer.