Ask HN: Math for Programmers? | Hacker News Reader

Ask HN: Math for Programmers?

104 pointsDeathArrow45 comments

What do you think it's essential math a programmer should know?

I've learned higher algebra, calculus, some geometry which was eased by the already known algebra, number theory, probabilities, combinatorics, graph theory, statistics, formal language and automata theory, and that's mostly it. Most of my CS courses in university required some kind of math which we previously did.

I can't say I remember all I've learned but I can recall things if needed.

It was a thing that programming languages were explained in terms of formal language theory, plus some algebraic constructs. Now a days whenever I read about a programing language, I read about mostly category theory or functional theory and it mostly doesn't make sense.

So how hard or how many math domains should one learn to be able to understand all major CS theory?

View on HN

I'm writing a book just for you :)

It's called "Practical Math: A Tour of Mathematics in Production Software" (https://pmfpbook.org points to an announcement and mailing list)

The idea is to give a large swath of examples of math used to solve real problems, and collectively to give a good answer to your question. And to have the examples be appreciable by the average programmer, in the sense that they could reasonably expect to adapt the underlying ideas in their own work.

> It will be much easier than pimbook to sample and read casually. It could be a bathroom reader for the stalls of software firms.

Thank you, I have a copy and it went over my head a couple times so I guess I wasn't exactly the target.

Also not sure if this fits within the scope but something like this would be great for those not already familiar with it, helped me a lot - https://github.com/Jam3/math-as-code

The pimbook itself looks fantastic. As an aside, any recommendations on the inverse, a programming introduction for mathematicians.

I don't know about mathematicians, but everyone should read How to Design Programs book when they are starting out. Fantastic way of forming great mental models.

I can't join your mailing list, as my Email-Address looks "fake or invalid".

Perhaps the concepts of vector embedding and the attention mechanism could be explained. Also you could use some prompts for LLM to generate new content for your intended audience, mostly to classify them, so that your reader could copy your prompt and ask for further clarification.

Example: In chapter 5 here is a prompt that resembles the content of this chapter, to solve some of the exercises or clarify some points, your prompt would be along the lines ...

Just some ideas.

Category theory has very hard definitions and not many theorems. Would not recommend. It has Grand Unified Theory vibes so pseudo intellectuals have a hard on for it.

Don't waste your time trying to understand people who write to make themselves sound smart. You aren't missing out on anything.

Here is a recent article from TOPLAS, imo a top programming language journal. No category theory here...

https://dl.acm.org/doi/10.1145/3564619

In my sole experience working at a company (startup) that embraced those who understood category theory, those types were not the ones producing any discernible value for the company. They instead attempted to browbeat certain thinking into others at the expense of their focus. I'm a fan of "sane FP," but the pseudo-intellects left a horrible taste in my mouth on the subject of the underlying math that I very likely will never recover from.

I want to clarify, I am not trivializing the work of category theorist. It is a field of mathematics which provides a useful framework for algebraic geometers and algebraic topologists, and has even been fruitfully applied outside of math.

A personal favorite of CT generalizing an existing idea is in knot theory the khovanov homology detects the unknot, which is an amazing feat for a knot invariant. Bar Natan covers the construction well: https://arxiv.org/abs/math/0201043

However for some reason this particular field of math is a lightning rod for quacks. I don't really know why, but that is the way things are now.

Agree 100%. CT is fun if you like abstract mathematics. But it is not a practical useful tool for developing software. A much better choice would be “Software Foundations”. Now that is also hyper abstract math but useful for developing real world software. As demonstrated with the CompCert and seL4 projects. Even pro mathematicians are starting to dive on (see the LEAN mathlib project for example).

I wrote Geometry for Programmers (https://www.manning.com/books/geometry-for-programmers) and even so, I don't think that math knowledge is really essential for programming. It is, of course, a non-perishable competitive advantage so from a career perspective, it makes sense to invest in mathematical education as early as possible. But it's not truly essential. Mathematical illiteracy does not disqualify a programmer from being a programmer.

In a modern technological climate, math savviness feels more like a superpower from the 50s comic books applied to a real-world profession. Would X-ray vision make you a better doctor? Probably. But nowadays, there are tomography machines in every hospital. Would flying make you a better urban planner? Maybe. But there are drones and satellite imagery. Would learning geometry make you a better game programmer? Sure. But there are engines and frameworks that do your math for you.

Surely, knowing geometry behind your tools enables you to outperform the "no math" solutions. Sometimes drastically. But today, it's more like a bonus not a necessity.

Counter point - I am terrible at Maths, like awful.

I failed Maths for Computer Science in my degree but managed to get a pass on it as all my other grades were great. Overall it has really not affected my life in any way or my career throughout my 15 years as a code monkey through many roles into now a CTO position for the past 5.

There are a lot of things and patterns you pick up as you go, and maybe I just got lucky and was in the sweet spot of tools developing fast enough to act like a permanent crutch for lack of mathematic ability.

I guess this is different if I were planning to be a game programmer but for a web focused engineer it has been absolutely fine

I would agree, your typical dev writing CRUD apps needs an ok grasp of algebra, and that's about it.

What would you say they need the algebra for?

Let's suppose the business logic in your basic CRUD app needs to calculate some value based on one or several DB fields. You need to know enough algebra to convert that into a formula that you can use in code.

I've worked with lots of engineers and programmers who are in the same boat, and they do just fine. I personally think that a person has to be genuinely interested in math beyond maybe high school algebra, in order to do anything with it beyond passing their college classes.

A few of us are happy to take care of math for everybody else. I'm in that boat. And I can't even honestly say it's gotten me further then the math-averse folks, but maybe it's helped me compensate for some of my other deficiencies.

Depends on the domain you’re going to work in. Nobody understands all major CS domains in depth - you typically specialize and dive deep in the relevant area. It’s like any other field: no physicist knows all major areas of physics, no mathematician knows all major areas of math. Perhaps 50 years ago when the field was really new it was possible to understand all of CS, but it’s a big field now that covers a really diverse set of topics that each are relatively deep.

Even then, you often don’t need to dive deep into the math unless you’re a working researcher for the most part. I work in languages research, and to be honest, for most of the work you don’t need to dive deep into type theory or category theory (regardless of what some internet people would lead you to believe). You generally only need to dig deep into those if you find yourself working in foundations topics where they play a strong role.

This is not a math textbook. It's a listing of the math that I am glad to vaguely be aware of, without anything that requires true understanding, because I don't have that, but with references to what the people who do have it can do!

It's 90kb of pure text and covers a lot of domains with just enough info you could figure out how to make a CAS do the things, with maybe some googling. It's had some reviews, and it fact I think is the only project I've ever got PRs on, but may be incomplete or inaccurate, I can't prove most of this for myself, just test it or find references.

There are examples but no excercises, my assumption is that math is a computers job, and I only give a starting point for any study beyond "This exists and a machine can do it".

I assume geometry is for the CAD and arithmetic is for the calculator, and actually knowing how to do it yourself is for the curious who Google it after finding out what it is.

Further contributions welcome, especially in the domains I haven't covered (Graphs, automata, and lambda calculus would be great of there's anything to add that matches the level and tone I'm going for).

I'd love to have more CSy stuff, right now it's all from an embedded, wedbdev, and 3d printing perspective.

https://github.com/EternityForest/AnyoneCanDoIt/blob/master/...

That's an awesome github post!

I shared it direct on HN - https://news.ycombinator.com/item?id=35357865

My Personal Opinion.

If you can do real life tasks like balancing a checkbook or figuring out how to split a restaurant bill, you probably know enough math to be a pretty good or even elite programmer, except for specializations like working on 3D graphics.

Most programmers aren't inventing new calculus concepts from scratch as part of their problem-solving. Most are doing fairly simple order of operations (PEMDAS) types of math problems and glueing CRUD APIs together. The level of math required for that can be done by a solid middle-schooler.

If all you are doing is programming of the everyday industry sort, none. You can have a fruitful interesting career from frontend to back without thinking about formal math at all.

If you want to start specializing in graphics or designing languages or algo, then, yeah, you'll need math, but at that point, you'll know what math. No need to learn it beforehand.

Ok, given your university background in CS, it sounds like you have a good base of understanding. I think there are many narrow verticals, but I think you're through the base of understanding. I think people can answer specific questions, but not more general ones.

Traditionally, try response would be: have you taken databases, compilers, operating systems, or more extensive computational complexity theory. Honestly, the complexity theory enables napkin math for quick mental, or mid-discussion, feasibility arguments.

I would recommend AI and ML familiarity, including transformers, which I view at first glance as an interesting amalgamation of applied statistics and computational complexity. Numerical stability was (and is) a hard problem about practices needed to get consistent and useful results from numerical algorithms.

I think your bases are covered. It sounds like you know what you don't know. I might recommend collecting interesting problems, and learning how knowledge from each of those fields contributes to solving a class of problems, quickly, easily, simply, or efficiently.

What do you hope to do, to direct your mental energies toward?

Getting started, algebra was enough for me. However, as I got slammed by algorithmic complexity problems like nested loops and needing to pick out the proper type of collection (data structure/sorting) for holding information, things like Big-O notation and Discrete math became more interesting. Once I started doing work with financial analysis and other optimization problems, calculus became quite useful, even if it was just to calculate an occasional slope. When I wanted to understand the underpinnings of deep learning, linear algebra concepts like working with vectors and matrices as well as more calculus was quite handy. Now, I'm really interested in quantum computing and having only scratched the surface of linear algebra, I need to go back to the books to build a foundation to move forward on. I suppose it's an individual choice for the direction you want to go in.

The average programmer could probably get by with high school maths and just enough college-level statistics to know the difference between mean, median, percentile, and standard deviation.

I'd argue the majority of programmers and software engineers don't need anything beyond being able to work with percentages. But I also argue that the majority of programmers and software engineers are not computer scientists and normally do not need or use much computer science knowledge. Computer scientists probably use math a lot.

Probably not required, but understanding the concept of a statistical distribution has come in handy many times for me when trying to improve the latency of systems. If you are unaware of distributions, you might measure latency multiple times and then calculate something like a mean or median. When thinking in terms of distributions you are more likely to draw something like https://matthiaslee.com/content/images/2017/05/bimodal_dist.... which contains much more information. That example tells you that you have at least two different kinds of latency. It could be 2 different use cases, 2 times of day, cache hit vs. cache miss, etc. It's enlightening to know that there are multiple factors and how much they differ!

Algebra-driven design (book):

https://algebradriven.design/

Category theory for programmers (free book):

https://github.com/hmemcpy/milewski-ctfp-pdf

“Software Foundations” is IMHO a much more useful choice than CT.

>So how hard or how many math domains should one learn to be able to understand all major CS theory?

Understand "all major CS theory"? ...you need all the math

However, you'll never understand "all major CS theory" (maybe Donald Knuth does...but no one else does, that's for sure) - so focus on what you want to know/do, then study the math that's related to it (at least enough to be passingly conversant)

Best way to figure out the math you need to know is to survey the math requirements of several CS degree programs ...people far smarter than you have been working on this far longer than you've been alive - and their work is show in the degree requirements :)

As others have mentioned, it really depends on how expansive is your definition of major CS theory. In CS research, one finds math results being used from all over the place ranging from dynamical systems to non-monotonic logic.

My guess is first-order logic, set theory, combinatorics/graphs, rigorous probability/analysis, and abstract algebra. At least being familiar with concepts from these areas would help make sense of what's going in, say, complexity theory or cryptography.

I put together a guide to learning complexity theory at the graduate level [0]. This may be a decent proxy for your question.

[0] https://bcmullins.github.io/complexity_theory_resources/

Set Theory and Abstract Algebra really came in handy for Comp Sci.

All of Relational Database theory seems to be based on it.

Same with Logic and Predicate Calculus, Linear Algebra and Statistics.

You know what I have barely used though: Calculus and Trigonometry. Basically most things you study for a Physics major.

But that's just my experience.

Depends on what you mean by major CS theory? I have sadly learnt that Software even at a high level at a big company mostly doesn't need any math skills beyond max high school math.

Off course there are exceptions. But mostly there is not much essential math a programmer should know.

Over the last 30+ years, I've found automata theory to be pretty much indispensable. I can't even begin to tell you the number of times I've run across hideously complex logic that I replaced with a simple and maintainable finite state machine.

My whole career has been in-house industrial stuff. I've never used past trig--and that only to draw a picture of a toilet. (Legitimate--while we didn't do anything with toilets the graphic marked where it would be, thus where our stuff couldn't be.)

I have found a possible use for error correcting codes but that's way beyond anything I ever studied and thus it probably will never be implemented.

You asked about math for programmers, not for computer scientists. Two different things.

The math you know is more than enough. Some specific domains you go into may need additional math, but that's because of the domain, not because of programming in general. If you need more, learn what you need when you find out that you need it.

It really depends on the area of CS. The math used in ML and FP don’t have a lot in common.

> I've learned

You’ve got all you need. You can learn programming now.

They mentioned probabilities, but not measure theory. They might want a deeper and more intuitive understanding that they'd get through measure theoretic probability before going too deep into programming.

I hope that was sarcasm, because it is amazingly wrong. Measure theory is absolutely not a "learn this before programming" thing. It's peripheral at best, completely irrelevant at worst.

Concrete Mathmatics 2nd edition this has saved my bacon several times

Advanced math make my write faster code. My code always outperforms noobs' code by a factor of 10x-40000x, in single threaded pure python without numpy or so. And for some particular operations, my pure python implementation even outperforms numpy behind C and Rust.

This is a joke but programmers only need to know how to +1 to a number

Succ in lambda calculus (or set theory)

Not a joke: all you need is S and K

Algebra: Boolean and general. Everything else is domain specific.

Reply on news.ycombinator.com