Math For Programmers
steve-yegge.blogspot.com
steve-yegge.blogspot.com
The problem I get is that whenever I see a big equation I tend to see a few greek symbols that I have never seen before and I have literally no idea whatsoever what they mean.
At least when looking at some unfamiliar code etc I can usually tell from function and variable names approximately what something is supposed to be.
Wikipedia is hopeless for learning because the math articles seem to be more interested in being complete and accurate than being accessible. When I hit a math page on something on wikipedia it tends to link off to a whole bunch of other pages on increasingly abstract parts of Math leaving me more confused than when I started.
I'd never have time to go through all of these topics and learn them properly, if I was going to do that I'd have become a mathematician.
Khan Academy is good, but seems to focus more on the mechanical skill of doing math rather than explaining concepts.
It depends what you are reading, but a good discrete math textbook provides a lot of groundwork for mathematics commonly used in computer science, but expects little in terms of prerequisites, so notation is usually explained.
At some level, learning the fundamentals is just necessary, but for computer science discrete mathematics and linear algebra (the basics of which are easy and also don't have much in terms of prerequisites) will get you far.
On the other hand higher math is ruthlessly cumulative. If you find that you need to understand Lagrange multipliers, for instance, you'll see they relate to the gradient, which has its own symbol, which is defined in terms of partial derivatives, which have their own symbol, which relates to limits of vectors, which have their own symbols and notations, etc.
Physics 1 Advanced, Simple Harmonic Motion is being taught. When told to find a case where the second derivative of a function is a negative constant times the function, I think e^ix. The answer is sin(x), but Euler's equation now makes sense. The most beautiful equation now makes sense.
A month later, glancing in a book of useful mathematical equations, I see sin(x) = (e^ix - e^-ix)/2i. Hyperbolic trig suddenly makes sense, which is useful over a year after finishing it. The weird equation for the normal curve from statistics starts to make more sense, because e^(-xx)=(e^(ix))^ix, which helps explain the 1/sqrt(2pi) weirdness. I start to understand why the most beautiful equation has its name.
I think that most math is cumulative, but only some parts are hard enough to make you notice it.
There are many good resources to learn math, most in the form of books (some of which have digital versions). Personally, I find it best to find a subtopic that interests me and dive in. Though I will say that I would recommend "Chapter Zero" to most people wanting to get back into math after a long break.
I simply mentioned it here since Steve advocates just diving in and following links on wikipedia as a way to learn a subject.
This might be a great way to find out what stuff is out there to learn or for somebody more familiar with Math to get a synopsis of a topic but as a learning resource for a newcomer to any particular field it seems likely to reduce you to a jibbering wreck very quickly.
http://betterexplained.com/articles/an-intuitive-guide-to-ex...
My goal is to explain ideas as I wish they were shared with me: informally, with the primary focus on intuition (there are plenty of places to practice the mechanics). Hopefully it can come in useful for you.
Textbooks are fantastic, though. They are written to be self contained and will start at the beginning (usually with an introduction to fundamental subjects such as set theory). If you're serious about learning math, they are the way to go.
Mathematical notation is optimized for symbol manipulation using a pen and paper, not communication of information. Even to us programmers, it's a slew of one-letter variable names that very rarely seem to have the same meaning.
Furthermore, those single-letters are not from a character set that has been stuffed into our heads since we were 3 years old, so there's an amount of translation going on, similar to the process of learning a foreign language (hear spanish -> translation to english -> process thought -> generate response in english -> translate to spanish -> speak spanish vs. hear spanish -> process throughout -> speak spanish).
Bret Victor's project seems to be attempting this a little bit: http://worrydream.com/KillMath/
That said, I read the book after I had a math degree, so while it seems very accessible (more so than most undergrad discrete math books), perhaps someone who read it as an undergrad can comment.
I don't know what the point of my ramblings are. I guess math finally made sense after learning programming first, much like yegge proposes in his post. But I had the basics down first, and I was at least familiar with the concepts.
The difference is that to them linear algebra (for example) is something abstract you use to solve linear algebra problems, while to someone with a more deep and solid understanding of math, linear algebra is a simple and general tool that can be applied in all manners of situations.
Being able to see things like "what I really need here is this part of a sine curve, stretched along the X axis, compressed along the Y axis, and moved up into the positive", and then write a simple one-liner to do slideshow animation, has proven useful again and again.
To be honest, I didn't really understand the point of it during class, and actually failed that part of the exam. A few years later when I needed it, it just popped out of my brain attic and made sense.
-- Paul Dirac
http://news.ycombinator.com/item?id=87393
Hmm... a feed sprinkled with systematic reposts of things that you enjoy, that motivate you, or that you don't want to forget. Has anyone tried that?
edit: or rather, a few intervals to allow for reposts even after that year period
I ask because sometimes I tried to submit an article that was already there and HN just pointed me to the original discussion, so I don't understand how can duplicates happen.
I guess in some sense it is, but for myself in 2012 I recall first hearing about the work of both Claude Shannon and Andrey Kolmogorov as a university student in 1982 in a first year maths / CS bridging lecture called computational mathematics or some such.
Pretty much everything to do with signals, compression, error correction, etc. happened post the invention of the telegraph - you had Gray Codes (patented Frank Gray ~ 1945ish) popping up in recreational mathematics in the 1880s and being applied to the telegraph by Émile Baudot, from that time forward we see the unfolding of much of what is CS / computational mathematics today.
I guess everything last century is "kind of new" but it is starting to feel as though it's all been around for a while now.
That takes you all the way to the mid 19th Century for what little of Riemann's work the course focuses on, with the bulk (Newton's and Leibniz's contributions) coming from almost two centuries earlier.
Beyond that I think information theory often tends to be treated as more an area of engineering than math, along with digital signal processing, to take another common example. So it's usually taught by engineering or CS departments if by anyone. At schools with a strong mathematical bent to their engineering/CS departments there are usually good courses on both, though.
http://www.cs.ox.ac.uk/softeng/subjects/SEM.html
Which is open to anybody willing to pay ~$2k for a week and get their ass to Oxford - you don't need to be enrolled as a student.
If you don't have that kind of money to spend (or can't get to Oxford), the course is essentially the first few chapters of: http://www.usingz.com/ which is available for free.
However long you spend learning maths, probably more than any other subject, you are only going to end up knowing an awful lot about less and less (the usual curse/joy of academic specialization).
Would be interesting if Steve Yegge can do an AMA on reddit.
I figure i've really only missed out on a couple years of math education compared to the people I know who I consider 'good' at maths. And to my advantage I did do some statistics at university. With all the improved learning materials available to me, plus an alliance with programming, and my improved bullshit-detector for bad teaching and studying practices, it should be a breeze to catch up.
One problem I still have though is that maths just gets so incredibly boring... at least classes do. I used iTunes U and Khan Academy to study calculus and linear algebra. I had to start skipping past some of the really mechanical parts, because as the article said, as a programmer you just think 'put that in a function and never worry about it again.'
Breadth not depth is definitely what I'm after, although I do worry that it's the equivalent of being a musician who knows lots of diverse harmonic theories but still hasn't mastered some scales that would let him/her jam with other musicians.
(One thing the endless triangle problems do provide is practice at converting geometric problems into algebraic ones. But yes, too much of them I'd say.)
I guess teachers consider complex numbers an abstraction too far in order to teach applications to geometric problems. They don't want to field endless 'woah that blows my mind' type questions about i at that stage. The downside then being that you have to memorise (or read off a cheat sheet) a lot more identities in order to manipulate things algebraically. And to convince yourself of the truth of those identities you need to rely on geometric proofs, which come more naturally to some than others. Probably makes the subject more intimidating than it should be.
For example, if you write code for some user interface, you'll have a very hard time to get smooth animations if you don't understand derivatives.
As a physicist (turned programmer turned physicist again) I'd just like to say that I quite like the idea of a liberal arts mathematics course - I would be much better at it if I had an overview of the field, and thus knew where to look for my solutions (like I do when debugging a program under X environment or framework).
That paragraph made my left eye twitch a little. Not a big mistake, perhaps, but what he calls probability theory is actually combinatorics. Probability theory is decidedly non-discrete (integration of density functions?) and involves only two integers: 0 and 1. (A joke).
Many people are introduced to combinatorics in the context of probability theory. It (combinatorics) can be used to compute various probabilities, but still. You wouldn't call trigonometry calculus just because a course in calculus might involve the derivative of sine and cosine, so don't confuse probability theory and combinatorics.
Probability does use some tools from combinatorics and vice versa, but the two subjects have vastly different goals.
Similarly I find it highly questionable whether combinatorics is "clearly more relevant to general programming". In fact, what is "general programming"? Probability theory pervades all of Computer Science - from probabilistic algorithms (QuickSort), through cryptography, optimization algorithms (genetic algorithms, simulated annealing), networking (information theory, queuing theory), machine learning, the list goes on and on. For business programming, statistics (based on probablity theory) is crucial. Some of those applications include combinatorics, but I find it harder to find such a long list of applications of combinatorics being used without probability theory - it is a more specialized field.
But, it's important to appreciate that, even for probabilities on finite or countable sets, you get pulled into continuous math. I don't think we disagree here, but maybe some examples would be worthwhile, just for definiteness.
Expectations (means, variances) of discrete variables will be real-valued. There are continuous processes that are intimately connected to what you thought were purely discrete outcomes (e.g., the relationship between Poisson counts and exponential waiting times). There will be limiting processes of discrete structures that bring in continuous probabilities (the Binomial -> normal limit, and all its generalizations) and provide considerable insight. The Stirling formula, which is key to insight about factorials, comes from calculus.
Finally, generating functions are one of the main tools for solving the discrete summations you mention, and these are continuous, and pull you straight into complex analysis. ("Who changed the subject here? I was just trying to add some binomial coefficients and now we're talking about derivatives of analytic functions at zero?")
Or, as my edition of Concrete Mathematics says (sec 5.4): "We come now to the most important idea in this whole book, the notion of a generating function."