Math Symbols Explained with Python
amitness.com
amitness.com
1. "Pipe" should be called "bar" (IMO)
2. "Vector norm" is a much deeper idea than just "2-norm" and if you limit yourself to the 2-norm, you will run into problems
3. "Set membership" is not called "epsilon"!
4. I wouldn't describe functions as operating on "pools", but that's just me. ("Pools" seems to imply vector arguments, or at least set arguments, which is not how functions are usually thought of operating, though they are defined that way.)
5. The description of R^2 as "2-D array" is wrong.
6. I've never seen that notation for elementwise multiplication, though that doesn't mean much.
7. I've rarely seen that notation for dot products. Usually the center dot is much more common.
8. Hat is... nonstandard. It has many, many meanings.
The lack of universality of meanings of symbols is a feature, not a bug, of math notation.
(There's a talk about it but I can't find the link right now.)
Proofs are just tools for articulating arguments about these subjects. Deduction just tends to work better in math, so the methods are more refined and agreed upon.
Often times the symbols may not even be concrete numbers. Often the “real” number may not even be representable on a computer. Often a symbol does represent a concrete number but indirectly defined by some set of rules (e.g., x is the smallest eigenvalue of M; or y is the greatest value less than the supremum of f) that may require a whole library of code to calculate.
The mathematical notation is “meant” to be flexible and manipulated, not to be interpreted (necessarily) as a rigid computational process. It should also be noted that while some notation comes from convention, a lot of it is improvised and contextual!
How many people will be coming to the maths of machine learning but _not_ expecting to implement it in a programming language?
Not denying your point: the maths is fundamentally broader. But the article doesn't refute that. The opening sentence makes that pretty clear:
> When working with Machine Learning projects, you will come across a wide variety of equations that you need to implement in code.
I’m not saying there isn’t any value showing somebody how an elementary summation corresponds to a for-loop. I am saying it’s a lot more than that, though.
As a teaching tool I think it's valuable. But it's important not to identify the math with the analogies that are used to teach it.
Eh? Code can be just as generalized and "deep". The code may (probably) not be usefully executable, but it could describe any concept. At that point it's just "math" again, albeit with different format and symbols.
So I guess your point stands, but its a very thin distinction of common usage.
Code, at the end of the day, more often than not, is written to express some logic to be executed by a computer. At least 80%, if not more than 98%, of code is written to express imperative commands to a machine. I say this as a fan of abstraction, Lisp, Prolog, etc. I just don't buy that
s = 0
for x in a:
s += x
return s
s := \sum_{x\in a} x,
The danger here is if you start equating math to code and you write an algorithm in, say, an imperative language on a uniprocessor, you can become blind to declarative/parallel solutions. The math is the Platonic ideal of the thing in question, and it doesn't change. Always keep the math in mind and don't get lost in the code.
It's hard to imagine someone learning important fundamt math concepts without using math notation, so anyone who benefits from this article should probably take a detour through a math book before continuing their Python ML, or else risk having beautiful code that implements nonsensical math.
That seems akin to suggesting someone should learn music theory before ever attempting to pluck a guitar string. I personally learn best by putting a subject to practice and figuring out the nuances by experimenting. No one expects beginners in any subject to be capable of producing quality work, whether programming or learning to cook.
On the other hand, you can never become a great, or good, or even mediocre ML engineer without understanding the math. If anything, in a production business environment, if you are responsible for creating the model yourself, you'd just be doing more harm than good. Yes, you'd be able to use pre-baked frameworks to run data through an algorithm or model that you have no idea of what it's doing or why and get some numbers and graphs out, but you won't be doing actual science or modeling. You might know how to use Python to do something as simple as run a linear regression, which if you don't know the math, you'd just have a vague understanding of it "finding the best fit line", whatever that means. However, you wouldn't understand that (under the L^2 norm) it's minimizing sum of squared errors, the properties of that (BLUE [1]), whether or not you'd want to apply regularization [2] or not, statistical tests of whether or not a linear fit is even applicable, the importance of outliers due to their overweighting under the L^2 norm, etc.
If I were responsible for a modeling or prediction project, I would never trust it to a software engineer that didn't understand the math of what's actually going on.
> so anyone who benefits from this article should probably take a detour through a math book before continuing their Python ML, or else risk having beautiful code that implements nonsensical math.
I couldn't agree with the GP more. That is 100% spot on.
[0] https://www.youtube.com/watch?v=zucBfXpCA6s
[1] https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem
I disagree, you're equating being good at something to being a world class expert. You can become a great python developer without ever knowing how the language implements a dictionary. You can't expect to remove the black box aspect from everything you use, that's just impossible.
>so anyone who benefits from this article should probably take a detour through a math book before continuing their Python ML, or else risk having beautiful code that implements nonsensical math.
My response to that comment was you can learn both at the same time. I preferred to learn music theory while also learning how to play the guitar. I also preferred to learn mathematics subjects while applying them, such as machine learning.
Given that I look at Python everyday and only consult math books when I need to learn a new algorithm, seeing the basic symbols translated into my everyday language is pretty helpful.
Most mathy people just multiply left-to-right, while a person with some CS grounding will use dynamic programming [1]. If you are working with Dask-sized arrays, this is important.
[1] https://en.wikipedia.org/wiki/Matrix_chain_multiplication
x-dot and x-tilde are also in the same family and I'm sure there are lots of other symbols that people have put on top of x too.
People come here from many angles. Mine was largely free of math. I gather yours wasn't.
There are plenty of articles on learning hiw to learn but fewer on the process of learning what to learn (plenty of lists of “what you should know” though). If you’re just past the very beginning of learning something new, being exposed to stuff quite a bit outside your competency set can be valuable — if you can mostly grasp something then the stuff you can is pointers to expand your knowledge. While a fully advanced treatise is likely gibberish.
I also come to HN for the more technical links but often read the comments even on basic articles like this as they may have interesting pointers.
Also: I’ve been writing systems code for 40 years: bring up an os on bare hardware, write small real-time kernels, build high performance distributed systems, work on compilers, etc. I could probably write a (crappy) algorithms or datastructures textbook coz yes, I use a lot of that stuff. But quickly write a secure, reactive web front end for a simple CRUD app that let you browse a catalog? I would struggle. So one person’s obvious is another’s “no clue”.
However, the approach of understanding math through code is still very helpful, I think. Personally, implementing things that are fuzzy mathematically provides immense clarity once I write them down.
For example, a simple monty hall simulator[1]. Or implementing matrix multiplication multiple ways to understand why each is equivalent[2], and why multiplying A(BC) can sometimes be faster than (AB)C[3].
I am not sure why this helps me. It may be because I was "raised" as a coder, and so that is how my brain works. But I also think that implementing something in code is very close to constructivist mathematics, in spirit. You cannot prove anything if you cannot construct (/implement) it.
[1] https://github.com/mitchellgordon95/implementing-paradoxes/b... [2] https://github.com/mitchellgordon95/implementing-paradoxes/b... [3] https://github.com/mitchellgordon95/implementing-paradoxes/b...
https://mitpress.mit.edu/sites/default/files/titles/content/...
"Structure and Interpretation of Classical Mechanics" by Gerald Jay Sussman and Jack Wisdom
> There has been a remarkable revival of interest in classical mechanics in recent years. We now know that there is much more to classical mechanics than previously suspected. The behavior of classical systems is surprisingly rich; derivation of the equations of motion, the focus of traditional presentations of mechanics, is just the beginning. Classical systems display a complicated array of phenomena such as nonlinear resonances, chaotic behavior, and transitions to chaos.
> Traditional treatments of mechanics concentrate most of their effort on the extremely small class of symbolically tractable dynamical systems. We concentrate on developing general methods for studying the behavior of systems, whether or not they have a symbolic solution. Typical systems exhibit behavior that is qualitatively different from the solvable systems and surprisingly complicated. We focus on the phenomena of motion, and we make extensive use of computer simulation to explore this motion.
They are basically making the computer do the work, with emphasis on unambiguous, computable notation.
(It could be considered a companion to SICP.)
X = [1, 2, 3]
from statistics import mean, fmean
mean(X)
# may or may not be preferable to
sum(X) / len(X)
Product of a terminating iterable:
import operator
from functools import reduce
# from itertools import accumulate
reduce(operator.mul, X)
from numpy import linalg as LA
LA.norm(X)
Function domains and ranges can be specified and checked at compile-time with type annotations or at runtime with type()/isinstance() or with something like pycontracts or icontracts for checking preconditions and postconditions.
Dot product:
Y = [4, 5, 6]
np.dot(X, Y)
Unit vector:
X / np.linalg.norm(X)I would write the explanations like:
result = 1
x = [1, 2, 3, 4, 5]
for number in x:
result = result * number
print(result)
E.g. this (Latex):
$C_{ml} = A_{ijkl} B_{ijkm}$
becomes (in Python):
C = einsum('ijkl,ijkm->ml', A, B)
[1] https://docs.scipy.org/doc/numpy/reference/generated/numpy.e... [2] https://en.wikipedia.org/wiki/Einstein_notation
E.g. Gradient Descent: formula, code, and visualization:
https://github.com/stared/thinking-in-tensors-writing-in-pyt...
can be simplified to:
math.sqrt(sum(v2 for v in x))
since when 1/N means "len(n)"?