And some linear algebra book by Strang (as others have said.) I don't remember which one I learned from, but I do remember it was excellent.
I can't recommend a good stats/probability book. People recommend Statistical Inference by Casella and Berger, although I wasn't super impressed with it.
You'll also want a good understanding of set theory and proof writing. I don't remember what book I learned from, but try to find some introduction to math thought.
This is all assuming you have a good understanding of algebra. The above mentioned books require geometry/trig, but you don't actually need that for machine learning.