In my opinion, physics students learn the best framework for thinking and get a very good mathematical intuition. For example, here's a problem from an introductory QM book that really threw me for a loop when I was studying:
A needle of length L is dropped at random onto a sheet of paper ruled with parallel lines a distance L apart. What is the probability that the needle will cross a line?
A circle of radius L is centered x far away from a border 0<x<L. This is because one end of the needle will always end in some zone (the center) and the other end will be L far away (the circle).
How much of the 2pi boundary is outside the zone?
When x -> 0 then it's going to be 50% since one boundary line becomes a tangent and the other goes through the middle. When we move x by k (e.g. f(x+k)) then 2k new points will be added on the left side while 2*k points leave the boundary on the right side. When x=L/2 then the boundary lines will split the circle in four equal parts (since they're tangent to the radius at r/2 on both sides) so intuitively its 50%.
This is true. I can't really say why, but after my Discrete Mathematics class a lot of my Computer Science problems became a lot easier to reason about.
Thickness of line is needed right? Otherwise P approaches 100% as thickness approaches 0?
The "lines" are like a sample of a point from a uniform distribution U with width L, and h is an interval inside U. The probability a number sampled from a distribution of width L will fall within interval h is h/L. Substituting for h gives p(cross|x) = sin(x).
Then assuming the needle is equally likely to drop at any angle, for any one angle theta we get probability density p(theta=x) = 1/(pi/2-0)= 2/pi.
The probability the needle drops at angle x AND crosses a line is the product of p(theta=x)p(cross|x)= (2/pi)sin(x). As mentioned, x can range between 0 and pi/2. To get the probability the needle drops at angle x1 OR x2 OR x3, etc and cross we need to sum all these. So take the integral of (2/pi)sin(x) from 0:pi/2. This gives 2/pi.
You will be surprised: you can make significant gains by including the zip code - i've seen that happen in a competitive setting. Where you live probably contains some signal about your credit worthiness.
Having said that, of course it doesn't make sense to simply feed the raw zip code to the tree. An appropriate encoding (most people would use a one-hot encoding, though there exist better ones) of the zip code will be key to extracting signal in a robust way.
>> ... the tree will have serious overfitting issues
Isn't it an almost standard practise now to use an ensemble of trees, such as a random forest? Decision trees have long been known to be prone to overfitting.
I guess I should still brush up on math though, it seems.
2. math prereqs at UCSC:http://people.ucsc.edu/~praman1/static/pub/math-for-ml.pdf
3. math prereqs at UMD: https://www.umiacs.umd.edu/~hal/courses/2013S_ML/math4ml.pdf
- UML
- http://goodfeli.github.io/dlbook/
- 3 books: Barber, MacKay, and Rasmussen/Williams from this list: https://www.reddit.com/r/MachineLearning/comments/1jeawf/mac...