This textbook is a classic, and downloadable as a PDF:

http://statweb.stanford.edu/~tibs/ElemStatLearn/

Don't be put off by what you don't (yet) understand. Even someone with a math PhD, but not specifically in the areas covered, wouldn't recognize much of the terminology they're introducing.

But feel free to jump around, and look for concepts that seem more approachable -- particularly where case studies are presented, often with very nice charts and diagrams -- and you can get a feel as to whether it's something for you.

Learning math is like learning a language -- it takes a lot of time, and not a lot seems to happen right away. But in the long run, it can be very rewarding.