there's a pretty good book out there "Data Science from Scratch" it's all in python and really lays out the workings of popular data science / analysis algorithms without relying heavily on external libraries.