I agree. I'd start with something like Joel Grus's Data Science From Scratch to get a handle on the basics in Python (or whatever the R equivalent is, I'm not familiar with R books).

I do however find myself more and more wishing I knew data science-specific Unix commands, and I think I know what book to get to solve that problem... :)