I agree with Feller as the classical book on probability; it is beautifully written and full of insight and deserves to be read over and over again.
For theoretical statistics, justifying why you'd use one method versus another, the graduate level classic is Cassela and Berger, Statistical Inference. This teaches you to design your own tests and compare Bayesian vs. frequentist methods and interpret the paradoxes - and is full of beautiful explanations - but it's not a bag of tricks.
Applied statistics book tend to be more specialized. There's plenty of cookbooks with statements like "and then you should do this regression and compare the p-value to 5%," but that's not what you want. Any of several books with titles like Applied Regression Analysis or Multivariate Statistical Analysis will be at a more substantial level. After that there are specialist topics - experiment design, survival analysis, heavy tailed distributions, robust statistics, multi-level modeling, asymptotics. Statisticians get involved in a lot of areas, from quality control in manufacturing to clinical trial design to economic forecasting to genetics, so there's a lot of overlap with other fields in the academic literature.
Econometric Analysis by Greene is the economists' favorite stats book, and is very self-contained, covering everything from computational tricks for Monte Carlo simulations top some very modern multivariate methods.
As a computer scientist/hacker, your natural starting point is Bishop. Pattern recognition and Machine Learning or perhaps http://www.ai.mit.edu/courses/6.867-f03/lectures.html
I've heard machine learning defined as the application of statistical methods to engineering; people with statistics degrees grumble that CS/machine learning types learn to do what's computationally feasible and not properly justify their methods, doing the sort of analysis you'd see in Casella and Berger.
In terms of software, a lot of statistical computing in the open source world now gets done on R platform. For modern computationally intensive methods like MCMC (Markov Chain Monte Carlo) the program BUGS/WinBUGS is a standard.
Also, http://videolectures.net/ has a whole lot of lectures (mostly from the Machine Learning perspective), starting with intro to probability and going right up to modern research.