I have not seen mentioned so far in this thread the following book, which I can't recommend more highly:

Understanding Machine Learning: From Theory To Algorithms – Shai Shalev-Shwartz