Introduction To Machine Learning (Smola & Vishwanathan)

I've found Andrew Ng's notes to be very comprehensive and very relevant to the topics discussed in his online lectures : http://academicearth.org/courses/machine-learning (the notes are linked to on the individual videos)

Also note that this book is not for the faint of heart.

For introductory material you may want to take a look at some of these courses:

Shalizi, 35-350: http://www.stat.cmu.edu/~cshalizi/350/

Shalizi, 36-402: http://www.stat.cmu.edu/~cshalizi/402/

Ng, CS229: http://www.stanford.edu/class/cs229/materials.html

Ng, CS294: http://www.stanford.edu/class/cs294a/

Roth, CS446: http://l2r.cs.uiuc.edu/~danr/Teaching/CS446-10/lectures.html

Jebara, COMS4771: http://www.cs.columbia.edu/~jebara/4771/handouts.html

Jebara, COMS6772: http://www.cs.columbia.edu/~jebara/6772/solutions.html

Jordan, CS294: http://www.cs.berkeley.edu/~jordan/courses/294-fall09/

Kumar, EECS6898: http://www.sanjivk.com/EECS6898/lectures.html

Collins, COMS6998: http://www.cs.columbia.edu/~mcollins/courses/e6998-3/index.h...

Jaakkola & Collins, 6.867: http://courses.csail.mit.edu/6.867/lectures.html

Hamilton, CS831: http://www2.cs.uregina.ca/~hamilton/courses/831/

Girolami, MLM: http://www.dcs.gla.ac.uk/~girolami/Machine_Learning_Module_2...

Bengio, IC-49: http://bengio.abracadoudou.com/lectures/

Lin, CMSC838: http://www.umiacs.umd.edu/~jimmylin/cloud-2010-Spring/syllab...