Information Theory for Machine Learning [pdf]

As a learning exercise/fun project for the author, I think this is ok.

But for any serious study, I fail to see what this offers over the completely free, very readable, more carefully written, and more thorough "Information Theory, Inference, and Learning Algorithms" by David MacKay: http://www.inference.phy.cam.ac.uk/itila/book.html.

As an example of the problems with the pdf in its current stage, Theorem 2 (asymptotic source coding) includes the term "negligible loss" without even defining what "loss" means in source coding. Lossless and lossy coding are very different things, all the preceding stuff is really discussing the lossless coding problem. Pedagogically, these need decoupling.

Bookmark comment; ignore.