If you ever feel like trying out the algorithms contained in the book without going to the trouble of reimplementing everything from scratch feel free to come over to https://github.com/Svalorzen/AI-Toolbox. This is a library I have maintained during the past 5 years and implements quite a lot of RL algorithms, and can be used with both C++ and Python. It's very focused on being understandable and having a clear documentation, so I'd love to help you out starting up :)