Interesting article!

I would just like to point out that concurrent downloads can be handled much more efficiently in Python > 3.4 thanks to the asyncio library. For an example, look at Guido van Rossum's crawler [0].

[0]: https://github.com/aosabook/500lines/tree/master/crawler