Its a shame to think people seeing this as a "self promotion", when its not. What i am promoting is the data that this crawler obtained, and all the insights people can get for free, just by using it. As opposed to an API, that will limit either the access for the data or the way you can access it, acessing the raw data on a database directly has no limits.

Anyway, thanks for the reference, i will make sure to add it to the project's wiki as a related link.

Okay... So, having been down this road and beyond already, what do you think of this guy's approach which is nice and visible for inspection?

Might help not only him but some of us learn further, us tinkerers, those who might have a faint curiosity in c# etc if you shared. Then you'd be able to plug away shame-free.