Hacker News Reader

I'll ask dumb question - if they are "open source" then why they are bothered by it? Is it scraping itself? Are their data not freely available for download?

View on HN

Their data is freely available to download. There are weekly dumps of the entire planet and several sources for partial data. There's no need for most legitimate use cases to scrape their API.

So problem is someone is stupid enough scraping without realizing they can just download 100gb at once?

And there are so many of such idiots that it's overwhelming their servers?

Something doesn't math here.

That's exactly the case. It's been an issue for a while.

Someone has to pay for bandwidth. And that someone would like the bandwidth to go to human users.

Reply on news.ycombinator.com