Hacker News Reader

Comment by daft_pink | Hacker News Reader

I suspect they are trying to block bots like everyone running a website with useful data these days.

The bots can still do a git clone and index everything, this just inconveniences normal users working on "some other" PC (or browser or incognito tab), where they are not logged in, and/or don't want to log in (coworkers PC, 2fa, whatever).

> The bots can still do a git clone and index everything

Of course they can, but then they're gonna be chewing up their own disk space and bandwidth for anything after the initial hit. I think the real problem is that the bots hit the GitHub servers over, and over, and over again.

Maybe if the bot operators have the resources, but it's far from trivial to keep an up-to-date mirror of every project on Github, especially if Github is actively putting up barriers to prevent it. Once login is required, it becomes much harder to bypass rate limits because the company can rate-limit signups from unknown domains, enforce 2FA, etc.

They can do all this (rate limits, etc.) for unregister accounts already, and most users would never notice that (since a human only does a few searches per time unit), but they decided to require a login anyway.

A lot of bot traffic is just mindless "follow any link" traffic, not specialized bots to do X. It really is hugely pointless and wasteful to have tons of these bots request tons of comparatively expensive search links.

I run a small website that tracks releases of a software, it probes releases every few hours, with each run consuming a several hundreds of API requests. GitHub API token limits are pretty generous. There is no paid offerings to increase the limits, so I suppose if you ask GitHub nicely, they will increase the limits given a reasonable justification.

>it probes releases every few hours

I can't think of any software I've ever used that I was this concerned about release schedule. Even if I was a user of your site, I might check it daily, but even that is doubtful.

Reply on news.ycombinator.com