Hacker News Reader

Comment by 542458 | Hacker News Reader

I’m a bit confused as to why 700 automated clone requests per day would be an unreasonable amount of traffic for a project like GMP. That doesn’t sound like much traffic to me, especially since they reportedly have a 24 core CPU with 256GB of RAM. Is cloning a mercurial repo highly server intensive?

View on HN

It's 700 automated clone requests at the same moment due to inheriting a cronjob.

Sure, but even if each clone tied up an entire core of their Epyc 7402P for 5 seconds (which seems pessimistic to me) the entire traffic spike would be over in 2.5 minutes.

I dunno. To me it seems like if a web server can't deal with 700 simulatenous requests it should probably call it quits. They're of course within their rights to just block the connections but it certainly doesn't look like a denial of service attack to me. Submitting a link to HN probably causes even bigger traffic spikes.

700 simultaneous static requests certainly. 7,000+ in fact. Even 700 average requests for dynamic content. But each of those request was to clone a repo which is not an average dynamic request: each require a fair amount of CPU time (and possibly IO thought hopefully the server(s) have enough RAM for cache for that but to be the case).

The original email said that a single instance would make 100 requests in parallel, so 700 instances would make 70k clone requests at the same time.

I'm confused as well. Seems like a negligible amount of traffic at first glance. The administrator mentioned compression, could it really have such a big impact?

Especially if you’re running powerful server-class hardware and great connectivity to the Internet.

^ So much this. From the assumption of malicious activity (which is a far leap given such a predictable pattern of calls), to bragging about “server-class hardware” and “great connectivity”, this doesn’t seem like a super solid setup.

In before “it’s just one guy”: there are a thousand ways to solve this that are not expensive or complicated.

Should the offending party have been a better consumer? Yes.

Is it fair to entirely blame the user versus implementing additional caches and safeguards to make it an annoyance and not the end of the world emergency GMP is making it out to be? Definitely not.

Hyperbole never ceases to amaze me. On one hand, I know how wasteful "CI" systems are. We offer a docker image from a custom docker registry. Since we started offering it 3 years ago, we've had 1 billion pulls. 990 million pulls were from CI systems. But to claim that the operator of those CI systems is "attacking" us would be pretty bizarre. Though bizarre seems to govern online discourse when it comes to most things

Those 700 clones would all hit at exactly the same time. That's quite a load for a single server, especially since Mercurial can't be cached that easily.

So if they're all hitting at exactly the same time, why not set up a firewall rule to block traffic from GitHub at exactly that time of day?

Setting up a rate limiter is the correct way forward, and using IP block as the nuclear option.

Mercurial clones can be cached quite easily with 'clonebundles', a Mercurial feature that allows redirecting a clone to instead download a single 'bundle' (which could come from a different server or set of servers).

See https://wiki.mercurial-scm.org/ClonebundlesExtension

I don't know if it's an unreasonable amount of traffic but it's certainly worth mentioning, 1 repo vs. 700 is a big difference.

Reply on news.ycombinator.com