Hacker News Reader

Comment by ksec | Hacker News Reader

Personal Opinion only.

I dont think Fastly has ever explicitly called out a 35us against Cloudflare 5ms. ( Please Correct me if I am wrong ) They may have some marketing material about 35us being faster than the rest of industry. But that doesn't imply it is Cloudflare because there are lots of player in the industry. First name comes to mind would be AWS.

Compared to Cloudflare blog post directly naming Fastly in a benchmark is completely different.

Second being Fastly was unhappy about this test and posted on twitter for weeks even quoting and mentioning @cloudflare.

But I agree everyone should run their own benchmarks.

View on HN

The material commonly mentions "isolates" or V8 taking 5ms to cold start, which seems clearly aimed at Workers.

But the same point applies when comparing against Lambda. It's not a cold start if the app code is already resident in memory in advance. Workers and Lambda also proactively load some apps in advance, and we don't call it a "cold start" in those cases.

>> So, I would encourage everyone to run your own test and don't just go on other people's numbers.

> But I agree everyone should run their own benchmarks.

But they literally can't because the ToS forbids it.

Anti benchmark clauses usually prohibit publication. This wouldn’t exclude running two systems with your workload and documenting the results for internal use.

Except the CloudFlare ToS (posted as an image from the tweet thread above) says "perform or publish benchmark tests or analyses..."

They don't forbid it. They request you get permission first. No one seems to have bothered asking to find out if CF say no.

Benchmarking accurately is hard, so CF trying to make sure they're not going to be subject to a "report" by someone whose tests are garbage, or by a competitor who's benchmarking poorly to make their own offering look better isn't totally unreasonable. Ideally, as far as CF are concerned, they'd be so far ahead of the competition it wouldn't matter, but presumably that wasn't the case when the ToS were written. The fact the CEO posted to say the clause is being removed means we can all freely test in future.

This is how things are supposed to work. Someone does something, someone else reacts, and the first person updates their position. Expecting everyone to get things perfectly correct on the first try is nonsense.

It would be nice for the first person to actually own it and not just say "I have no idea who put it into our TOS". Besides, it's either disingenuous or something is seriously broken in Cloudflare if their CEO is more attentive to HN comments than their own SMM people, as Fastly called that clause out on Twitter weeks ago.

Reply on news.ycombinator.com