Hacker News Reader

Comment by refulgentis | Hacker News Reader

I suggest deleting this. I'm a completely neutral third-party with little interest in the subject at hand, and their post doesn't come off like mud-slinging, at all.

This then makes your response look accusatory and helps further a broader narrative post about Cloudflare's behavior here, which it seems your goal is to dispel, given your statement on whether engineering was involved.*

A suggestion for something actionable that'll make things better in this difficult moment: email product counsel about starting the process to override your EULA and enable Compute@Edge to do benchmarking. It'll take forever and it's the only way to have an objective, evolving conversation about benchmarks, which your leads will (at least, should) want after this happened.

* intentionally vague here, in order to give you space to delete and leave no record

View on HN

> their post doesn't come off like mud-slinging, at all.

The post's current title is Lies, damned lies... and the article's synopsis is,

> This nonsensical conclusion provides a great example of how statistics can be used to mislead. Read on for an analysis of Cloudflare’s testing methodology, and the results of a more scientific and useful comparison.

That is definitely mud-slinging.

No it isn't. It's their response to CF's crap test which CF then posted on their blog <---THAT is mud slinging.

And Kenton himself agrees: "On a serious note, I've been saying for decades that benchmarks are almost always meaningless..."

Benchmarks can be used to mislead which CF was clearly doing and which Fastly pointed out with their "Lies, damned lies..."

I think you misinterpreted me a bit. I don't believe CF's benchmark was designed to be misleading or biased. I think the benchmark had a perfectly reasonable setup and was actually representative of one relevant use case -- the case of an app that just returns a response without a lot of processing. I think Fastly's criticisms of the benchmark are surprisingly weak compared to the strong language they are couched in, and the strong language itself is what I refer to as "mud-slinging".

My point about benchmarks in general was that unless your application is extremely similar to the one tested -- which is unlikely -- then these numbers don't really tell you much about how your own app will perform. This is a criticism I have of every benchmark that attempts to compare performance between very different technologies. I don't think it makes such benchmarks unethical, just not very useful.

Sorry and thanks for the clarification.

I don't understand why you're arguing for the parent comment to be deleted, it's useful to hear from someone, (especially an engineer and the tech lead) who is working on the product. I hope that the comment is not deleted.

I found parent interesting and relevant.

But mostly importantly, kentonv's last paragraph.

At the end of the day, benchmarking for your use case is the only true north. Everything else, on all sides, is marketing fluff.

You're implying that he did something wrong.

But the title of the post validates his opinion and his claim about fastly and cold starts can literally be found anywhere in their blog and the compute@edge landing page.

I think it's a valid and valuable comment.

Fwiw, I don’t have an issue with it. It’s good information.

I think the comment you were replying to was just fine. They have opinions about another service, I think that’s ok.

> their post doesn't come off like mud-slinging, at all.

Sure felt like mud slinging to me. Just the whole list of complaints they had at the start of the post was hard to get through.

They could just tell us what they did differently and show the result.

Reply on news.ycombinator.com