Hacker News Reader

Comment by pengaru | Hacker News Reader

> There have been some unfair benchmarks posted in the past surrounding io_uring that were called out by people on the liburing repositories.

Could you provide any links to these discussions?

View on HN

Sure. https://github.com/axboe/liburing/issues/189

Original claims were in the 90% and above performance increase over epoll. Then issues were found, and the figure was adjusted to 60% over epoll. Then more issues were found, and now real-world performance tests are showing minimal speedups if any.

Unfortunately the sibling commentors don't see "computer science" as a science but instead as a "feel good hobby", it seems. My point wasn't to hurt feelings, it was to provide a word of caution with these sorts of groundbreaking claims with respect specifically to the io_uring efforts, as they have been disingenuous quite a few times historically.

I don't doubt Jens does fantastic work. I don't doubt that he's seen these speedups in very specific cases. But people are celebrating this as a win where they'd be skeptical of e.g. "breakthrough" treatments of cancer (footnote: in mice). It's the same thing.

But if you read the thread, you realize he is testing on a 2011 laptop, and the older architecture may be causing the issues. He reported a 3x speedup running the benchmark on a Raspberry Pi 4 in the same thread[0]

Someone claimed that you don't get to see the benefit of io_uring in a hypervisor[1], but they did not provide benchmark results.

[0]: https://github.com/axboe/liburing/issues/189#issuecomment-94...

[1]: https://github.com/axboe/liburing/issues/189#issuecomment-73...

If that's the best example you've got I'm writing your entire comment off as a steaming pile of FUD.

It's FUD to have personally run benchmarks across a number of platforms and machines and have seen very trivial, if any, speedup?

The referenced benchmarks are of poor implementation quality, don't even produce consistent results, and the specific linked issue is rife with discussion better suited to twitter than anything scientific or professional.

Some of the referenced benchmarks are by Jens himself. That's the problem. He has promoted tweets and press releases touting io_uring as being insanely fast when the benchmarks used to come to those conclusions were flawed to begin with.

That's literally the entire point I'm making.

Why did you omit the response you received from a netty developer that showed a large performance improvement[1]? Focusing entirely on the few cases where people report regressions and ignoring all of the reports of improvements isn't a good way to make an objective assessment.

[1] https://github.com/netty/netty/issues/10622#issuecomment-701...

I didn't omit it, and the Java example isn't the benchmark that was used to derive the initial >90% speedup that was tweeted at Jens originally.

Reply on news.ycombinator.com