Hacker News Reader

Comment by jfindley | Hacker News Reader

I was initially surprised at their benchmark results, until I read more closely and noticed that they were benchmarking on a cloud VM. I wish people wouldn't do this for this form of test - it adds so many variables it's hard to take their benchmark too seriously.

View on HN

We executed the benchmark suite multiple times, and results were consistent. It was way more stable than the same execution on a PC with multiple programs. Anyway, the maintainers of glibc performed the same checks and confirmed our measurements.

Thumbs up with reaching out to upstream to mainline it.

Did you talk to musl team as well?

Not yet. Working with each open-source demanded a lot of effort, so we were hoping that integrating it into a few dominant libraries will attract the attention of the rest. We will happily assist them if they need/want our help.

The musl mailing list is very active, and communication with the maintainers is generally effortless. It would be interesting to see how this plays out. They are also in the process of rewriting their malloc implementation.

Thank you for your work!

And musl is just working in merging its new mallocng.

In my experience, cloud VMs are often a more consistent benchmarking platform than your personal machine.

A desktop environment and consumer-grade hardware add a lot of noise.

Ideally you'd always benchmark on completely idle dedicated servers, but those are rarely available.

You can just run it on your local machine and dedicate some of the cores to it with a tool like "cset shield".

If you _really_ want to avoid any contention, then build the benchmark as a Linux kernel module and call stop_machine(), which will prevent anything else including interrupt handlers from running.

Not that surprising: CPUs can perform scalar in parallel with computation, so I would expect a few scalar ops to be free in any code that is not scalar-bound.

Reply on news.ycombinator.com