Hacker News Reader

Comment by astrange | Hacker News Reader

Tuning is highly overrated in software performance, eg compiler optimizations don't help that much and searching their params helps even less. I think it comes from people wishing they could change their software without actually needing to learn how to change it.

View on HN

It is overrated when you are tuning the wrong thing, but if it is the bottleneck in a significant process for you use case then it is often very significant especially at scale. For someone running a compute cluster for their own needs or as a cloud service a couple of percent improvement like this represents a massive gain in throughput or a saving in the need for extra kit (or power costs - there may even be a small environmental benefit).

Of course on the scale of you or I, such optimisations may be little more than a curiosity most of the time. A process that normally takes 100 minutes (a video transcode, perhaps) now taking 95 followed by the machine being idle while it waits for us to look back and see the result, is not really benefiting from the benefit.

> without actually needing to learn how to change it

This is certainly a problem sometimes, but not an argument for tuning bring overrated in general. It usually comes down to tuning the wrong thing, like someone playing with mysql engine & kernel IO parameters to eek out a fraction of a of % bonus when they could improve index structures, or fix queries with no sargable predicates, or both in unison, to get benefits measured in orders of magnitude.

It can be overrated in the case of average gamers tweaking their hardware to get a few extra FPS on top of the many tens they already get, but again this not being worth the time for some, even many, doesn't mean it can make a huge difference to a pro-gamer or someone using old kit where "a few extra" is a relatively large gain.

That's quite a bad take. You can bank on PGO+LTO of a C++ server being good for 20% throughput easily, and not compiling for k8-generic when you have a modern machine also makes a big difference. I would agree that just flipping compiler flags might not get you very far, but there are key inputs that you shouldn't leave on the table.

By "not that much" I mean orders of magnitude. 20% is reasonable, especially for C++ that needs a lot of inlining.

But a more modern language would benefit from defining some optimizations like inlining as mandatory, the same way tail calls are (and of course GCC has this.) Then there isn't a chance of deploying a build with extra-slow behavior.

A performance improvement of 20% would be a massive win for something as large and widely used and as the linux kernel.

That would average out into a few percentage points of performance improvement in every application across the globe which runs on linux / android. Merging a patch like that in linux would be like quietly reaching into every device across the planet and giving them a small, free CPU improvement and drop in power usage. Not game changing for any individual user, but huge in aggregate.

And if you run a lot of servers it means you need fewer servers. These kind of changes can literally save millions.

Which is why the organizations which stand to save millions by building their kernels with PGO already do it.

Some do. I bet lots of companies miss out on massive cost savings like this simply because nobody at the company has the skills and access to improve things.

I heard a story - years ago Google hired an engineer who happened to be an expert from a previous life in video codecs. He wasn’t working on YouTube at Google, but out of interest he pulled up the YouTube source code to see what it did. They were just using the defaults for some of the encoding parameters. He tweaked a few of the encoding parameters and in doing so saved Google millions of dollars per year in compute/storage/network traffic. A few hours of his work was probably more beneficial for Google than years spent in his primary role.

And if tuning opportunities like this abound at Google, you know they’re everywhere. It’d be much better if Linux distributions simply shipped kernels which are already compiled with PGO, with a reasonable profile based on normalish use.

That could've happened because of pointless detuning - x264 comes with good defaults for everything, but then ffmpeg on top of it used to set everything to "off" no matter what it was, and they were probably using the ffmpeg settings.

The correct answer would've been for ffmpeg to not ship that way.

Anyway, I am an expert on video codecs just like that guy is, and I said what I said ;)

Probably, but there are many smaller outfits that probably don't because they have a server fleet of "only" a few dozen servers or so. The cost savings for those won't be in the millions, but having to purchase ~5% fewer servers is still a pretty good win.

Most smaller orgs are wasting resources on poor utilization. Tighter code just gets those orgs even worse utilization.

Counterexamples:

Running your DB with the default parameters.

Not running Linux kernel parameters depending on your workload. (+10Gbit NICs, lots of connections...)

There are reasons, why those knobs are there, as there are always trade-off, and having them automatically adjust is very hard.

But maybe that was what you're referring to.

That highly depends on your problem. I have some scientific code that gets more than a factor 10 improvement on runtime just by turning on some compiler flags (mainly -O3 and -march, but some others as well).

I also have a similar scientific code base which gets 10-15x wall-time improvement just by enabling compiler optimizations.

Not a scientific codebase, but ffmpeg seems to benefit massively from optimization

I use it to encode my DVD and Blu Ray TV/movies to HEVC. Doing so reduces the filesize on DVD's by roughly 80% and Blu Rays by 60%

The downside is the encoding is an incredibly CPU intensive process. Hardware encoders like Nvidia's Nvenc or Intel's QuicSync look absolutely terrible and are a non-starter for archival storage

On a stock Fedora XFCE install, I would get roughly 0.5 FPS for a 1080p Blu Ray file (29.97 FPS at 1920x1080)

A Gentoo Linux installation with a global -O3 -march=native as well as LTO, PGO and Graphite enabled globally boosts it from 0.5 to roughly 1.3 FPS. Still slower than realtime, but an absolutely massive improvement

That's an amazing compression ratio, how is the quality?

ffmpeg recommends a crf of 23 to preserve 1:1 for H.264 -> h.265

I do a probably overkill CRF of 20 just to be safe. But everything looks absolutely perfect, even blown up on my 65 inch (1080p) TV

Can you share your complete parameters, so I can do the same thing for my DVD archives & other stuff?

No hard feelings if you don't want to though :)

Sure! This is the script I wrote for it. I use MP4 containers so I can import them into MAGIX (formally Sony) Vegas but it will iterate over an entire folder of MakeMKV files and convert them https://dpaste.com/DBPA9C59M

Thank you. It's greatly appreciated. :)

I suspect -march=native is the only thing doing any work here. The other optimizations are as likely to find compiler bugs as they are to improve things, once you get off well-tested paths.

I think it's the opposite. A good annotated code which can use SIMD instructions accelerates tremendously with -O3. -march=native -mtune=native generally improves things if you can saturate the cores with instructions, which can be measured by running perf record and looking to IPC & instruction retirement numbers.

When I was using Eigen on my code, the biggest performance boost came from -O3. -march and -mtune did minimal improvements on the systems which I've ran benchmarks on.

ffmpeg and associated projects do not rely on autovectorization, which pretty much only works on naive scientific code. They write their SIMD in assembly and don't need to care about compiler settings.

If you're compiling hand tuned assembly, -march and -mtune probably will have more effect when compared to compiling and optimizing C/C++ code.

OTOH, I'd like to underline that heavily optimized scientific code and libraries are neither naive (in terms of algorithmic complexity/implementation) nor straightforward :D

E.g.: This is how Eigen configures its internal vectorization parameters: https://gitlab.com/libeigen/eigen/-/blob/master/Eigen/src/Co...

And to illustrate the point more, I have found that -Os often produces faster code than -O3. Not by factor 10, but still clearly measurable.

Getting better performance with -Os over -O is some kind of edge case and people shouldn't generally expect that. -Os has disastrous consequences for C++ algorithms because it refuses to inline functors, so while you may rightly believe that a C++ std::sort is slightly faster than a C qsort, due to superior opportunities to optimize the C++ code, with -Os you'll find that std::sort is an order of magnitude slower. Definitely pays to check the result with a full-scale benchmark.

Interesting claim. I had to try this, and with the Xcode version that I have installed, std::sort with a simple lambda as comparison function gives about the same result with -O3 and -Os. I would not call this a disastrous, but of course opinions differ. Interestingly, qsort is significantly faster with -O3 than with -Os but of course nowhere near std::sort.

Results will vary for small vs. large programs. I've seen catastrophic space optimizations that out-of-lined very small methods like std::vector::at, because the call was one or two bytes smaller than the inline. Lambdas are inlined even with Os because they don't have names or multiple callers and can't be made smaller by out-of-lining. A functor class, or any function with multiple call sites could trigger the problems with Os.

Ok, I'm not continuing with this. Just note that I didn't write that -Os would be always or even usually faster. I could try to come up with an example where -O3 produces a huge loop preamble for loop that's iterated once or just generates enough cache misses to be overall slower, but I don't care enough.

I don't know if you would consider it tuning, but a low latency kernel is a massive UX improvement on consumer devices, where you don't want the UI to stutter and lock up under heavy load, and don't mind paying (could be wrong, top of my head) like a 5% throughout penalty.

We just improved the performance of one copying tool by a factor of 2.9, by tuning the size of a memory buffer. (BTW a shout out to both Flamegraphs and hyperfine - both excellent tools for profiling and benchmarking respectively)

Tuning kernel/process priorities has made a big difference in my audio code. Tweaking the GPU and USB IRQ handler priorities let me do realtime audio together with realtime OpenGL data visualization and screen recording, where before the audio would skip or the visualization would hang.

Reply on news.ycombinator.com