For software performance, can you always trust inlining? | Hacker News Reader

For software performance, can you always trust inlining?

lemire.me

34 pointschmaynard17 comments

Article doesn't really address the question of whether you can ALWAYS trust inlining. I can tell you from experience the answer is no. We have some libraries at work that implement generic evaluation of functions over pages of objects, as in the context of a database. The authors did this in the "proper" C++ way, which involved a lot of generics and templates. They also sprinkled inline directives all over the place, apparently without ever actually benchmarking each individual case.

What it ended up with is a bunch of absolutely enormous modules, each generating tens of megabytes of mostly identical code, but with the leaf functions in each case handling floats, ints, strings, etc.

Removing some of these inline directives probably would not impact the code at all, since most of the inlined code was cold. But it would likely benefit build performance quite a bit. It might even improve runtime performance by putting less pressure on the cache.

This is probably not a surprise, but inlining is best for hot leaf code. Cold code should probably not be inlined unless it is used exactly once.

I doubt that would help, since generic code is implicitly inline.

Generic code has to be declared before use, in C++, if you're not predeclaring the types the template supports. But that is orthogonal to whether it is actually inlined in the generated code.

AFAIK nothing requires generic code to be compiled inline in C++.

Heck with explicit template instantiation the generic code may not even be in the same translation unit as the calling code. At that point it'd require link time optimization to inline it.

Nothing requires inline-specified code to be inlined either.

Not true! This is true in monomorphized languages like Rust, but C++ has vtables which allows for generic code at an assembly level.

Maybe I’m misunderstanding you.

run-time polymorphism is generally not called "generics"

What jcelerier said, but also as for:

> This is true in monomorphized languages like Rust

It's not necessarily true for monomorphized code. Inlining is enabled by monomorphization, but is not required.

You can see it with templates in C++: function template instantiations aren't always inlined. They are instantiated in every TU separately, if they are defined in headers, but then they may be called in this TU just like any other function without inlining. This requires the linker to deduplicate the instantiations later. In another scenario you may declare a function template in a header without the implementation, putting the implementation in a CPP file along with explicit instantiation. In that case the compiler is not able to inline this function in another TU (without LTO).

As for Rust, Rust is not universally monomorphized. The user of generic code in Rust gets to decide whether they want to use monomorphization or vtables, regardless of the ideas of the author of said generic code.

Thanks for this comment, I have a lot to learn.

you are talking about monomorphization.

this thread is about function call inlining.

I've heard a lot that when you want to focus on performance, you should benchmark instead of relying on intuition, and I think this is another example of that.

i.e. “Stop thinking and start looking”

And of course there is still the difference between "has an inline directive" and "gets inlined" - it's merely a strong suggestion in most compilers.

Sure. In this case I was reading the disassembled code, so I could verify that large swathes of code were in fact getting inlined. I believe the code in question was using the clang-specific always_inline attribute.

https://clang.llvm.org/docs/AttributeReference.html#always-i...

I'm not so sure it is even a strong suggestion. Actually inline nowadays mostly has a semantic meaning - the symbol can be defined in multiple compilation units without causing multiple definition errors from the linker. Usually you can't remove it without moving the definition from a header file to an implementation file.

Edit: it is correct though, that the inline keyword often is a neccessity to allow the compiler to inline in the first place. After all, the compiler can only inline a function at the callsite, when both are in the same translation unit. Either the function is only used in one translation unit, or it has to be defined as inline (or worse as hidden to the linker by static or anonymous namespace). Disregarding LTO, of course.

> It might even improve runtime performance by putting less pressure on the cache.

Not so much a possibility as a certainty on modern processors. Easiest way to tell: run it through perf stat and read the output.

> I do not expect you to read this gibberish, but notice how the result is now tighter. It is also going to be slightly faster.

Well… I’m definitely no expert, but I don’t think that’s great reasoning. His code has jumps in it that appear to be loops, just because the code is tighter doesn’t mean I can infer anything about where it will jump and how many times JUST based on how many lines or how “tight” it is.

He’s right I’m sure that his second example is faster, I just don’t like that for an assumption.

Reply on news.ycombinator.com