Should small Rust structs be passed by-copy or by-borrow? (2019)

How are you using the word simpler? Because to me that implies a combination of more obvious and number of lines of code. Something that a benchmark shouldn't be involved in.

For example asking someone to delete 10 lines of code and instead use go's ` net.SplitHostPort` would be an example of "simpler".

Even if it's not smart enough to do that today, it could implement this optimization in the future. This could work even without inlining, since the Rust calling convention is unstable, and an optimization based on type size could be incorporated into it.

> I always ask them to either prove it or write the simple thing. If the code in question isn't hot enough to bother benchmarking it, the performance benefits probably aren't worth it _even if they exist_.

One of my philosophies is that death by a thousand cuts is fine, but death by ten thousand cuts isn’t. A team of 10 engineers can probably fix most of a thousand cuts in two or three months. But if you have ten thousand cuts you’re probably doomed. And those don’t show up cleanly in a flame graph.

Now for some context my background is video games. Which means the team knows they need to hit an aggressive performance bar. This isn’t true for many projects. shared_ptr is a canonical example of death by ten thousand cuts.

That said, I strongly agree with the principle of “just do the simple thing”. However I think it’s important to have “sane defaults”. A project can easily have a thousand or ten thousand papercuts that kill performance. But you can’t microbench every tiny decision. And microbenches are only a vague approximation of what actually matters.

I’m also wary of “the compiler will make it fast”. Because that’s true… until it’s not! Although these days you don’t have any choice but to lean heavily on the compiler and “trust but verify”.

No one wants a super complex solution if it’s not needed. However I am very amenable to “do a slightly more complex thing if you know it’s correct and we can never think about this ever again”. It’s much easier to do the fast thing upfront than for someone else to try and speed it up in two years when we’re doing a papercut pass.

Please, don’t optimize unless you have reasons to do so and numbers backing that up.

I really dislike these takes. I see engineers optimize these cases and then go ahead and make two separate SQL queries that could be one, ruining any false optimization gains they got by lord know how many times.

Yeah, you can loop over that 100 element list twice doing basic computation if you want, it's not going to make a difference for many engineering workloads, but could make a big difference in readability.

Even if they do, they also need to make a case that in this specific case, performance matters enough to pessimize code simplicity and maintainability.

Also, if performance is that critical, it's imperative to benchmark again after each compiler release to guard against codegen regressions. And benchmark after changing this piece of code. Otherwise, we can say that performance doesn't really matter.

I've seen this happen a lot with JavaScript, particularly in the last 5-10 years as JS engines have developed increasingly sophisticated approaches to performance. Today's optimization can be tomorrow's de-optimization. Even given an unchanging landscape of compiler/interpreter, tightly-optimized code can become de-optimized when updated and extended, as compared to maintainable code that may not suffer much performance degradation upon extension.

This is cumulative thou, 1 may not make much difference. 100,000 of those will. There is that story of chrome slows down that was does discovered to be in strings each one of which is too little to make a difference

Of course, if your struct is truly enormous, you may want to break this rule to avoid large copies. But in that case you probably want to Box<T> the struct anyway.

Of course, if your struct contains something that can't be copied--like a Vec<T>--you'll have to decide whether to clone the whole struct (and thus the vector in it), pass the struct by-borrow, or find some other solution.

> Then you would actually be testing by-borrow be by-copy instead of how good rust is at optimizing.

I don’t think the question is actually: “what is faster in practice, a by-copy method call or a by-value method call”, I think the question is: “as an implementer, which semantics should I choose when I’m writing my function”.

For the second question: “Rust is usually pretty good at aggressively inlining, so… if you’re willing to trust Rust’s compiler, you’re often okay going with by-copy implementations, but you should keep an eye on it”. Whereas, as you note, for the first question it’s not an answer.

But, I do think if someone was going to put more work into it I’d be very curious what the answer to the first question is. If I’m choosing to implement with by-copy semantics and trusting the Rust compiler to hopefully inline things for me, I’d like to know the implications in the cases when it doesn’t.

Also Fortran has "in", "inout" and "out".

in - regular function arguments

inout - mut function arguments

out - function return

Is there any additional information that a compiler can infer from Ada’s parameter syntax?

And I would bet 9 times out of 10 it won't be the bottleneck or even make a measurable difference.

That's my approach too as a Rust newbie. Borrow by default and take ownership only when needed, for the best ergononmics.

If there's no inlining at play, I'd expect vast differences to be possible. For example, imagine a chain of 3 functions - f calls g, g calls h, where one of the arguments is a 1kB struct and the options are passing by copy or by borrowing. In this case, each stack frame will be 1kB in size in the copy case and there will be a large performance overhead as opposed to the by-reference case. One would expect simply calling the function to be similar in overhead to an uncached memory load.

Within a single crate the inlining is possible, with multiple crates it's only possible with LTO enabled (and I'm not sure how _probable_ it is that the inlining would occur).

In either case, the difference between a 32 byte and 8 byte argument in terms of overhead is likely meaningless - the sort of thing to be optimized if profiling says it's a problem as opposed to ahead of time.

So, what the author actually measured was the difference between llvm and msvc throughout the article. Particularly when they talked about rust being better at autovectorization than C++.

Do you mean AsRef, or do you mean magic which automatically borrows parameters and is specifically what rust does not do any more than e.g. C does?

Though you can probably get both if the by-ref version is faster (or more convenient internally): wrap the by-ref function with a by-value wrapper which is #[inline]-ed, this way the interface is by value but the actual parameter passing is byref (as the value-consuming wrapper will be inlined and essentially removed).

FWIW, the `Borrow`, `AsRef`, and `Deref` traits all exist to support different variants of this.

And, for simple operations like this, you really should just look at the assembly output. If you are only generating 20ish instructions, then look at those 20 instructions rather than trying to heuristically guess what is happening.

Depending on calling convention, the structure may be spread out into registers.

FWIW the AMD64 SysV v1.0 psABI allows structures of up to 8 members to be passed via registers. Though older revisions limit that to 2 (and it's unclear whether MS's divergent ABI allows aggregates to be splat at all.

As sad as it's unsurprising, it does not look like LLVM (linux?) has followed up, on godbolt a 2-struct passes everything via registers but a 3-struct passes everything via the stack. Maybe there's a magic flag to use the 1.0 ABI, but a quick googling didn't reveal one. ICC doesn't seem to have followed up either.

There’s literally a section called Source Code…

But yeah I have no idea why a benchmark framework wasn't used for Rust.

But nevertheless: I agree it would have been interesting to test with GCC as well.

What did you have in mind?

I'll take Rust shouting at me for missing "mundane details" any day of the week.

Rust uses syntax that feels familiar but means completely different things than in pretty much any other language.

For example '=' doesn't mean assign handle or copy. It by default means move.

'let' doesn't mean create a name for something. It means create physical space for something (of known size) that can be moved into or moved out of.

You don't deal with objects and values of primitive types. Instead everything in Rust is a value. When you move, you move the value. If you compare, you compare by value. If you pass something from variable into function, you move the value into the function.

And when the space where you keep the value goes out of scope value dies with it if it wasn't moved out to somewhere else.

Scope for variables (which are just named spaces for values) ends with the end of the block, but some values, created by functions and returned from them, if they are not moved into any named space, can die sooner, even in the middle of the line where they were acquired from function call.

Everything else stems from that fixed size moved value semantics. If you don't want to move the value into the function when you call it you need to pass something else instead, so you create and pass in the borrow. But you have to ensure that the value doesn't die or get moved anywhere (even inside the container you borrowed from) before borrows to it all die.

Because of this you are better off with borrows that are short lived and local. Often it's better to keep the index of an element of a Vec instead of the borrow of this element. If you must create types that contain borrows you must know that they become borrows themselves and you need to treat them exactly the same trying to limit their scope and life time.

It's hard when you come from any other language because borrows are superficialy similar to pointers or references to objects. So you try to use them as such. And crash into the compiler because they are not that. What's worse their syntax is very minimalistic which triggers intuition that they must be fast and optimal solution for many problems which they sure can be once you fully internalize their limitations but not a moment sooner.

Another thing is that values in Rust must have the fixed size. So even as simple thing as a string requires a bit of hackery. Basically in Rust the default strategy to have something of variable size is to allocate it on the heap and treat pointer to it (possibly with some other fixed sized data like length) as the fixed sized value you can move around clone and borrow.

So if you want to have semantics you know from other languages you can't just use basic Rust syntax.

You need constructs such as Box and Rc, Cell, RefCell. Make your things clonable and sometimes even copyable and avoid creating borrows whenever possible initially. When you do it Rust becomes as flexible as any other language and you can use it pretty much just as comfortably. Then the value semantics shines as you can very easily compare your data by value, order it, create operators for it, create has for it so you can keep it in HashMaps and HashSets. Then it's delightful.

My advice is when you create a long lived type just wrap it in Rc and treat this Rc as your 'object'. And avoid borrows in your types unless you have a very good performance (measured) reason to have them or you are creating something obviously dependant and usually short lived like an iterator.