Why Rust easily optimizes out smart pointers where C++ cannot? | Hacker News Reader

Why Rust easily optimizes out smart pointers where C++ cannot?

28 pointscodedokode60 comments

I think that this example can be both entertaining and educating. Smart pointers are often used in high-performance code (like browser engines) so it is interesting to learn how their usage impacts the generated code.

Here is a simple function that adds two numbers using smart pointers. Rust compiles it into two instructions [1], while C++ creates about 100 instructions [2] including unnecessary memory locks.

I wonder, is there a way to improve C++ code so that the smart pointers get optimized out? Did I make some mistake that prevents optimization?

[1] https://godbolt.org/z/YqYK3sPdd

[2] https://godbolt.org/z/bMrhTsjxz

View on HN

That's because you are explicitly telling the compiler to create a copy of your shared pointer, that involves incrementing/decrementing the counter:

  static int add_numbers(std::shared_ptr<Data> p) {
      return p->a + p->b;
  }

Try this:

  static int add_numbers(const std::shared_ptr<Data> &p) {
      return p->a + p->b;
  }

You'll get 4 instructions (don't forget to put __attribute__((noinline)) on add_numbers()):

  add_numbers(std::shared_ptr<Data>&):
          mov     rdx, QWORD PTR [rdi]
          mov     eax, DWORD PTR [rdx+4]
          add     eax, DWORD PTR [rdx]
          ret

The only difference is that Rust physically placed the structure in 2 registers (hence just 1 instruction), while C++ physically placed it into memory (hence the need to 2 extra instructions fetching the values).

> You'll get 4 instructions (don't forget to put __attribute__((noinline)) on add_numbers()):

That's 4 instructions only for add_numbers(), but add_two_numbers_smart_way() still is unoptimized.

Well, you can somewhat simplify it:

  int add_two_numbers_smart_way(int a, int b) {
      auto x = std::make_shared<Data>(a, b);
      return add_numbers(x);
  }

But you still get what you ask the compiler to deliver. If you don't specifically ask it to count references, it won't:

  static int add_numbers(Data *p) {
      return p->a + p->b;
  }

  int add_two_numbers_smart_way(int a, int b) {
      Data x(a, b);
      return add_numbers(&x);
  }

Here's the result:

  add_two_numbers_smart_way(int, int):
          lea     eax, [rdi + rsi]
          ret

To answer the original question, it won't optimize the smart pointers away because the smart pointers are not implemented by the compiler. They are a generic construct telling the compiler to allocate memory, increase reference counter, decrease reference counter and free memory. How does it now this is not a unit test for memory allocation API or atomic operations?

A better question would be why C++ went for implementing certain things on the library level instead of the compiler level, and the answer here is probably "historical reasons" and someone's judgement call on flexibility vs performance under specific circumstances.

It’s not “C++’s” decision, it’s the compiler writer’s. Compilers are full of special handling of standard functions (memcpy is a common case) — those cases are called “compiler intrinsics”. They can also choose to call a function when you don’t expect it (e.g. when your hardware doesn’t have floating point support).

Does rust try to follow the platform C ABI? C++ implementations typically do when it is semantically meaningful.

Rust doesn’t follow the platform C ABI unless you ask for it specifically. If you do, it will, and if you don’t, the ABI is unspecified for a reason.

I wish c++ could get away with this. But the two languages (c++ and rust) explore slightly different parts of the design space.

STL lets you supply a custom allocator for all STL types. Afaik, Rust doesn't and this is probably the price you have to pay for that feature.

Box is parameterized by an allocator: https://doc.rust-lang.org/stable/std/boxed/struct.Box.html

Rust doesn’t let you utilize this fact on stable yet, which is what you’re thinking of, but the implementation in this case already exists.

If by "STL" you mean the standard library, it's not true that _all_ containers accept allocators. At least valarray and filesystem::path do not accept allocators.

Also, C++ allows you to be explicit about things. Like std::unique_ptr<> that does not have to count references and guarantees at compile time that:

* There will be not more than 1 reference to the object at any time.

* The object will be deleted once the reference dies.

That's a zero-overhead GC that is enforced at compile time.

And that is the main appeal of C++. You can get close-to-optimal instruction-level behavior while still describing things at reasonably high levels of abstraction. Yes, it has a hell of a learning curve, but once you master it, you can beat any other language out there.

unique_ptr doesn't have any aliasing guarantees, unique_ptr::get() will happily hand out references to the interior pointer.

It does give you non-ambiguous ownership semantics which is a good thing but there's still plenty of ways to misuse/abuse it.

That said if you can make the aliasing guarantees(such as Rust's borrow checker does) that opens up the possibility of automatically applying restrict semantics[1] which enables further set of optimizations available to the compiler.

[1] https://en.cppreference.com/w/c/language/restrict

There's a lot of UB in that spec. Are compilers capable of warning/erroring UB in this case?

How is std::unique_ptr different from Rust's Box? Everything you said applies to Rust.

Also, I definitely would not call std::unique_ptr (or Box) a GC.

They’re the same at a high level, but when you get into the details, there are different. A famous one is that uniq_ptr can’t be passed in registers (mentioned upthread) for ABI reasons, but Box doesn’t have that restriction.

Another one is what happens when you move a value out, in C++ it’s in an “indeterminate state”, but in Rust you can’t access a value after move. Which means that you can get a null pointer exception in C++, but not in Rust, and means that destructors for uniq_ptr have to check for this case, whereas in Rust they don’t.

I understand that. I was talking about the high-level idea of using RAII to manage memory. Parent was talking as if that was unique to C++, while Rust also has it and does it better, for the reasons you stated.

Rust defaulting to move-semantics and requiring explict copies is one of the best decisions in the language.

Note that the ABI issue with trivial types is not a language issue but a gcc/clang backward compatibility issues with their own ABI. A bit of forward thinking could have prevented the issue (like annotating the type when it was first introduced).

> That's a zero-overhead GC that is enforced at compile time.

You should check out Rust's concept of ownership. The goodness of ownership that is implemented in unique_ptr in C++ exists for every type in Rust.

>There will be not more than 1 reference to the object at any time.

Untrue:

    auto x = std::make_unique<int>(5);
    auto& y = *x; // There are now two mutable references to the object represented by x.

>That's a zero-overhead GC that is enforced at compile time.

It's not zero overhead unfortunately. Because unique_ptr is not trivially movable or trivially destructible, there are numerous optimizations that every current C++ compiler does not perform, for example being able to pass a unique_ptr in a register.

unique_ptr is reasonable given C++'s limitations, but it doesn't come close to being as good as Rust's Box, which is genuinely zero overhead and also guarantees that there's only 1 mutable reference to the object at any given time.

While we're doing code review, in general there's no reason to pass a shared_ptr, even by const ref, unless you need to (e.g. if you might copy the shared_ptr within the function). It's better to pass a raw pointer to indicate that the function only requires a temporary non-owning view into the data.

  static int add_numbers(Data *p) {
      return p->a + p->b;
  }

It's best to pass a reference to specify that the argument shall not be null. Also avoid writing static functions in C++, make use of anonymous namespaces.

    namespace {
      int add_numbers(const Data& p) {
        return p.a + p.b;
      }
    }

Agreed, though at some point we run up against the inherent triviality of the example :P.

There's no reason to avoid static functions. It's just a better syntax and it's a shame it cannot be applied to class declarations as well.

> That's because you are explicitly telling the compiler to create a copy of your shared pointer

This is also happening in the Rust code (x.clone()).

I don't know Rust, but the documentation seems to say std::rc is single-thread reference counted pointers? C++'s shared_ptr is multithreaded, so it naturally incurs much higher overhead. (In fact I think most C++ documents would discourage using shard_ptr unless you have to.)

Haven't used much C++ recently, so I don't know if the standard library has a good single-thread implementation these days, but it's not "too hard" to roll your own if performance is critical - it may not be as good as just two instructions, but it should be much leaner than shared_ptr.

Indeed. Switching the Rust example to Arc gives code that's more similar to the C++ implementation: https://godbolt.org/z/Eoh7Pz6oh

why do you x.clone()? it isn't in the C++, and that shaves off a few instructions ;)

I think that there is implicit clone() in C++ code because when we pass a shared_ptr to a function, its reference count gets incremented (and decremented later) and it would be unfair if we used move instead of clone in Rust.

If you don't, the Rust version will be moving the Arc, but the C++ version will be copying the shared_ptr. To make both versions comparable, you must either use Arc::clone on Rust, or use std::move on C++.

Indeed! Rust fails to notice that Arc cannot be accessed from different thread and makes unnecessary locked increment/decrement in add_two_numbers_smart_way().

Your observation of the resulting output is correct, but not your conclusion about how it got there is not. The compiler doesn’t try to verify that the Arc is actually necessary, because there’s no point in optimizing the rare case where someone used an Arc but failed to Send it to another thread. The common case is that the author knows they won’t be sending it to another thread and so they use an Rc, or they know that they will so they use an Arc.

The C++ language standard folks had to make a different choice. For historical reasons the C++ compiler is not able to prevent the program author from misusing a non–thread–safe smart pointer by sending it to another thread, so all smart pointers are required to be thread–safe. The compiler may try to optimize away the extra locking, and it may or may not succeed in any given program. It may not be able to prove that it is safe to do so even when you as the author know that it will be.

Incidentally, every Rust program comes with a free test runner which is multithreaded; it runs multiple tests in parallel. This trips people up from time to time because the compiler will insist that certain types be thread–safe (tagged with the Send trait, in Rust) even though the author is not actually using any threads.

Does the test runner actually pass any user-values between test threads?

Not on its own. It’s basically “hey take these functions, each is a test, run them on multiple threads” and that’s it, nothing fancier.

Right. It just affects other decisions about the language, like the type of static items. In a purely single–threaded program it would be ok if statics weren’t Sync, but it would be a lot of work for the compiler to verify that. And since every program needs tests anyway, it might as well always require them to be Sync and then it doesn’t have to check.

Totally.

I take a slightly different view in the statics thing; like a static means you want something to be available across threads. So by using a static, you’re opening yourself to that possibility. If you want something global to a thread, use thread local storage, and Rust will allow you to use stuff that’s not okay if multiple threads were involved. It’s about expressing your intent correctly.

TIL thread local storage!

When I mentioned Send above, I meant Sync. I mix those two up more often in the evenings.

This looks like a bug to me, the fact that Arc is thread safe shouldn't matter with the removing the allocation here.

I opened a bug about this on the rust repo. [1]

[1] https://github.com/rust-lang/rust/issues/97751

It's because shared_ptr uses an atomic count to synchronise between threads while the Rust version is assuming only one thread.

There is no equivalent in the standard C++ lib. Though it's very easy to write one. If you use gcc's libstdc++, the internal shared_ptr is actually templated on the lock policy so you can do something like: template<typename T> using nt_shared_ptr = std::__shared_ptr<T, __gnu_cxx::_S_single>;

Then if you use nt_shared_ptr instead, the code will be much simpler.

That said, your C++ code is incorrect as well. It gives the ownership to the shared ptr of a stack variable. During the destruction of the shared_ptr, delete will be called on a stack address leading to undefined behavior

You are right. First version of code actually used std::shared_ptr<Data>(new Data) (and produced similar code), but then I thought that it is unfair that we create a struct on a stack in Rust, so I changed the code. I should have used gcc with -Wall that detects the problem (clang doesn't).

Then, instantiate the smart pointer with a custom no-op deleter and if you use the nt typedef I mentioned above, you'll get a similar compiler output to Rust's.

e.g nt_shared_ptr<Data> x(&d, [](Data*) -> void{});

https://godbolt.org/z/d3qv4cE1v

Why call new? I work on a several hundred k code base that has a single call to new.

It is a rookie mistake to wrap the result of new into a shared_ptr. It makes two heap allocations. You should use std::make_shared. It combines both and makes only one allocation.

Boost has local_shared_ptr:

https://www.boost.org/doc/libs/1_65_0/libs/smart_ptr/doc/htm...

Because unique_ptr (and shared_ptr) have non-trivial destructors, due to the Itanium C++ ABI they cannot be passed in registers.

See https://stackoverflow.com/a/58340952.

Everything in C++ is zero cost, until it's not...oops!

> due to the Itanium C++ ABI

But you don’t have to follow anyone’s ABI inside your own code do you?

In C++ and other system languages you generally do.

In general they assume object at a time compiler (in the compiler sense, not types), and different objects may come from other compilers, instances, or even languages. So they use the standard ABI.

For an internal function they don’t have to use the standard ABI as by definition all users of that function are from within the current object. In such cases compilers can optimize the calling convention. I recall that msvc at least did this - but in those cases they’re free to change anything - they can in-line and not include the function at all in the resultant object. That’s why you often see “static inline”

I believe compilers may not stick to strict ABI in a LTO build since there’s no real object files and the compiler has a full view of the build. I don’t know if clang/GCC do things to take advantage of that but I feel like they would be doing that for easy perf wins.

Why are we using shared_ptr here? With unique_ptr The C++ version is trivially short. It's hard to reason about why the compiler does something in this case but I feel like the short version is that nobody would write this program, therefore nobody cares what the compiler does in this case.

https://godbolt.org/z/r3bh3s3Y5

Because we might need add_numbers() to be able to work with reference-counted data. Obviously this program by itself is useless, but these functions could be a part of a larger program.

Sure, but if you need it to work with actual reference-counted pointers from external callers, how could it be simpler? The Rust version doesn't have that ability.

The best we could expect from this example is for the C++ compiler to deduce that the shared_ptr is irrelevant because the callee has internal linkage.

As I understand, one cannot convert shared_ptr to unique_ptr, so if we declare an argument to add_two_numbers() as unique_ptr, then we won't be able to pass a shared_ptr to it.

No, we don't.

add_numbers() doesn't need ownership of the data, so you would never give it ownership of the data. If you happen to have the data in a reference-counted type you would get the data from it (https://en.cppreference.com/w/cpp/memry/unique_ptr/get / https://en.cppreference.com/w/cpp/memory/shared_ptr/get), and pass it to add_numbers(). The data would still be guaranteed to exist because the caller of add_numbers() is going to hold a reference at least until add_numbers() returns.

C++'s std::shared_ptr<T> is not at all like Rust's Rc<T>. std::shared_ptr<T> is more like Arc<T> or Arc<Box<T>> depending on the deleter.

Also, your Rust example uses -O (which is -C opt-level=2), but your C++ example uses Clang's -O3 (which is like rustc's -C opt-level=3).

Here are examples which are more similar: Rust: https://godbolt.org/z/We7ah1ehe C++: https://godbolt.org/z/r31jrcbjc

> Smart pointers are often used in high-performance code (like browser engines) so it is interesting to learn how their usage impacts the generated code.

Browsers like Firefox and Chrome don't use std::shared_ptr. You should research how the compiler optimizes those implementations.

You're passing a shared pointer by value. That's two layers of misuse. Let alone writing out a constructor for basic types which the default one will handle.

I can't demonstrate it because godbolt is utterly broken on mobile, but passing unique_ptr reference is the clean way.

Importantly, you can just pass raw pointers or references around because adding objects does not semantically modify them or their ownership.

Mostly by careful control of aliasing and mutability.

Maybe, but in the example functions have only one argument with a pointer, so there cannot be any aliasing. And there is no need for locks because there is no pointers shared between threads.

I think the answer is a combination of both and inlining. With Rc, the compiler can inline the reference count increment/decrements as well as the non-public function add_numbers and the surrounding code is simple enough that it can do dead code elimination as well as other optimizations to remove the unobservable behavior. If you switch to use Arc (which is a better equivalent of shared_ptr as it also does locking/unlocking), the generated code is more comparable (funnily enough, part of the code for drop in Arc is deliberately not inlined so it prevents this optimization [1]).

Also, if one makes add_numbers public, the generated code is much larger. I don't know why the compiler chooses not to inline it in this case, probably (really speculating here) because it can be called in multiple locations and the IR for the function is over the inlining budget.

[1]: https://doc.rust-lang.org/src/alloc/sync.rs.html#1088

But someone could come along in six months and make a pointer to one of those fields, which would be an alias and would simultaneously allow mutation. The C++ language allows that, and so the C++ compiler must be very careful about optimizations.

Meanwhile in Rust if someone added the same kind of reference then the compiler would force it to be immutable (trying to create a mutable reference would cause a compiler error). Mutating the field would require that some part of the code own the value, not just own an Rc with a reference to the value inside of it. In Rust you can have mutability _or_ you can have multiple aliases, but not both at the same time. Thus the Rust compiler is free to make optimizations the C++ compiler cannot.

(But you’re right about the specific program here; it’s too simple for aliasing and mutability to really matter; I was just answering the general question.)

Also, if someone passed this Rc along to another thread then they would also get a compile error. If they really need to do that then they can make the jump to the Arc type, which is thread–safe, and then the compiler will allow it. But they can never make the mistake of sending an un–thread–safe value to another thread where it will cause race conditions, data corruption, etc.

As time has gone on, “if someone comes along in six months” so to me, a fantastic reason to use Rust. These things can feel onerous at first, but the pain it saves you later is wild. Niko has an anecdote about how he had an Rc<T> somewhere deep in a program, and later he came back and added in threads, and the compiler was like “oh no! You gotta fix that first!” and that would be a latent bug in 99% or programming languages.

Humans are fallible, but compilers don’t forget.

Constraints liberate - liberties constrain

Reply on news.ycombinator.com