Comment by nkurz | Hacker News Reader

Comment by nkurz | Hacker News Reader

Stroustrup says:

  The real novelty here is the return statement: Note that
  I return a potentially huge vector by value. In C++98,
  that would typically cause the copy of all elements of
  res, potentially many thousands of elements. That would
  be a serious performance bug. In C++11, vector has a
  “move constructor,” so that rather than copying elements,
  the representation of res (merely three pointers) is
  “stolen” for use in the caller and an empty vector is left
  behind. After all, we are just about to return from 
  find_all() and won’t be able to use res again. Thus,
  returning a vector by value costs at most six word
  assignments independently of the number of elements.

  Move constructors is a simple facility available to every
  programmer and used by all standard-library containers
  implemented as handles. This allows us to return large
  objects from a function without messing around with
  memory management (explicitly) using pointers and
  free store.

Are most C++ programmers excited by this? Is the idea that people should start writing code depending on this behind the scenes behaviour, or that we now have a way to speed up poorly written code? It feels like an awful lot of effort to avoid returning a pointer. And if I were actually worried about the performance, I wouldn't feel comfortable just hoping it happened. Is there any confirmation by the compiler that it handled this in the way the programmer wanted?

I don't feel it's behind the scene at all, the behaviour is fairly straightforward and easy to understand. It seems pretty error-proof as far as I'm concerned.

We already trust the compiler to do a lot more tricky optimisations (loop unrolling, inlining, dead branch elimination...) so it doesn't really trouble me.

Loop unrolling, inlining, dead code elimination does not change the asymptotical complexity of the program. This optimisation does.

It's not an optimization: it's move semantics. And it's default behaviour for any standards-conforming compiler. Moves can be optimized away still, but by default, returning a std::vector from a function is a constant-time operation.

Constant folding can (theoretically) turn a O(N) loop into constant-time, and may trigger further dead code elimination. Your typical template metaprogram for computing Fibonacci numbers will do the same.

The time when mortals could more or less 'see' what the memory usage and execution time of a C++ code fragment are is well beyond us.

The compiler is required to put calls to nove constructors in specific placez, it doesnt change based on compiler or optimisation level.

The problrm with returning a pointer is now someone has to worry about when the memory that pointer refers to gets ckeaned up. In my current 50,000 line c++ project, there are exactly 3 places where I call new or malloc, and then have to manually worry about cleaning up the result.

You don't just hope that it happens. Move semantics increase the usability of value-type semantics in C++ - you can now do more things with "values" without shooting yourself in the foot (performance-wise) i.e. you can replace some previous uses of references, pointers etc. (note however that moves, while potentially cheap, are not free. There is still RVO.)

It behove users to learn about move semantics before using them (or rather before assuming that they are implicitly using them.) The topic is unfortunately complicated.

There's precedent for this sort of thing in C++, the move semantics are added on top of this: https://en.wikipedia.org/wiki/Return_value_optimization If you are depending on code in the constructor of an object, this optimisation can catch you out.

The alternative to returning a vector by value would more likely be passing an empty vector by reference and then adding elements to it, then returning void. Returning a pointer instead (or unique_ptr) would be replacing value semantics with reference semantics which would be less efficient and inconsistent in interface. That's what move semantics bring to C++, consistent interfaces and efficiency.

I've basically freshly come to C++ in a work environment, and C++11 is all I've learned so far. Range-for loops and smart pointers and move semantics are wonderful. I still haven't adopted to a perfect idiom for calling functions, though: It's straightforward if I call a function and it returns a single object with move semantics. But what if I want to mutate multiple things with a function call (say, go through a vector and throw out indexes below a certain threshold, updating an unordered_map so as to specify the names and locations of certain columns). Do I return one, while changing the other by reference? Do I return a pair or a struct that only exists for me to quickly unpack? Do I make the function void and change both by reference?

If you're living without a garbage collector, avoiding that pointer return is potentially quite a big deal.

"Just use a GC" is a viable answer to this in many circumstances, but probably not all.

"Just use a GC" is not a viable answer in the C++ world, because for C++, contrary to Java, there are no GCs of acceptable throughput and pause times.

You can use automatic reference counting.

It is sometimes handy, but generally compared to GC it is slow and incurs lots of contention in multithreaded programs.

Imagine you return a reference-counted pointer from a function. Then every call to this function is a lock/unlock and CPU cache synchronization. And if the pointed object is shared between threads (even for read-only access) this might lead to lot of contended locks/unlocks, which is extremely costly.

The synchronization you mention is really rare, it only happens when you make a copy (not a move!) of a std::shared_ptr.

If you have a std::shared_ptr<X>, then most of the time you will be passing around "X &" or "std::shared_ptr<X> &" and no copies will happen.

Do you have an example of why a function would be returning a std::shared_ptr<X> by value that wasn't a move?

To return a shared_ptr you first had to create it. So even if it optimizes the move, it doesn't optimize away the creation of the pointer and its destruction.

As of using naked references, you're right, but if you can use naked references, why ever use shared_ptr? Naked references are fine when you know the lifetime of the object.

Don't get me wrong - shared pointers are useful in many scenarios, but they are not a general replacement for GC.

BTW: Here is the post of someone who thought he can use shared_ptr as a poor-man's GC: http://stackoverflow.com/questions/2502394/the-cost-of-passi...

shared_ptr took 17% of time. This is far beyond what GC takes even in a very memory heavy Java application (typically GC stays below 1% overhead).

I'm not saying you shouldn't use shared_ptr at all, but only: be careful.

Novices also sometimes erroneously pass std::vector by value, that doesn't mean std::vector is slow. Maybe you did not read the entire post before linking it. The conclusion was he changed his code to pass by reference and 17.77% became .27%.

Thanks for pointing that out, I had heard horror stories before and was willing to take 17.77% at face value!

You can use naked references because most functions aren't acquiring ownership of a shared_ptr.

I always hear about how use of mutexes is costly but i never actually encountered a significant performance hit from using synchronization. Unless, of course, you have like 10 threads frequently accessing the same memory, which is more a design issue.

Uncontended interlocked operation is typically about 100-300 CPU cycles. Uncontended mutex is about 30x-50x that. If there is contention, multiply everything by another 100x. Additionally these instructions impact not only the calling thread but also other threads by forcing cache synchronization.

Therefore a relatively simple operation like pointer assignment can be one cycle when using GC and a hundred to several thousands cycles when using refcounting. Sometimes you can ignore this overhead, sometimes not - depends on how often you do that.

http://software.intel.com/en-us/articles/choosing-between-sy...

On what CPU? on my i7s it's much faster. on the order of 50 cycles for a uncontended mutex lock and release i think. I can dig out the values if anyone cares.

https://en.wikipedia.org/wiki/Boehm_garbage_collector They use this in Mono, what do you mean by throughput and pause times? Is it far inferior to the Java GC?

They stopped using Boehm since 3.2 release of Mono. In certain cases the new GC is 10x faster.

It is. Actually it is in a completely different league: 1. non-generational (= low throughput) 2. imprecise (= no guarantees on freeing unreferenced memory) 3. stop-the-world (= long pauses) 4. non-compacting (= allocation is slow and fragmentation still might be a problem)

I am not that good at C++ so it would be really nice if somebody can enlighten me:

Isn't this already the case with some (all?) compilers in pre-C++11?

http://www.parashift.com/c++-faq/return-by-value-optimizatio...

NVRO works by constructing the object once... (EDIT: detrino points out this is wrong, the object is not constructed first before passing, it is only a memory address that is passed, it's the return value that is constructed on to this address)

It is passed as a hidden argument to the function, assigning the return value to that argument passed in. Prior to the NVRO, the object was constructed once, a temporary was constructed and then copy constructed to give the result. You can test out the NVRO yourself by putting a printf in an object's constructor, and have the object returned from a function on to an object of the same type.

Stan Lipmann's Inside the C++ Object Model explained it best for me. NVRO applies to the vector object.

The difference here is that Stroustrup is talking about the elements of the vector, not the vector itself, and how the move semantics remove the need for constructing/copy constructing them.

What you are talking about is really just the calling convention common on many machines. When you have a function returning a struct, what really happens is that the return address for the struct is passed as an extra argument. No construction of the struct happens prior to this, its just an address. Compilers are able to take advantage of this calling convention in many cases to elide copies, and the result is RVO/NRVO.

Are you alluding to some sort of hardware mechanism by mentioning machines? In what context are calling conventions machine dependent?

The reason it's notable in C++ is because the code that would normally be called in the constructor is not called as many times as the person writing the code might expect.

In C++ structs can have constructors, but usually have an empty default constructor supplied by the compiler, this is more of an issue when structs/classes have user supplied constructors.

Here's an article giving more specifics of the history of NVRO from Stan Lipmann. Interestingly, NVRO was not added to Visual C++ til 2003, and Lipmann prefers NVRO off by default. I think I recall an NVRO flag in that compiler. NVRO was available in cfront and Zortech compilers in the early nineties. [1]

[1] http://blogs.msdn.com/b/slippman/archive/2004/02/03/66739.as...

The calling convention your compiler is using is dependent on a number of things [1]. It is this calling convention that dictates how values are returned from a function, not whether or not RVO/NRVO is happening. Also, what is being passed as an extra argument, according to many calling conventions, it not really an already constructed object, but an address for an object to be constructed into. Under certain situations the compiler is able to elide copes, which as you pointed out, leads to your copy constructor to never be called at all, even if it has visible side effects. I think these are important distinctions and hoped pointing them out would be beneficial.

[1] http://en.wikipedia.org/wiki/X86_calling_conventions

Yes, thanks for pointing the calling conventions out, I didn't really know what you were getting at.

The hardware calling convention does dictate certainly what code the compiler can put out.

If the compiler does not support NVRO and emits code that causes multiple copy/constructions it doesn't matter if the hardware supports more efficient behaviour.

I edited my earlier comment with your correction, thanks.

The compiler is not required to perform RVO/NRVO and when it can be done depends on some surprising things such as the calling convention of the machine you are targeting. It's just an optimization the standard allows compilers to do (eliding copies).

What C++11 provides is a new kind of type that says "it is safe to steal the guts of this value". You can use this to overload a copy constructor or assignment operator to do "moves" instead of "copies". Because it is baked into the type system, it is guaranteed to happen when you expect it. While eliding a copy essentially has 0 cost, and the compiler will still prefer this over a move, moving a vector can have a very small cost (copying a few pointers), but it is still O(1) instead of O(n).

Reply on news.ycombinator.com