Hacker News Reader

Comment by overgard | Hacker News Reader

My (least) favorite footgun is "auto" when it comes to references.

If you want to get a pointer from something, you would write this:

    auto fooPtr = mywidget.getPtr();
    fooPtr->doCoolStuff();

But we all know references are better than pointers right? So we should just write...

   auto fooRef = myWidget.getRef();
   fooRef.doCoolStuff()

The problem is... this is valid and will probably not do what you want. It will make a copy. What you want is actually

   auto& fooRef = myWidget.getRef();

I once spent an entire day debugging a bizarre crash because of that. I was asking for a reference to a scene graph, and what was actually happening is I was getting a clone of the scene graph (which was an object that couldn't be safely copied), and then destroying a bunch of shared pointers when the function returned. Fun times.

View on HN

As others have pointed out, the idiomatic solution here is to delete the copy constructor if an instance is unsafe to copy — however, I suspect the reason why auto behaves differently from other inference in requiring this to be explicit is probably something along the lines of making it harder to have a dangling reference. It’s shockingly easy to wind up with a dangling reference with fairly innocuous code, something like QString().toUtf8().data() so maybe this makes sense. (Doesn’t help for that case since it’s a pointer to raw data, but you get the picture.)

Worse yet is an object which is safe to copy in some cases, but you intended to not copy it at the moment, and you create a dangling reference to a field within. I wish C++ copy constructors were explicit by default. (But declaring the copy constructor explicit turns off aggregate initialization, so I can't do that.)

I think, all things considered, the way it works is probably the best way, since you might want to use "auto" to actually make a copy. It's just very surprising, because when I have explicit types I'm used to looking for that sort of error, but with an auto I wasn't (at the time) used to looking for that.

Wait, you had an object that was unsafe to copy, but you told the computer it was safe by not deleting the copy constructor? I guess it should have done what you wanted, not what you told it.

No, everyone gets LHS auto wrong at the start. They don't realise it makes a copy. I have to point it out every other code review.

The rule is simple: auto behaves like a template parameter

Ah, templates... Universal references... Good stuff.

Yep, except now we're supposed to call them forwarding references because the motivation Scott Meyers had for naming them universal references turned out to be incorrect in subtle ways.

Even the notable experts get things wrong when it comes to how complex and bloated C++ is.

There is nothing simple about that (not to mention it's not actually true either).

Yes it is ? Quoting the standard:

> If the placeholder-type-specifier is of the form type-constraint auto, the deduced type T' replacing T is determined using the rules for template argument deduction.

https://eel.is/c++draft/dcl.spec.auto

Now if you program in C++ regularly and don't know that in

    template<typename T>
    void f(T t) { }
    f(something);

does a copy of something, I don't know what to say - I have never met any professional c++ programmer not knowing the language works that way

You need to continue reading that paragraph to the very end and note the exception for when an initializer list is used. The following are not equivalent nor do they apply the same rules:

    template<typename T>
    void f(T) {}

    f({1, 2, 3});
    auto x = {1, 2, 3};

>Now if you program in C++ regularly and don't know that in...

I program in C++ daily and no, I didn't know that a copy is made. What I do know, as an actual professional C++ programmer, is that whether a copy is performed is incredibly complex and depends on numerous factors such as whether a move constructor exists and "something" is an lvalue but not an rvalue reference, or if "something" is an rvalue (which is not the same as an rvalue reference) and T defines a move constructor.

And finally if something is the same type as T and T has a copy constructor, then the final question is whether copy elision will be performed, which the standard specifies is a valid optimization even if said optimization would change the observable behavior of the program.

That's what I... as a professional C++ programmer know but I fully admit that C++ is such a complex beast of a language that I am almost definitely missing a few corner cases.

> You need to continue reading that paragraph to the very end and note the exception for when an initializer list is used.

Exceptions don't mean that the general rule does not apply ? I don't know how it is possible to be more explicit that "the deduced type T' replacing T is determined using the rules for template argument deduction.". That'd be like saying that "priority to the right" when driving isn't a rule because there is sometimes a "stop".

> I program in C++ daily and no, I didn't know that a copy is made.

you're kidding

> whether a copy is performed is incredibly complex and depends on numerous factors such as whether a move constructor exists and "something" is an lvalue but not an rvalue reference, or if "something" is an rvalue (which is not the same as an rvalue reference) and T defines a move constructor.

but `something` cannot be a rvalue in f(something);

maybe in f(something()); or f(some + thing); but just a variable named "something", as is, passed to a template argument or auto will necessarily lead to the creation of a new value of the same type. Whether it is copied, moved, materializes three different types because people forgot to mark their constructors explicit or whatever else frankly does not matter much in my experience (after all people survived for 30 years without move semantics) - what matters is whether a new value is created (which I personally and improperly call a "copy" no matter what - I should have been more precise but it's really the most useful distinction to make in practice imho as it means that some function will be called) or just a reference to existing value which is always a no-op no matter what.

> And finally if something is the same type as T and T has a copy constructor, then the final question is whether copy elision will be performed,

I've been at it for 15 years now and I don't remember one time in non-toy-example-code where copy elision taking place or not did matter. Thinking about is is a self-inflicted problem ; expect that a copy happens, if it's slow, profile and if it's not, be happy.

Simply by using C++, you tell the computer all sorts of things, such as "I know the entire language by heart", and "I have a death wish".

While C++ is a dangerous tool indeed, ignoring the rule of 5 is analogous to ignoring the safety guidelines, not simply using the tool.

A C++ object can be perfectly safe to copy in one situation and conjure up nasal demons in another situation. I don't think rule of 5 would help in that case.

If the copy behaviour is that different between the two cases it should be two different classes.

I'm reminded of a quote about C, sadly I forget where it's from:

> It is fundamental tenet of C's philosophy that the programmer is always right, even when they are wrong.

Honestly I think that's what makes it (and C++) a great tool, despite the frustrations. I'd rather have a footgun than a nerf gun.

> I'd rather have a footgun than a nerf gun.

That's not really the choice we're faced with though. Ada and Rust are both far safer than C, and they're also plenty powerful.

Yeah I learned that the hard way when I found out that std::vector apparently feels free to move your objects around in memory for you, no matter if you point to them from somewhere else.

I mean in hindsight it's obvious, but still not exactly what I wanted to happen.

For reference, what I did does actually work, temporarily.

> Yeah I learned that the hard way when I found out that std::vector apparently feels free to move your objects around in memory for you

In standard terminology, this is described as "invalidating iterators". There are a bunch of member functions in std::vector that either do or don't invalidate iterators, e.g. push_back(...) does but size() doesn't. And as the name implies, if you call a function that invalidates iterators, all your existing iterators/pointers/references become invalid.

Iterator invalidation is different from this. Iterator invalidation, as the name suggests, only applies to iterators. The problem OP was having had to do with dangling references.

Some containers guarantee references won't dangle on mutation such as unordered_map. An unordered_map may invalidate iterators if objects are added or removed, but will never result in dangling references (unless the object is removed). That is, it is safe to have a pointer to an object owned by an unordered_map and continue using that pointer even after an iterator to that same object is invalidated.

Ah, interesting. Even within that dichotomy, I'd think that if references aren't invalidated on mutation, then dereferencing iterators would be valid, but incrementing them would not be.

Iterators are historically a generalization of pointers. A pointer is a kind of iterator.

I mean, literally. The iterator of an std::vector<T> (not a bool vector) is a T*.

This is a poor understanding of what an iterator is. For one, the iterator of a std::vector<T> is not a T*, it's a std::vector<T>::iterator and you are welcome to verify that the following assertion fails for all T:

    static_assert(std::is_same_v<T\*, std::vector<T>::iterator>)

Link provided below for convenience [1].

Furthermore just like a cat is animal does not imply an animal is a cat, a pointer being an iterator does not imply that an iterator is a pointer and holding such a confusing thought is the source of many bugs and poor C++ code. An iterator not only represents a form of indirection to an object, it also represents traversal through a collection of values.

unordered_map guarantees that the objects it stores will remain alive until the unordered_map is destroyed or the object is removed from the map. However, unordered_map, as the name once again suggests, does not provide any guarantee about the ordering of objects. Hence, if you have an iterator to an object owned by such a map, and you add or remove objects to that map, the iterator becomes invalid not because the object the iterator references gets destroyed, but because the iterator loses information about where it's located within the unordered collection.

For further information, please review the following link [2], specifically the quote "References and pointers to either key or data stored in the container are only invalidated by erasing that element, even when the corresponding iterator is invalidated.".

[1] https://godbolt.org/z/Y5f4d74Kb

[2] https://en.cppreference.com/w/cpp/container/unordered_map

You're being far more patronizing than is necessary here, especially because the parent poster was correct in iterators originating out of a being a generalization over pointers, although not in their latter statement. But, while implementation defined, a vector iterator must behave identically to a pointer, and your assertion fails in both Clang and GCC not for any interesting reason but more because they wrap the pointer into a class for what I understand to be better encapsulation and earlier detection of nonportable code, not because pointers are somehow unfit to do the job of std::vector<T>::iterator. It so happens that iterators have become a useful way to generically iterate over noncontiguous containers as well, but it is very clear that they are "logically" meant to mirror the usage of a pointer (down to the operators you use to work with them) and suggesting that holding this opinion is a sign of inadequacy is condescending.

>because the parent poster was correct in iterators originating out of a being a generalization over pointers

And an animal is a generalization of a cat, but that doesn't mean an animal is a cat. What is true of a particular doesn't have to be true of the general.

> But, while implementation defined, a vector iterator must behave identically to a pointer

No it does not. MSVC implements a vector iterator with additional runtime safety guarantees in DEBUG mode and while those guarantees are not mandatory as per the standard, they are fully in compliance, so saying they have to behave identically is patently false as MSVC's behavior is a superset of the behavior provided by a pointer.

> suggesting that holding this opinion is a sign of inadequacy is condescending.

If you care about writing correct code, then do not mix the very concept of pointers, arrays, references, and iterators with one another. They are all related to one another but have very important differences to the point that calling them literally identical to one another is a sign of inadequate understanding. It happens far too often and inevitably leads to poor code, bugs, and a fundamentally poor understanding of what these things represent conceptually.

First of all, calm down. Consider that the people you are talking to may not be idiots.

Second of all, nobody is saying that pointers, arrays, references, and iterators are identical in general. They are clearly related concepts but differ in fundamental ways. However, my point was that a vector iterator needs to behave like a pointer would and can in fact be implemented as a naked pointer. Pointing to MSVC's implementation having safety checks does not change that fact, because pointing to what an implementation does on undefined behavior is clearly out of scope when discussing specified behavior. I'm not going to bring address sanitizer into this discussion to show how pointers can actually be a superset of "pointer behavior" because that would just be irrelevant. The fact remains that iterators were originally meant to act like pointers, and in many cases still do. Believing this does not mean I have a poor understanding of the language.

Not knowing something does not make you an idiot, there are tons of things I don't know about C++ and I make tons of mistakes. However, not knowing something and doubling down on that lack of knowledge by defending something that is wrong because you don't like the tone of the person correcting you about it is highly problematic and suggests you are prioritizing your own sensitivity on the issue and protecting your own ego instead of taking the opportunity to learn something new.

>Second of all, nobody is saying that pointers, arrays, references, and iterators are identical in general.

The comment I replied to said, and I quote:

"I mean, literally. The iterator of an std::vector<T> is a T*."

That is what I originally replied to before you felt it necessary to interject yourself into the conversation to police my tone.

>However, my point was that a vector iterator needs to behave like a pointer would and can in fact be implemented as a naked pointer.

No, you said it is required that they behave identically, which is false, once again, I quote you:

"a vector iterator must behave identically to a pointer"

There is no such requirement that they behave identically, and there is a demonstrable example of a compiler where they do not behave identically. Furthermore your talk about it being undefined behavior is false on the basis that in general, two pointers of type T* may compare with one another, for example the following is valid:

    auto v1 = new int();
    auto v2 = new int();
    v1 == v2; // This is valid.

However, this is undefined behavior.

    auto v1 = vector_a.begin();
    auto v2 = vector_b.begin();
    v1 == v2; // This is undefined behavior.

Only iterators to the same vector may be compared, which is not true of pointers.

>Believing this does not mean I have a poor understanding of the language.

I did not say you have a poor understanding of the language as a whole, I said if you believe that an iterator to a vector is identical to pointer as you have claimed, then you have a poor understanding of iterators.

I stand by that statement and I would suggest that you not be so sensitive about being wrong about a corner case of the language and instead simply accept this to be a fact, learn something new, and move on with your day. I learn new things about C++ all the time, sometimes from polite people, sometimes from jerks, it's all the same because what matters is the learning. Something doesn't become wrong just because the person who told it to you said it in a way you disapprove of.

Having said that, all the best to you Sir and thank you for engaging in this discussion. I have said all I think is appropriate given the topic.

> However, not knowing something and doubling down on that lack of knowledge by defending something that is wrong because you don't like the tone of the person correcting you about it is highly problematic and suggests you are prioritizing your own sensitivity on the issue and protecting your own ego instead of taking the opportunity to learn something new.

Reminder that I was not the one who passed up the opportunity to reply to 'einpoklum with a simple "iterators can behave like pointers, but vector's iterators don't have to be pointers. You can see this here: https://godbolt.org/z/Y5f4d74Kb". I'm just annoyed that you went after this person for coming to an incorrect conclusion from what is clearly a correct position, which is that iterators were intended to "morally" be pointers, especially in cases where they iterate over contiguous containers. You are correct that I claimed that vector iterators must behave identically to pointers, but I really meant to say that this is only in cases where the iterator has behavior defined on it: I alluded to it previously when I mentioned that GCC and Clang wrap pointers in their own custom iterator classes to prevent people from accidentally using operations on them that are not legal. There's a bunch of other things that the iterator doesn't guarantee, like being directly casted to an integer.

But, stepping back a bit: do you really disagree with a claim that a vector iterator is meant to be a little pointer into the internal buffer it has, with a bit of additional restrictions on top that are reasonable to add? This seems to be a very strange thing to disagree with, and I would like to hear more about why you hold that position.

FWIW I believe that the parent was pedantic but correct.

This being C++, being pedantic is a Good Thing.

The distinction between reference and pointer invalidation, and the requirements and axioms of the various iterator concepts are all valid things to point out, especially in a thread about C++ footguns.

I am always down for C++ pedanticness! But in this case my issue was not with factual accuracy, but putting down the opinion of "iterators act like pointers" as coming from ignorance rather than simplification. It was mostly a complaint about tone, not content.

> It was mostly a complaint about tone, not content.

You say that as if you think that makes it better.

Absolutely!

> > > It was mostly a complaint about tone, not content.

> > You say that as if you think that makes it better.

> Absolutely!

Indeed, you absolutely did. Which sucks, because it absolutely isn't.

HTH: https://en.wikipedia.org/wiki/Tone_policing

Hacker News is a place for curious conversation, rather than a place to sneer at people's ineptness for coming to reasonable (if incorrect) conclusions. You can be completely correct, but if you're going to be a petty jerk about it then you're not helping the conversation. I provided an example response that conveys exactly the same information, except you'll note mine doesn't start with "this is a poor understanding of [general concept]". If anything, "link provided below of convenience" and "please review the following link" is far more sealioning than anything I wrote, because it hides the message of "you're an idiot who needs to read up on this topic" behind faux politeness. It's the very thing that I found disconcerting about that comment, not its factual content.

Ok, as you and others have pointed out - the iterator of a vector doesn't have to be T:
https://en.cppreference.com/w/cpp/container/vector

It _may_ be a T, or anything satisfying LegacyRandomAccessIterator.

> just like a cat is animal does not imply an animal is a cat,

It's not "just like" that. Among other differences between the pairs of concepts: The concept of an animal didn't arise as a generalization of cats.

This has its own problems. Pointer stability limits the internal implementation so much that unordered_map is embarrassingly slow given what we know about modern hashmap design. There is a good reason why the swisstables that Google built dropped this behavior.

I think if you come from the JVM / CLR world, you are so protected by the runtime’s patching up of references that it might not even occur to you that a (raw) pointer to a data structure’s internals can dangle after the data is moved around. The runtimes mentioned pause your code, move things around and even compact the heap and your references magically still point to what they did before!

Except, it's not magic. It costs processing cycles, and latency, and invalidations of cache rows you were using, and memory bus traffic. Lots of all of them.

Most of the costs are not counted as runtime for your process, so benchmarks invariably appear to show GC as costing less overhead than it does. Among the costs, as with all the myriad varieties of caching done in modern systems, is that GC makes it hard to know the costs of design choices you make. Many of the costs are in making the caches less effective.

Well it's not so much being protected by patching up references, but more that C++ is (as far as I know) the only language that doesn't complain when you create a reference to an object that can move away.

Most languages either don't have the concept of classes or ensure that invalidating references is an explicit operation (such as calling the destructor).

This is why things like std::list still have value: iterator stability. You even get this in most map/sets too.

But vector is usually what one should reach for unless one has a good reason not to. The only danger really comes when the lifetime of iterators and the thing they point to become a bit decoupled, even due to insertion!

You can also use an offset instead of a pointer (or a pointer to the object and and an offset into its heap storage, or whatever).

This is actually on my list of "neat utilities to write": an iterator that encapsulates this "offset" for vector.

If anyone is looking to write a custom iterator, I definitely recommend Boost's new stl interfaces: https://github.com/boostorg/stl_interfaces

Or you could just use Boost stable_vector and save yourself the trouble

https://www.boost.org/doc/libs/1_76_0/doc/html/boost/contain...

deque is also good for reference stability when doing push_backs (but of course not when inserting in the middle).

> Yeah I learned that the hard way when I found out that std::vector apparently feels free to move your objects around in memory for you, no matter if you point to them from somewhere else.

It's not as much as "feeling free" as it is having to reallocate the array because you added elements beyond it's capacity and they had to be stored somewhere.

Well the problem I had was that I apparently didn't forbid it from using the move constructor, after I changed that it correctly failed to compile.

So in my case the problem was very much that it took the liberty of doing something I wasn't expecting it to.

Interestingly, the dangers of reference invalidation is actually one of the reasons why auto doesn't deduce references by default:

   vector<int> v = {42};
   ...

   auto x = v[0]
   v.clear();

   // if x was deduced as reference, using x at this point 
   // would be UB
   std::cout<< x;

> told the computer it was safe

> by not deleting

I think there's a problem here.

Probably one of the only common language-lawyer pieces of criticism of C++ I fully agree with. I'm a pretty firm believer that a language should be as constrained as possible with the ability to make it explicitly less-constrained.

C++ fails at this pretty spectacularly.

Rule of three is such a common C++ concept though, I would be surprised if anyone with more than a few weeks experience with the language is still running into such issues.

I've been coding C++ off and on since 1998. But not all the code I use is written by me... not even most of it. I'd love it if everyone followed best practices... but they don't.

> I think there's a problem here.

There is a problem indeed, which is OP's failure to implement exception-safe code.

If someone opts to use exceptions, they have the responsibility of writing their code in a way that complies with scenarios involving throwing exceptions.

Failing to support a scenario by not following the most basic principles is something that's on the person writing the code, not the language.

It would make as much sense to blame java for the problems you create by not initializing objects in some code paths, because your code started to throw null pointer exceptions.

Are you talking about the same thing as everyone else?

auto fooRef = myWidget.getRef(); vs. auto& fooRef = myWidget.getRef();?

That's not a problem caused by exceptions.

If you're talking about the article, the standards committee seems to think it's a confusing interpretation that never should have existed, and it's pretty ridiculous to say it's "basic principles".

Well, the scene graph code wasn't code I wrote, and this was back in 2014. But sure, obviously it was an error on my part... we are talking about footguns afterall, I'm the one that pulled the trigger.

If you develop on Windows and are using Visual Studio 2019 "auto fooRef = myWidget.getRef()" will get a little squiggle under it to warn you that you are making a copy.

> and aren't using Visual Studio

OK, I tried this in Notepad, and it did not work.

yeah, same with WordPad, but see Word would give you some squiggles :)

I am very happy tooling is helping to make these sorts of mixups easier to find, although sadly C++ is such a hard language to write tooling for that those kinds of nice features are pretty rare. I remember back when I had to work in Xcode a few years ago I could rarely even get basic intellisense to work.

It's not that these features are rare for C++, it's that Xcode is generally a pretty bad C++ IDE (imo, of course).

In both Visual Studio and CLion, these types of features work well, in addition to plenty of more advanced features like address/thread sanitizers, fuzz testing, etc.

You're too nice, Xcode is an atrocity. Still though, trying to unambiguously parse C++ without compiling miles of headers is not a simple thing... I find my typescript and C# tooling is so much nicer when C++ is the language I need that magic most for. VS is nice but compared to what I get with like resharper or intellij its a bit disappointing.

It was already possible in the early 90's.

https://dreamsongs.com/Cadillac.html

"Lucid Energize Demo VHS 1993"

https://www.youtube.com/watch?v=pQQTScuApWk

VisualAge C++ Version 4.0

http://www.edm2.com/0704/vacpp4/vacpp4.html

https://books.google.de/books?id=ZwHxz0UaB54C&pg=PA206&redir...

However they were too resource hungry for the hardware of those days, then Java and .NET took over the spotlight, and until clang/LLVM came into the scene, the industry forgot of what was already possible.

Sounds like the root cause was due to not obeying the rule of five: https://en.wikipedia.org/wiki/Rule_of_three_(C%2B%2B_program...

Probably! I don't remember the code exactly, this was back in 2014 and I think most of the code I had to consume was primarily written by a 24 year old out of college (smart of course, but C++ takes a long time to get good at).

Nowadays, Rule of Zero. Let the compiler generate all the special members implicitly, when the members can each do their own cleanup. This turns out to be quite often.

> What you want is actually > > auto& fooRef = myWidget.getRef();

That variant has issues if getRef() actually returns a temporary object. You may want auto&&.

If it returns a temporary, you'll get a compile-time error, as the temporary cannot be bound to a non-const ref. I would prefer "const auto&" as you shouldn't be modifying temporaries, or "auto x{std::move(myWidget.getRef())}" if I want a mutable value and to avoid copying the return value of getRef.

auto by default removes top level const, reference qualification, and decays arrays and function types to pointer, same as for template argument deduction [1]; this is also true when auto is used as for return type deduction.

You can use decltype(auto) to preserve the actual type of the rhs.

[1] as discussed elsewhere initializer lists behave differently, but initializer_list itself was a mistake to start with.

I too have fallen victim to this and similar traps. it's a good idea to use a deleted copy constructor when you know it's not safe to copy the object. you can't completely stop others (or yourself) from doing bad things, but you can at least give them a moment to reflect on what they are doing.

It's not about safety, it's about efficiency. It's usually an unintended copy. For example, if your function `foo` returns a `const std::string&` the code `auto x = foo()` creates a copy.

well I don't really see a way of solving that. I may or may not want to copy the object, depending on the object's lifetime and what I plan to do with it. warning on copies would be a step too far imo. I would immediately suppress that warning.

A copy of the reference or a copy of the string?

The string,

I wish C++ copy constructors were explicit by default.

The problem is that structures are implicitly (and trivially) copyable in C and requiring adding an explicit copy constructor would break interoperability. It would also break trivial copiability which is a different issue.

So, yes, backward compatibility and historical baggage.

Note that if you use explicit wrappers for all object-owning pointers you would get either the right behavior or a compilation error.

So, if I write:

int x; auto y = x;

do you expect y to be a reference? Surely not.

No, my friend, auto is for values, not references. You want a reference - you have to say so.

(And same goes for rvalue references - auto&& .)

My point was simply that auto to assign pointers doesn't cause them to lose their pointiness, but using auto to assign a reference causes them to lose their referenceness. Yes I understand why it works the way it does. I'm not saying it's wrong just that it's been an occasional footgun.

I think it would be a footgun the other way too.

I tend to think of pointer a as a general type aspect whereas reference is specifically on the variable that it's on and doesn't transfer. Sort or like how auto doesn't copy the static specifier.

Also the rules for auto are the same as for template argument deduction.

    template<typename T>
    std::vector<T> MakeVec(T val) {
        std::vector<T> vec;
        vec.push_back(val);
        return vec;
    }

    int main()
    {
        int x = 1;
        int& y = x;
        std::vector<int> vec = MakeVec(y);
    }

Notice how in this case MakeVec(y) makes a std::vector<int> not a std::vector<int&>. If you want your template argument function to take a reference, use T& not T (or use T&& which is a whole new complication). Similarly use auto& not auto.

Ok, that's a fair point. It's easy to make the distinction when you think about how in C++, things become references or lose their reference'iness pretty easily, but nothing ever implicitly becomes a pointer.

Reference is a type qualifier, pointer is a type. Best to just internalize it, even if you'd rather it be different.

Reply on news.ycombinator.com