The Evolutions of Lambdas in C++14, C++17 and C++20 | Hacker News Reader

The Evolutions of Lambdas in C++14, C++17 and C++20

fluentcpp.com

167 pointsingve73 comments

> Even if you don’t need to handle several types, this can be useful to avoid repetition and make the code more compact and readable.

...

> namespace1::namespace2::namespace3::ACertainTypeOfWidget

Deeply nested namespaces are problematic in themselves due to namespace resolution (e.g. see https://abseil.io/tips/130), but templates should never be an answer to "my type is too long to type out". You're hurting yourself at compile-time, and your code is now more error-prone as you can pass in any argument.

Just add a using-decl (or a namespace alias if the base name isn't meaningful enough for you).

Yes, deeply nested namespaces are not nice. I think boost really dropped the ball on this. boost::asio::ip::tcp should have been boost::tcp. Are they trying to be as bad as java?

As the article explains it's not "as bad as Java" it's worse, what Java does actually makes sense in Java, maybe you have to type slightly more characters but Java is already a verbose language best suited to heavy tool-assist. However it doesn't make sense in C++ because the benefits Java gets don't apply and the price is heavier.

There are a bunch of things like this in C++ where superficially C++ feature X is like feature Y in another language, and so C++ programmers wrongly assume the problems with feature X must also plague feature Y. I'm sure the reverse happens too.

> There are a bunch of things like this in C++ where superficially C++ feature X is like feature Y in another language, and so C++ programmers wrongly assume the problems with feature X must also plague feature Y. I'm sure the reverse happens too.

Yes! I've noticed this also. Several recent C++ changes are copying idioms from other languages, and sometimes the standards committee seems to have missed the point or misunderstood how the features are used in the language they're copying.

The results are superficially similar, but not quite what I'd expect after knowing the feature from the source language.

Your argument seems to be "Java can be bad at this because it assumes an IDE, but C++ can't because it can't assume an IDE". Which is nonsense. If anything nested namespaces make sense in C++ but don't in Java, since Java has no nesting semantics and doesn't have using aliases. In Java it's just raw noise, whereas in C++ it's actual structure. Java isn't getting "benefits" from deep package paths.

Absl's arguments are that you shouldn't use that structure much, so you should avoid deep nesting (specifically deep nesting, not no nesting). But it actually does something in C++, which isn't true in Java. And that's really the core of Absl's complaints about nesting - don't copy your Java package name (which is irrelevant useless noise) to your C++ code (where it's actual structure, and collisions can result in issues)

I thought Titus's tip spells it out pretty well. In Java we know Doodad is some.long.package.hierarchy.Doodad and not some.Doodad or some.other.Doodad or an.entirely.different.Doodad because we explicitly wrote that down. The tooling helps you with spelling but the language provides exactly what is needed here.

If you introduce ambiguity, the code won't compile.

In C++ too bad, you wrote Doodad and thought you were getting ::some::new::package::hierarchy::Doodad however that doesn't actually exist, you were thinking of ::some::long::package::hierarchy::Doodad and your compiler didn't complain because it silently concludes you wanted ::some::Doodad which does exist (maybe it was added years ago and is rarely used, or maybe it was added in the new version you got last week) even though you never actually wanted anything from ::some itself. C++ loves ambiguity, because now it's your fault this happened, you were less than perfect and as such unworthy of the power and grace of C++

> templates should never be an answer to "my type is too long to type out"

Eh, in some cases it's a wash.

    std::vector<std::unique_ptr<mynamespace::MyWidget>> objects;
    // ...
    auto it = std::find_if(objects.begin(), objects.end(), [](auto& widget) { return widget->x == 1; });

Sure, you could repeat the type `std::unique_ptr<mynamespace::MyWidget>` in the lambda, but that's just noise. You don't spell out the types like that either in, say, C#. Yes, compilation time is an epsilon slower, but that consideration loses to readability any time of day. (Otherwise you could just remove all comments and indentation from your code - that also makes compilation faster!)

Is it really slower ? The compiler has to compute the type of the right hand side in any case. It just has one less computation to do now (type checking the conversion to the left hand type)

I am fairly certain template expansion and skipping over whitespace while parsing are two entirely orthogonal things. using a template might make compilation an "epsilon" slower in one instance of such a use, but then those epsilons add up...

When the definition is far from the use of auto, that's exactly the case where I think repeating the type is meaningful.

Yes, it's noise to write:

    std::unique_ptr<MyWidget> widget = std::make_unique<MyWidget>();

But I don't think it's so noisy in the case you've provided. If I encountered that code, I'd wonder whether that operator-> is safe, or if you're inadvertently dropping some qualifier (e.g. is it actually a raw pointer?).

And as I suggested, it's not much harder to add a using decl:

    using UniqueWidget = std::unique_ptr<mynamespace::MyWidget>;
    // ...
    std::vector<UniqueWidget> objects;
    // ...
    auto it = std::find_if(objects.begin(), objects.end(), [](UniqueWidget&) { return widget->x == 1; });

(I also recognize that my opinion is very heavily flavored by the Google style guide: https://google.github.io/styleguide/cppguide.html#Type_deduc...)

auto is an evil temptation that we've gave ourselves easily in.

Template lambdas is in my opinion the most useful evolution and has been a long time coming. I figure it was held back by template haters who want terse syntax and don't understand how it's much more powerful than auto/concepts.

I wouldn't say it's a smaller evolution than previous iterations.

Agreed. C++11 was a major evolutionary step for the language, but C++14 made it fun, too. C++20 is as large a step as C++11, but it will take as long for people to absorb. C++23 will bring usability improvements for async/await and maturity to library ranges.

Another new lambda feature in C++20 is default constructability. I am surprised this is not mentioned in the article.

C++23 drop the word auto?

    myLambda = [](&&x){ std::cout << x << '\n'; };

what's the point of having to write auto everywhere

syntactical disambiguation, there's already something called "the most vexing parse". C++ is full of little holes like these and getting rid of auto would surely at least double compile times or at worst make parsing C++ actually impossible.

That is what happens when a language makes the compromise to be copy-paste compatible with https://cdecl.org/

More to the point, it means the language continues to interpret header files from C libraries the same way that the C compiler always has.

It goes both ways, back in CFront days, and when it was picking up steam across MS-DOS, Mac OS, UNIX, among others.

Unfortunely it is also an hindrance to fix some issues that prevent C++ to ever embrace safety, or better implementation of static analysers.

Both which ways? C was adopting C++ features before C90. Void type, function prototypes, scoped struct member tags.

Declaration syntax is not an impediment to safety or to static analyzers, except insofar as it is trickier to code a parser for them.

It is in the sense that pointers everywhere cannot be prevented and it isn't always easy for a static analyser to actually make sense of how they are being used, as shown at some CppCon talks this year from Microsoft.

Declaration syntax has nothing to do with "pointers everywhere".

Yes, those things creep up in various places. Some are fixed, like when you have std::vector<std::vector<int>> where it used to give an error because it thought >> was an operator. But recently I discovered this one:

    int x[100] = {}; // just a regular array
    int val = x[[]{return 42;}()]; // compile error!

Once again, greedy parsing is the culprit; here it thinks [[ is the start of an attribute.

On one hand, nice catch... on the other hand... why do you want to do that??

Problem is, function parameter names are optional, so you can already have something like `[](x){}`, where `x` is a type.

As seen from Python vs Lua: without a keyword for introduction it's less clear which scope a variable belongs to. Scopes are part of deterministic destruction in C++ and should not be taken lightly.

As a general preference, I'd always want a keyword to introduce a new variable, and most languages seem to agree even if they didn't have to.

In this case x must obviously be in a new scope as all function parameters. There isn't possibly be an ambiguity here.

I think grandparent is referring to myLambda - that could be a new variable or overwriting of an existing one.

I agree with them, in Python the number of times something like this happens:

    myVariable = 10
    myVariablr = clever_function(myVariable)

(note the typo in variable name)

>> in Python the number of times something like this happens

Anec-data but I’ve been writing python for 15+ years, I’ve contributed to various popular open source projects. I’ve never seen this in a code review. I’ve certainly never seen this kind of mistake released.

I've seen a variation of this bug several times in just the last year:

    myVariable = 1
    if not something_unusual():
        myVariablr = 2
    return myVariablr

This code will work just fine until something_unusual() returns True and then it crashes with UnboundLocalError: local variable 'myVariablr' referenced before assignment

This really sucks when it happens during a long-running job. Say for example you're looping through an array in chunks and distributing each chunk across GPUs. The last chunk will have fewer elements than all the others, maybe even 1 element. If you don't pad the chunk, it will go through a different (and likely underutilized) code path downstream and crash like this.

Dynamic languages often benefit from some tool assistance. Both your and GPs experience can happen at the same time: GP is probably using a linter or an IDE (like Pycharm) which will point out that "myVariablr" may be undefined.

If you're returning myVariable instead, that's "fine" - it is always defined.

You might get a warning that myVariabl_r_ is unused, but that's also not a given. The bar for being 'used' is pretty low. For example, in this

  def main():
      x = 10
      l = []
      l.append(x)
      return 5

pyflakes believes both x and l are "used", even though neither affects function return value.

In Python even the trivial definition of "used" (i.e. "bound name which is referred to afterwards") is not that easy (e.g. getattr), linters go for the literal definition instead ("bound name, which is statically referred to afterwards"). This works for 99 % of code. Your definition of "used" (~"influences the observable result") is of course fully within the dominion of Rice's theorem, though this particular example would be caught by any data-flow analyzer.

Those type of errors will be caught by the compiler. It's definitely a problem for python-like languages (but not for compiled languages) because there's no compilation step involved before running the code.

It’s a compiled language in the same sense that Java is compiled. Most people use the CPython implementation which compiles .py files to .pyc files before executing them. That compiler can’t catch bugs of this form because it’s not always a bug. That’s just how dynamic languages work.

But if Python had a different syntax like “let var = value” for creating new variables rather than mutating existing ones, we wouldn’t have this specific problem. That’s what this thread is about.

If C++ inferred “auto” for all assignments/initializations, it would have a similar problem where typos in assignments effectively lead to that line of code getting skipped.

VSCode (pylance) & PyCharm both immediately identify myVariable as unused - this is inline in the editor, before any compilation or testing.

Assuming somehow this code made it into a pull request (although as above, there would be no reason for that) - then flake8 flags myVariable as unused, your CI pipeline wouldn't get as far as notifying a peer that the PR is ready for review.

Oh I didn't even realize that OP had removed auto from myVariable. Of course I strongly agree that auto or some other syntactic marker is needed there!

I had a friend show me some of the stuff you can do in C++ nowadays. I had no idea C++ had evolved so much!

   Generalised capture

    In C++11, lambdas can only capture existing objects in their scope:
    int z = 42;
    auto myLambda = [z](int x){ std::cout << x << '-' << z + 2 << '\n'; };
    ...
    But with the powerful generalised lambda capture, we can initialise captured values with about anything.
    int z = 42;
    auto myLambda = [y = z + 2](int x){ std::cout << x << '-' << y << '\n'; };

Am I the only one who doesn't see much of a difference here?

Generalized capture is primarily useful for move semantics:

   std::function<int()> make_func(std::unique_ptr<int> ptr) {
      return [p = std::move(ptr)]() { return *p; }
   }

Without generalized capture syntax, there's no good way to transfer ownership into the closure (a regular `[ptr]` capture would attempt to make a copy, and a by-ref `[&ptr]` capture would lead to a use-after-free).

i was in a leetcode BS interview a while ago with a googler who thought i could not write code in c++ because i could not remember lambda syntax. i only use maybe 30% of c++ features in my software and have been doing this for over 20 years. it’s only recently i started using lambdas more, but still stay away from them because of the potential hidden allocations.

What kind of hidden allocations?

That are used to hold captured variables and values.

Internally, lambdas are function pointers + the context, and the context may or may not be dynamically allocated depending on how lambdas are used.

Internally, lambdas are structs with the "function call" operator overload. Context is done via members of the struct: capture by-value, and the struct copies them and stores its own, capture by reference, and the struct member is a reference. There should be zero heap allocation in any case.

You capture by value a variable that allocates and you get a heap allocation.

That's dependent on the captured type's copy (or move) behavior, and is completely different from your claim that how the lambda is used can influence this. Lambdas themselves never heap allocate.

If I'm reading this right, then "lambda" in C++ isn't "lambda" in Scheme/CommonLisp, where captured variables need be heap allocated rather than stack allocated to construct closures?

You're right. Lambdas in C++ are syntactic sugar over something the core language has allowed you to do since before C++98. Since you control which variables you capture into the closure, and how, when writing the lambda, all of the information regarding how to make it (e.g. it's size, data members, etc) is static, and so it does not require the use of the heap at all. A concrete example. The following C++ examples do the same thing, and neither touch the heap:

    // A. Using a struct. The old manual way.
    struct my_lambda {
        my_lambda(int c) : c_(c) {}
        int operator() (int x) const {
            return x == c_;
        }
        int c_;
    };
    
    auto f(vector<int> numbers) {
        my_lambda lam(2);
        // remove all the matching elements
        erase_if(numbers, lam);
        return numbers;
    }

    // B. Using a lambda
    auto g(vector<int> numbers) {
        int c = 2;
        erase_if(numbers, [c](int x) { return x == c; });
        return numbers;
    }

It is complicated. In C++ closures can close over local variables either by value or by reference (the choice can be made for each variable closed over).

When closing over a variable by reference, if the lambda need to survive the local scope heap allocating the closure itself won't help. Instead you need to explicitly heap allocate the closed over variable itself (and close over the, usually smart, pointer).

When closing over by vale, there is no such issue, closed over variables are copied over along the lambda and it can be safely, for example, be returned from a function.

Copying might be expensive if the lambda is closing over an expensive to copy object, but move semantics are always an option.

Lambdas are value types, they are usually copied around. so when closing over ither va

People who think that lambdas may cause heap allocations are confusing lambdas with std::function.

Lambdas are essentially syntactic sugar for an anonymous struct holding references to/copies of captured variables and an operator() method with your code. Not 100% precise, but memory-wise that's how it works (and has to work). No dynamic memory allocation is involved here at all.

If you put a lambda (or any other callable) in a std::function, the std::function will copy that functor object, either into its own small-buffer-optimized in-place storage or (above a certain functor size) into heap-allocated memory.

There is no dynamic allocation at all, unless you assign the lambda to an std::function, in which case it _may_ require dynamic allocation if the size of the captured variables exceed the internal storage capacity (typically 32 bytes) of std::function. There are types (usually called 'function_view') floating around that don't dynamically allocate at all, and that can be used as a cheap alternative when the function is not stored for later use.

If you're not capturing variables that allocate memory (like std::vector or std::string), there will be no allocations. And if you're that sensitive about hidden allocations, chances are that you're going to avoid most of these STL types in the first place (no allocating constructors or RAII), so you won't probably need to worry about this. Even the people using C-like C++ ("Orthodox" C++) seem to use lambdas liberally when it make sense.

(Or maybe you're talking about putting a lambda into an std::function, which is a totally different thing and will probably incur heap allocation?)

I interview a lot of C++ developers who have 20 or more years of experience I feel really bad when the vast majority of them fail to keep up with modern standards or take the time to understand their tools resulting in all kinds of misconceptions based on outdated information.

Lambda expressions do not involve any kind of hidden allocations, their definition is precisely formalized and can be reviewed in S 7.5.5 of the standard. Even if you don't care to read the standard, there is no shortage of resources online that explain what a lambda expression is, and none of them involve anything to do with hidden allocations:

https://en.cppreference.com/w/cpp/language/lambda

I'm sorry to pick on you specifically, but it's a major problem that is entirely unnecessary. It's almost heartbreaking that the people who should have the most experience in a subject based on decades of knowledge are often the ones who carry the biggest misconceptions and spread the most misinformation.

Maybe the allocations I was referring to came from using std::function. My point was that if you are not careful when you write software in C++ and use modern features, you will get behaviour you don't want or didn't expect.

I have no time to read the standard or keep up with it. Maybe that is what people at Google can afford to do? I'm guessing that is where you work.

I run a small business, and I have to juggle a lot more than just writing code. When I have a problem, I figure out where it's coming from, and fix it. If you run your own business I'm sure you understand that.

Not keeping up with the standard has nothing to do with ignorance or refusal, but with constraints. I also don't think it should be required to know the standard, or the syntax for features you don't need to use everyday of the top of your head. If you are not sure, look it up. That's what the internet is for.

In any case, no need to be arrogant and dismiss people with experience as stupid or ignorant, or blaming them for spreading misinformation, simply because they don't know certain syntax by heart or don't keep up with the standard.

>My point was that if you are not careful when you write software in C++ and use modern features, you will get behaviour you don't want or didn't expect.

This is an unsubstantiated point and simply untrue. Modern features have worked hard to maintain what is referred to as "zero cost abstractions".

>If you run your own business I'm sure you understand that.

I do own and operate a business, a quant firm engaged in HFT. It's part of the reason why I hire so many C++ developers.

>Not keeping up with the standard has nothing to do with ignorance or refusal, but with constraints.

The ignorance comes from going into an interview where one would presumably be expected to have a basic understand of modern C++ functionality and then proceeding to call that interview BS when in fact it was you who simply had a poor understanding of what was expected.

If you don't want to use lambda expressions or you feel like using C++ like it's still the year 2003, so be it, that is your choice... just don't be surprised and go around calling interviewers full of BS when you continuously refuse to keep your understanding of technology up to date.

The problem in this case is not with the "BS interview" or the interviewer, the problem is with you. If you do not keep your understanding of C++ up to date, for whatever reason, then an interviewer is right to reject you for it in favor of someone who does take the time to keep their skills up-to-date.

Regarding returning lambdas, of course it could be done even in c++11 via std::function.

That is not really the same. `std::function` is a heavy-weight type-erased wrapper over callable types.

Unfortunately it’s the only way to get a recursive lambda, as far as I’m aware.

You can use a fix point combinator.

Problems with std::function:-

1. It can not hold move only callable objects.

2. It heap allocate stored callable object if the object is large enough.

3. Type-erased (its primary use case, but hinders optimization)

4. Can't be constexpr

Also, std::functions are not comparable. This makes it harder to use for registration/de-registration or an Observer architecture.

Can you point to better alternate ways or idioms?

It's possible to write something like `std::unique_function` (which uniquely owns the stored callable and can be moved but not copied). That's often preferable for storing functors.

Example: https://github.com/facebook/folly/blob/main/folly/docs/Funct...

stlab has a task type that works around a couple of these issues: https://stlab.cc/libraries/concurrency/task/task/

(joke) : Common Lisp

I think lambdas also heap allocate, at least on embedded C++11/14 projects I remember having to have a little of heap for them.

Is there any flag in some c++ compiler that forces to you to use the newer features? For example, forbidding non-auto declarations.

No.

There is tooling along those lines. I wouldn't expect to find a tool that forbids non-auto declarations, but there are certainly some that suggest when something could've been auto.

Right. clang-tidy lets you specify rules you’d like it to enforce in the code you pass through it. It’s not a part of the compilation process, but uses the same underlying driver (if you’re building with clang)

On large projects it can be integrated into the build pipeline, so in a sense it becomes part of the product compilation process, even if not on the local workstation.

I am a big friend to push good coding practices into the CI/CD configuration, regardless of the programming language.

Anyone can do whatever they feel like on their own computer, but the overall product has guidelines to follow, specially relevant in projects that live from changing consulting companies all the time.

Quite apart from not being possible forbidding non-auto declarations also sounds like a terrible idea. I am not really that fond of the auto keyword. In some cases it is necessary and/or helpful but it is also very helpful to be able to tell what the type of a variable is at a glance. I think the auto keyword should be used sparingly.

Reply on news.ycombinator.com