C++ vs. Rust: an async Thread-per-Core story | Hacker News Reader

C++ vs. Rust: an async Thread-per-Core story

medium.com

79 pointslukastyrychtr41 comments

What library are you using? The pattern you are trying to achieve should be achievable in principle. This pattern is commonly called "scoped" threads (or "scoped" async tasks in this case) in the community.

The problem might be that if the executor you are using doesn't guarantee in its type signature, that the task you are trying to launch doesn't outlive the lexical scope there, the compile gets rightfully angry. There are some executors that guarantee that; see here: https://docs.rs/async-scoped/0.4.1/async_scoped/

This article was remarkably well written and reasoned. I would welcome more.

The author is correct that C++'s greatest weakness, and one that can never be fixed, is that other code can do things that violate the model your code depends on. It is not enough to have your own discipline, the rest of the system needs to observe the same discipline at interfaces. And there are many possible, correct disciplines to choose from.

Rust imposes one of the possible disciplines for memory-object management that enforces one set of non-negotiable tradeoffs and costs. You the programmer are relieved from choosing a discipline or discovering what was chosen, and from discovering that two subtly different ones were used in different parts of the system.

Memory-object management is not the only place where systems need a consistent discipline, but others can often be referred to it; and every program needs at least the one, besides any others.

Arguably the reason Common Lisp never took off is that practically nothing can be assumed about any other part of a system. C++ has enough forced consistency to drive adoption, but might better have had more. C++ has a lot more tooling than Rust does to capture semantics in libraries, so the library is the natural boundary for a discipline, and it is not unusual to find more than one safely co-existing, but it is also not unusual to find them not safely encapsulated, and in conflict.

Ultimately there is no substitute for sound design. Rust's limits help ensure a consistent design, but the cost in abstraction power is high.

> Rust imposes one of the possible disciplines for memory-object management

Could you give some examples of other possible disciplines?

There are lots. For a single-threaded program, moving around unique_ptrs or, equivalently, value objects, suffices. For parallel execution, maybe you add threads and mutexes, or futures, and possibly shared_ptr or a moral equivalent.

The point is that you have a lot of choices to make that are only peripherally related to the problem the code is supposed to be solving. Those choices might allow you to tune the performance of operations better to the problem, but they may be just a distraction from the goal.

For most systems, nobody knows how fast it should be able to go, or whether you got close. Almost as often nobody even cares very much, or cares at all. When they don't, it is wasted effort to spend a lot of time and skull-sweat optimizing performance. There, a straitjacket language that offers few choices is best. It lets you get on to the next thing.

The thing is anything which can break your code implicitly in messed up ways like memory violations require unsafe code which if possible should be avoided and most important makes it easier to pinpoint what to review if you run into unexpected strange bugs.

Sure reviewing unsafe code isn't perfectly ready often it relies on invariants uphold outside of the unsafe blocks (in the same type).

Still much better then having to review everything ;=)

Personally, I've written a few multi-threading implementations at various places and my accumulated wisdom is that any data received by threads has to be passed by values or be constant. Anything else is a lot of risk for little value.

I've also come to the conclusion that async/future are a bad idea for multi-threading. They're fine for managing mono-thread asynchronous events, but using them as a poor man's thread pool is begging for trouble. Among the problems are:

   - you do not know if the execution will be synchronous or not.
   - you often don't know and have no control on how many threads will be spawned.
   - it makes following the flow of data and execution much harder.

What I've come to rely on is the trinity of:

   - a thread provider (non-committal name, we all know it's a thread pool behind the scene), which provides the threads of execution.
   - a work item provider, which provides the input data to the algorithm.
   - a worker provider, which provide workers that receive a work item, execute an algorithm and produces a result.

Then something gets executed when all three are united. For the curious, the reason to have a work item separate from worker is that it allows the worker to be long-lived and re-used and, most importantly, allows containing heavier data. For example, a cache. The work item are light and are just the necessary data needed to process one... work item.

I've also sometimes divided the work item into two parts: a work item and a context. The work item is the changing data to be processed while the context is the almost constant data, and can be shared. (For example, configuration data.)

I generally prefer to code one active thread per process, with multiple processes when I want parallel work. They communicate through shared memory, generally single-writer enforced by the OS.

So, yes, there can be equally rigorous system-level disciplines that substitute for language-imposed ones, and that can offer much better performance. But it does take a lot of experience to choose well.

That sounds a lot like how the linux kernel manages it with its workqueues data structure, which gets consumed by kworker.

> Rust imposes one of the possible disciplines for memory-object management

Until you load a shared library not written in Rust into the address space.

There should be a way to define an async executor API that works like Crossbeam's scoped threads API, so that the executor lives inside a fixed scope and is guaranteed to clean up any outstanding tasks before the end of that scope. Then it would be safe to accept references to longer-lived values on the stack. My guess is that there's not much demand for such an API outside of toy examples, though. (Does anyone use `crossbeam::scope` in production? It's a beautiful and important concept, but it seems to occupy an awkward performance point of "I care enough to worry about heap allocations, but not enough to use a thread pool.") EDIT: Ah, just saw GolDDranks' similar comment.

There are, but you often run into limitations in praxis.

For example you can take a look at tokio::LocalSet::{block_on,run_until} which require neither 'static nor Send or Sync. But then spawn_local does require 'static.

Same is true for smol where we have smol::block_on, smol::{Executor,LocalExecutor}::run and here too we have the limitation that calling spawn on a LocalExecutor requires static.

That’s an incredibly long-winded way to say “don’t think of reference counting as a (potentially-avoidable) cost; think of it as an investment.” Then you, too, can be happy, as long as you don’t think too much about it.

Sometimes the response you get from someone after telling them something and showing them something are different.

Those who already agree with the point probably think it long-winded. Those who would argue the point if told it probably appreciate the nuance the context provides.

Man, that c++ code triggers my hatred of lambdas. Instead of a hopefully descriptive function name, whose code I can peruse separately from trying to understand the enclosing code, I have to understand every goddamn line in real time.

lambda more than 5 lines should be a compile error.

Why not move C++ towards readability instead of enabling laziness in the guise of features ?

There's becoming a fashion to write fucking awful shite because it's functional so it must be written that way because it's good because it's functional. People need to get out of this habit now.

All awful shite is functional. All functional is good. So all awful shite is good?

I think that might be a tad reductive. Also, concluding that "people" must do anything now isn't useful. No one will be persuaded by your argument. People that are knowingly writing awful shite aren't bothered by that fact. People unknowningly writing awful shite will assume you're talking about other people.

IMO it's awful shite. Other people seem to think nesting lambdas n deep is not shite. I've worked with code like this - I've been lectured that "it's the functional way", but it isn't, it's just a mess. And often inefficient to boot.

You can write fine code functionally but for some reason this kind of horror is considered "the right way". Why? since when did people forget about writing for the human not the computer?

What is awful shite?

There's nothing about functional programming that requires nested lambdas. I like functional programming at least a little, but I don't like nested lambdas. If you want to know why someone considers it "right", you'd have to ask someone who considers it to be.

Strongly agree re: writing for humans. Actual computer architecture seems a lot closer to imperative with goto, so I'm not sure who would argue that functional code is "for computers".

> you'd have to ask someone who considers it to be [right]

I did say why - it's because it's "functional style" so it should be done that way. I've been told this to my face. Trouble is, it's plain wrong.

> so I'm not sure who would argue that functional code is "for computers".

Agreed. I should not have put that.

I don't think that's what they're saying - what I read (and experience sometimes...) is old C++ programmers that have only known imperative styles get access to functional techniques and don't see the problem sticking stream-lambda-filter-map "loops" within regular imperative for loops.

AKA quit giving C++ devs new obfuscation techniques...

For a lot of things, I actually like being able to read what the code is doing inline, rather than having to constantly jump around to function definitions.

It is a subjective thing, and I think the permissive approach of the language with regard to lambda complexity is a good thing.

The biggest thing that can be done to make that code example more readable is to use coroutines (which were just introduced in C++20) which will get rid of the lambdas.

100% agreed, except "5" should read "2"

Perhaps you may enjoy Python ;)

I'll disagree - a lambda should be very short indeed[0] else just define it as a standard func with a descriptive name[1]. And python's lambdas regrettably single expression only, no statements, which can really limit things.

[0] My opinion

[1] Exception is where you use lambdas for passing to new control flow structures. Something you can't do with python, although python does finally have a way to do the same thing (I think htis, though I'm a bit rusty so maybe something newer https://www.novixys.com/blog/proper-cleanup-resources-python...)

Lambdas are compiler friendly. They reliably inline unlike standalone functions. It's not laziness, it's an optimization.

C# has inner (local) functions which are close in semantics to lambdas but have names. I didn’t think I’d use them at first, but actually I use them all the time and often prefer them to raw lambdas.

  #include <cstdio>
  int main() {
    auto square = [](double x) -> double { return x*x; };
    printf("%f\n", square(10));
    return 0;
  }

since 2011

> They reliably inline unlike standalone functions.

No. They don't. Non-constexpr ones can't. And lambdas with captures that outlive their creators are notoriously painful for compilers and libraries to support.

Yet, they do. Lambdas with captures are explicitly modeled as a struct holding the captured values, and a member function that takes its implicit "this".

That function, with its implementation fully exposed, is as easily inlined as any other. The struct is moved or copied as needed to keep it live after the originating context evaporates.

I stand corrected. OTOH, "functionally equivalent but makes debugging or verification harder" isn't actually a strong argument in favor of lambdas.

Fortunately they don't need a strong argument. Convenience and minimized boilerplate suffices.

Do you hate the C++ syntax for lambdas or the concept of lambdas?

The concept of anonymous blocks of code in the middle of a flow annoys me. I fully appreciate that sometimes they're useful, but like everything else in C++ there is a way to misuse them horribly and you will run across code bases that do it eventually.

I hate the syntax for lambdas in general, but not the concept of them. But the same is true for several things in C++, like functors.

I don't dislike all lambdas. I just wrote some code that made pretty heavy use of lambdas, but they were always called synchronously within their creating function, as a way to reduce code duplication within that function. Sure, I could have defined a separate method, with a separate declaration in the header file, with lots of arguments, but that would have been considerably less readable.

What I often find problematic is lambdas that outlive their creators. Especially ones that have lots of captures. Especially especially ones where those captures include objects with complex destructors. It's a common pattern in the code I have to work with, and it constantly results in serious bugs. A far better pattern is to wrap the things that need to outlive the caller in their own object - often a "request" object that should have already existed and had its own separate lifetime - and pass that around instead of passing lambdas.

I'd be perfectly happy with lambdas, even lambdas with captures, that couldn't outlive their creator. Trying to attach one to an object with non-local scope would be a compiler error. We already have objects to wrap up the same bits of data and control flow that a long-lived lambda would have. People should use them.

Or, you know, actual garbage collection (generational, JVM-style)

…which has its own issues, of course.

If you compare Cassandra and ScyllaDB, the garbage collection causes serious problem with 99.9% latency.

Only because Java is yet to support value types.

In languages like Nim and D, where all C++ allocation mechanisms are available alongside a GC heap, there are plenty of ways to improve performance.

The whole point of scylladb, which the author has extensively worked on, is that we can be a lot faster without garbage collection.

"Slow is Smooth, Smooth is Fast"[1]

Especially when the lifecycle cost of the corner case debugging is folded in.

[1] https://sites.google.com/site/lehijujitsu/slow-is-smooth-smo...

Reply on news.ycombinator.com