Coroutines are now in Clang trunk, libc++
twitter.com
twitter.com
Don't use it as I used to, but the improvements in the last years, specially after the introduction of AST libraries via clang, have done wonders to the eco-system.
Now we just need C++ IDEs to catch up with Energize C++ and VA C++ v4, on their Lisp/Smalltalk like experience.
Borland C++ for MS-DOS had pre-processor macros for generic data structures, back when ANSI was still trying to figure out how templates should look like.
On those days, Java or Oberon derivatives as alternative made sense, because templates weren't even a workable feature in 99% of C++ compilers, so most of us didn't had real exposure to them.
As someone who rarely dabbles in C/C++, but still has some CMake under his belt... What's wrong with Cmake?
I must admit for the use-cases I had, I found it quite a improvement over regular make.
As for criticisms of CMake, when I used it I thought it worked pretty well but I think its CMakeLists syntax is atrocious. Not that it's not an improvement over the autotools, but at least these ones have the excuse of being built on top of a mountain of legacy subsystems with various quirks.
I'd want it to look like a standard scripting language of some sort.
Granted, I've been moving more towards scons lately, as python is a much easier language to work with. My usual SConstruct will check whether I am making a python extension module, and will download pybind11 if I am. That sort of thing is still possible with make, but starts getting more painful.
If you aren't using globbing but are specifying source (and header files) explicitly, just running make will do the right thing. Just running make also checks if you have modified CMakeLists.txt and re-runs config generation.
In my mind, anything that needs to be specified explicitly is an opportunity for error.
With manually-specified source file lists, using the Ninja, GNU make, NMake and Visual Studio generators, I have had to re-run cmake by hand no more than a few times, each time seemingly due to a bug somewhere. (Possibly my fault... I can't remember having had to do this with my current project.) Aside from that it's good at detecting when relevant files have changed, CMakeLists.txt included, and reinvoking itself as part of the build.
CMake has plenty of horrid parts, and there's probably plenty of reasonable workflows that it doesn't support, but I have to say that I've found the day-to-day workflow perfectly satisfactory...
Advantages: Good Windows support.
Disadvantages: Dictates the directory structure, much less flexible than autotools.
If you know shell, an existing autotools project can be modified easily.
If you want to do something special in cmake, first you need to go to Stackoverflow. If your are lucky, the thing you want to do is supported (often it is not).
All in all, I feel locked in by cmake - to the point that once the build works I'm less inclined to refactor because the directory structure cannot be changed easily.
One line summary:
set(MYVAR 1234 CACHE FORCE PARENT_SCOPE) # Sets the MYVAR variable in the cache
set(MYVAR 1234 CACHE STRING "" FORCE)
# Sets MYVAR in the scope above the current level
set(MYVAR 1234 PARENT_SCOPE
set(<variable> <value> [[CACHE <type> <docstring> [FORCE]] | PARENT_SCOPE])
Yes, by no means exhaustive!
Acknowledging Scott Meyers is a great author, I'd say something morbid happened with this book. It feels that Scott simply bashes the language more than it introduces it - while he's obviously very knowledgeable and in the details he spots out glitches, pitfalls, death traps, and WTFs and keeps throwing these waves of "here be dragons" at you, that half way through I just quit reading it and quit attempting refreshing my C++ knowledge after my 15 year break up from it. You should get Stroustrups' "A Tour of C++" - it is a softer, more cheerful and proud book that'll motivate you (and god knows you need motivation with C++).
While everything he says is true, and I don't want to say "it's better not to know these things", since then half a year passed and I gave modern C++ another try _without_ this morbidness to follow my attempt. I've been happy following "soft" intros into modern C++, without the bummer parts. Truth be told I haven't bumped into what Scott has kept highlighting - yet.
So I would say Effective Modern C++ is for very experienced C++ developers with fresh memory of how the internals work.
It's a disaster to try and understand unless you are already starting from a very strong point in your C++ knowledge.
This matches my experience too. I regret buying them.
The series is not for those who were used to, but subsequently lost touch with, c++03 and earlier.
I found it better to learn the modern version by 'porting' earlier C++ code to use modern idioms.
- lack of a proper, cross platform, package manager: I know the history, I know that C/C++ are basically embedded in the system and they use the system package manager, but in this age it just doesn't fell right... I've been spoiled by many years of using Maven and other similar tools
- template meta-programming plus the C++ legacy features which are probably lurking in many third-party libraries (simpler languages have fewer features to abuse and even when for whatever reason you have to use a relatively subpart third-party library, you know they can't do that much damage)
There's also the compilation speed, but as far as I know there are workarounds for that.
These things are, if not unsolvable in C++, at least many years away from being available.
Maybe an experienced C++ coder can destroy my preconceptions :)
Royal you and we below, not specifically hellofunk or op.
You use the libraries, so your application uses the feature already. You should understand one layer of code beneath yours. You should have an interface layer between your code and the library. By not doing either ( because libraries use magical features ) you are coding yourself into a corner with a feature you don't understand.
The libraries use it so that their code can be more general and well-specified. The libraries are used by thousnands more developers than your code. Your code probably could also be easier to maintain if it was more general and well-specified. Imagine if thousands of people could reuse your authentication interface at your company! Think of the money you will save.
All it takes is time to learn a feature you must partially understand because you are using the library built ON that feature.
EITHER learn the feature, or use libraries that only use features you understand. If you can't use the language without the feature, you can't use the language without understanding the feature. If you allow libraries that use the feature in your app code, you should allow app code that uses the same feature.
Are you saying I should not use such a library unless I understand all the techniques it uses? I doubt a significant portion of developers fall into that category of library users.
A linked list is a perfect example. If you use a linked list, you should know the basic properties of a linked list. When given one from an unfamiliar library, you should go read its code. You should run some basic performance tests on it. You should separate it from your code via some basic delegating api, so that you can replace it with another implementation in the future.
And concurrency is another example - do you need, and does, your chosen library give you parallel execution, or merely asynchronous execution.
This isn't as onerous as it sounds -- you do it all the time. But blindly trusting any software without a proof of its correctness (encryption is an example of where you might not understand ecen the interface layet of the software, but use it directly anyway) is a dangerous habit.
If there's magic there that you don't understand and you choose the library, when that magic fails to satisfy some goal you need to understand why so that you can fix it, implement your own workaround, or swap it out with a library that does what you want.
If you use a concurrency library, and don't understand threadpools, block in a thread and threadlock your application, you are in trouble.
If you use IEnumerable, and don't filter out the results before you toList, you are in trouble.
If you use monads and think flatmap is stack safe, always, you are in trouble.
Note you don't need to know ALL the tricks. Just the ones employed by the functions you call.
As a user, I need to understand the top level abstraction of the library I am using - the api. I need to read that api's code and understand what it does, (not any deeper) in order to use it.
If it is a low-level api that directly implements a thread-pool, I need to understand that. If it is a high-level template that implements the async continuation monad[1], I need to understand the api surface that I call, and just the body of that surface and type signature.
1: https://www.fpcomplete.com/blog/2012/06/asynchronous-api-in-...
- Generally C++ uses the OS-level package manager, like rpm or dpkg. It's not cross-platform and it doesn't have a nice interface in the source code (#include is... unfortunate), but there is a nice solution here if your organization has the discipline and tooling expertise to make it work.
- Template metaprogramming isn't bad to use at all. Everyone uses it every time they construct a vector<>. It takes a lot of skill to write, though, and unfortunately even users of TMP can end up with nasty and bloated compiler errors. Hopefully this will be addressed by "concepts" in forthcoming versions of C++, but it might be a while before that filters down to stable C++ compilers and libraries.
- The real issue that should scare you is something called "undefined behavior". It's very easy to do subtle things that cause you programs to break in weird and hard to debug ways. Nasty side effects that could theoretically include any side effect. Like saving garbage to your production database. By doing something as innocuous as overrunning an array.
https://kukuruku.co/post/undefined-behavior-and-fermats-last...
https://stackoverflow.com/questions/367633/what-are-all-the-...
There are tricks and tools to avoid undefined behavior, but I find that most C++ engineering orgs don't prioritize the tooling (linters, UB sanitizers, etc.) needed to let engineers do this productively. Consequently, a lot of writing good C++ involves detailed coding standards and/or very detail-oriented gurus poring over code reviews.
Realistically, C++ isn't actually a well-defined language unless you specify platform and language. You don't program in C++. You program in a C++-derived language jointly defined by the compiler and the hardware.
Had a fun bug at work once where we somehow accidentally built against two subtly incompatible versions of the same library.
This actually mostly worked until some random virtual function call sometimes called the wrong function silently...
The things that concerned me about C++ were the pitfalls that could happen due to ignorance or omission vs. errors introduced by commission. For example forgetting to implement a copy constructor for a class that allocates member data. (That results in two objects that hold pointers to the allocated data following a copy operation and if one is deleted, the other points to now deleted data.) Perhaps there are better compiler warnings or static analysis tools that will reveal problems like that today.
My other concern is how big the language has grown. I recall my excitement when I first worked with C++ since it provided direct support for a lot of things I was doing with C (Object orientation, function pointers in structs and similar) with the addition of better type safety. Nowadays many of the language features are incomprehensible and beyond my skill level. I suppose largely because I don't work with C++ on a day to day basis.
This is just the tip of the iceberg. There are many dark corners, historic baggage, and complex interplay of language features in C++. This complexity breeds bad codebases plagued by subtle bugs created by misunderstanding of finer points of the language.
If these things scare you, run away in the other direction as fast as possible.
The thing I love about using C++ is that you get direct access to all these rock-solid libraries in C that everything is built on -- zlib, libcurl, etc. So instead of dealing with a tower of wrappers in a high-level language, each of which has quirks and bugs, you get that core lib, where you can safely assume 99% of the time any problem is your own fault. Even when a C++ wrapper exists, I use the C API just to reduce deps and avoid the kind of problems you're talking about, which I think boil down to templates most of the time. I don't use templates unless there is no other way.
This strategy works pretty well for me, but my codebase isn't huge and I'm not doing anything super complex -- just trying to speed up things that are too slow in Python, be it parsing or data structures. I think in practice it is best to use multiple technologies if possible in a project, each one suited to the task.
And WRT the coroutines in OP, I don't know the details, but the thing I miss most from Python in C++ is yield and generators, which hopefully coroutines will eventually help with. Defining your own iterators is very annoying.
> in C that everything is built on -- zlib, libcurl, etc.
> So instead of dealing with a tower of wrappers in a
> high-level language
That's what I love about Objective-C: you get that, the direct access, and at the same time transparent, non-wrappered access to a high-level, dynamic object-oriented language.
Side Note: Objective-C is such a criminally underrated language. I often see people complaining about the syntax, but it's just syntax. Once you get over it, I find that the language makes the OOP paradigm a joy to work with.
But after looking at its feature set, I wonder how easy it is to get by without templates? It seemingly uses weak typing instead like Go for scenarios that would normally require generics/templates. My attitude towards templates/generics is that they are absolutely necessary for a good standard library and core containers, but implementation of new templates in your own code should be rare.
Also, I do like namespaces in general, although not necessarily C++'s implementation of them.
Programming languages have advanced a fair bit since the 80s, it's time for Objective-C to die peacefully.
One thing that is much more reliable in pure C++ is any kind of threading or OpenMP etc. Theoretically Cython has it, in practice it can cause very weird problems. Also if you want to use C (or C++) libs in Cython, you have to manually declare all the function prototypes before using them.
Also Cython has a tendency to make minor breaking changes to syntax every version, breaking my code and/or making it version-dependent. Since I distribute to others, the best method for linking performance intensive-code to Python I have found is:
1. Write the core in C++
2. extern "C" a simple C API for it
3. Compile both to a SO
4. Access the SO from ctypes
It is more robust than Cython IME. Another advantage of this approach is that you have now have a generic C library you can use anywhere else if you want, not just Python. And you don't have to link -lpython so the C lib can be used in places where Python isn't installed. Finally, it can be nice to have some C++ binaries for startup time, and AFAIK you can't compile binaries with setuptools.
Disclaimer: I'm a contributor to pybind11.
In short, I simply think C++ can be used responsibly if you limit yourself to the subset of features that you (or your team) can understand.
For example, there are people who go crazy with Python metaclasses and such. A statement about interacting language features being dangerous could apply to anyone in any language who is trying to be unnecessarily clever.
I think the problem is deeper than that and that the problem could be that C++ does not have a standard ABI. It is compiler-specific. Therefore, having a system package manager does not work (as it does for C).
I compile my personal C++ codebase with four different compilers and different debug settings. The only way to manage that is to have a monorepo with all my dependencies checked in .
For example here is a test that verifies the optimized output for a simple (and misnamed) integer generator compiled at -O2. Notice the arguments to `print` are integer literals. (https://github.com/GorNishanov/clang/blob/Experiments/test/C...)
I'm no python expert, but I suspect C++ optimizes this code better.
(Even more, I love how we're seeing some convergence here: no-compromise improvements to weak spots on each "side" of the programming language spectrum.)
This one is the most apparent. Almost every example in the C++ v. Python in the article above the C++ version is notably uglier. Just look at Lambda expressions:
Python
def adder(amount):
return lambda x: x + amount
auto adder(int amount) {
return [=](int x){ return x + amount; };
}
Edit: turns out there are various forms of Lambda (of course there wouldn't just be one in C++), for ex:
[a, &b], [this], [=], [&], []
There have been discussions about adding a lighter syntax for single expression lambdas that doesn't require the curly brackets and the 'return' keyword, but so far no actual progress.
It's much better specified, gives you much more power than the Python one, tells you exactly what is happening, is so short it's cryptic, and is ugly as hell.
https://github.com/llvm-mirror/libcxx/blob/master/include/op...
Looking at this one small syntactic difference explains a lot about why C++ is as ugly as it is. It has to pack in a lot of complexity everywhere, even at the most basic level, such as how to label a lambda expression. The ugliness may simply be a cost of offering much more power (simplicity is difficult/impossible when you need to support complex base abstractions). But it's still ugly. And therefore a big reason why it's not very Pythonic as the OP said.
def f():
x = 0
def add(amount):
x += amount # UnboundLocalError: local variable 'x' referenced before assignment
add(10)
add(20)
add(30)
return x
print f()
I'm not making a good/bad argument that one is better than the other. The two languages have widely different usecases and rationale in the language design. But I'm not convinced C++ is (or can become) very Pythonic given the complexity it must support at all the basic levels.
def f():
x = [0]
def add(amt):
x[0] += amt
add(10)
return x[0]C++11 release date: 12 August 2011
The stuff in square brackets is the capture list. Unlike python, which implicitly captures the full outer scope by reference, C++ allows/requires you to specify which outer variables you want to reference in the lambda, and whether they are captured by value ("=") or reference ("&"). This allows C++ to implement the lambda with a simple static function in most cases, where python needs a bunch of tracking data to preserve the outer stack from collection.
The rest of it is all just following the existing syntax of the language: C++ uses curly braces where python uses bare expression. C++ is typed and requires an "int" declaration on the argument. C++ puts a semicolon at the end of its expressions. You don't have to like this per se, and you're welcome to call it ugly if you want. You're just 30-ish years late to the argument, though.
I'm not complaining. Just pointing out how it's notably uglier, a critical part of being 'Pythonic', which affects both aesthetics (emotional experience) and functionality. That's not a value judgement but an observation.
Python is obviously more limited in power (functionality), which is a product of decisions made about base abstractions. That filters upwards and becomes apparent in both visuals and utility (and as a result learning curves, developer happiness, applicability to problem sets, etc).
Simplicity and accessibility are extremely challenging things to support when you have a product that does everything for everyone.
The need for 5 different lambda expressions is a perfect example of that.
I personally think C++ still has a role in the world, as it clearly has lots of adoption. But if you want non-Ugly then those various use-cases have to be broken up into single-purpose (or niche) languages such as Rust for systems programming and Python for 'scripting' and Lua for glue/config code. All of those languages listed have made hard tradeoffs to be good at certain things. C++ and Java did not - largely because they are "yes" languages, as Steve Yegge calls it (the language designers said yes to everyone early on).
If Ugly languages were merely 'bad' or 'wrong' for being ugly then they both would be the most used languages. They clearly have long-lasting value in the industry - which people like Yegge believe is because they are okay with being ugly.
This isn't inherently about capture lists; Rust doesn't use capture lists, yet ends up doing the same thing as C++ here.
Rust implements lambdas the same way as C++, yet it doesn't need capture lists. It has two modes: by default it tries to guess whether to take each variable by move/copy or by reference, but you can specify 'move' on a lambda to have it move/copy all mentioned variables. Not as flexible, right? Actually, it's equivalent in power, because if you want to "capture a variable by reference" in a 'move' lambda, you can manually assign a reference (pointer) to it to a new variable, and move that. With Rust's syntax, the new variable can even have the same name as the original, so it looks very natural:
{
let x = &x;
foo(move || bar(x));
}
Like C++, Rust uses semicolons, but it largely unifies expressions with statements. For example, the following are equivalent:
foo(bar(42));
foo({ let x = 42; bar(x) })
Rust is statically typed, but it has type inference, so - among other things - you can usually omit the types of lambda arguments.
So what does the adder example actually look like in Rust? With soon-to-be-stable 'impl Trait' syntax, like this:
fn adder(amount: u32) -> impl Fn(u32) -> u32 {
move |x| x + amount
}
https://doc.rust-lang.org/book/closures.html
I'm not expert enough to decide which is "official". But honestly... this is really not a situation where Rust shines aesthetically.
Edit to clarify (to the extent of my rust understanding anyway): C++ didn't want to implicitly suck in external variables by reference and makes the programmer explicitly flag them as references. Rust has a borrow checker, so it can skip this part and simplify the syntax. But Rust's type system has no good way to implicitly figure out the full type of a lambda by its signature, so if you want to pass one around you need to specify its whole type signature twice, once in the expression itself and then again in the signature of the function that will receive or return it.
You pays your money and you makes your choice. Neither language is really a good fit for anonymous functions in the sense that Lisp was.
> But Rust's type system has no good way to implicitly figure out the full type of a lambda by its signature, so if you want to pass one around you need to specify its whole type signature twice, once in the expression itself and then again in the signature of the function that will receive or return it.
Rust has no problem figuring out the type of the lambda. But the type of the lambda is anonymous, and unique to that particular lambda. That type is guaranteed to implement the Fn trait, so that's what you specify when you need to return it. You don't need to specify it twice, however - I'm not sure how that follows from the snippet above?
And C++ has the same exact problem: C++ lambdas also each have their own unique type. However, C++ doesn't have traits at all. In C++, if you want to return a lambda, you'd use std::function, which is roughly analogous to Rust's Box<> for functions in this case - you can't just return the lambda directly (well, you can, but only in a context where the return type is implicit and can be inferred from usage - e.g. when returning from another lambda).
IMO, it's 'ugly' not because "[=]" is inherently syntactically ugly, but because of all the variations of capture lists and the subtlety of the differences.
> but that extra impl clause is... fine?
It's not great - in fact, 'impl Trait' is a pretty subtle feature as well. But I think it's more principled.
> would you at least agree that (relative to python) both language have significantly more verbose syntax
Yes…
> owing to the interaction between closures and the languages data models?
…Not really. I'll elaborate later in the post.
> so if you want to pass one around you need to specify its whole type signature twice, once in the expression itself and then again in the signature of the function that will receive or return it.
No, you don't. Generally speaking, it needs to be specified at most once, sometimes zero times. In my example:
fn adder(amount: u32) -> impl Fn(u32) -> u32 {
move |x| x + amount
}
The caller of `adder` generally wouldn't need to specify the type:
let a = adder(1);
println!("{}", a(2)); // 4
When would it have to be specified zero times? Well, for one case, if there aren't any function boundaries involved:
let amount = 1;
let adder = |x| x + amount;
println!("{}", adder(amount));
let v = Vec::from_iter(vec![2, 3, 4].iter().map(|x| x + 2)));
println!("{:?}", v);
FWIW, not doing type inference across across function boundaries is a semi-artificial limitation. Haskell can do it just fine despite being statically typed, and although Haskell has very different implementation strategies, that's not related to type inference. C++ now sort of does it with `auto`, but only does 'forward reasoning' - i.e. it can deduce the type of an operation's result from the types of its operands, but not vice versa. Rust has true type inference, but it eschews global type inference (across functions) mainly because of a tendency to cause hairy/confusing errors in larger programs. (There are also compilation time concerns, but they're not the primary reason AFAIK.) I suppose you can say that dynamically typed languages are easier to understand when they go wrong, compared to hairy Haskell errors - but I'm not sure to what extent that's actually true, as opposed to functional programmers just tending to use more complex types.
By the way, C++(14) actually doesn't require the lambda argument type to be specified in this case. This works fine:
auto adder(int amount) {
return [=](auto x){ return x + amount; };
}
Well, in this way. "impl Trait", which was just accepted for stabilization, will let you return unboxed closures.
I'd argue that C++ is more pythonic than Python. For instance:
> Explicit is better than implicit.
Since when is dynamic typing more explicit than static typing ?
> Errors should never pass silently.
Well guess what, with C++ they don't even pass compilation.
"Dynamic" doesnt mean hidden. It means types are features of objects at run time.
"Static" doest mean explicit. It means types are rules the compiler applies to the program text.
> Well guess what, with C++ they don't even pass compilation.
int * a;
*a = 5;
Segmentation fault. Oopsy.
To be fair, "g++ -Wall" does complain that "a" is not initialized. But it is a warning, not an error.
(And don't tell me that "you are not supposed to use pointers". They exist, someone is going to use them and proceed to shoot themselves in the foot.).
1) Use preemptive system threads that can execute in parallel. A task requiring simultaneous waiting is given an operating system thread of its own so it can block without stopping the entire program. But threads require significant memory and other resources per thread. Also, the operating system can arbitrarily interleave the execution of system threads, requiring the programmer to carefully protect shared resources with locks and condition variables, which is exceedingly error-prone.
2) Have a single-threaded program, where that single thread runs an event loop whose job is to react to external events by invoking a callback function that has been registered. While it doesn't require the same kind of complex synchronization that preemptive threads do, the inverted control structure of this approach requires your own control flow to thread awkwardly through the system's event loop, leading to a maze of event callbacks.
Coroutines work with the latter style in an attempt to tame its complexity. They are themselves complex, however, and, in my opinion, that complexity doesn't pull its weight when you consider in the end you only have one thread.
Can anyone tell me what I'm missing or how these problems with these approaches have been solved in places?
This is the general architecture of nginx for example (though it doesn't use coroutines, just callback based async io).
This of course requires extra effort to prevent blocking of OS threads and in the case where the OS expects thread locality. But IMHO it's the superior approach.
The best solution would be if OSes themselves would provide really lightweight threads. And I suspect that they will, eventually.
http://www.phoenixframework.org/blog/the-road-to-2-million-w...
The limitation is actually on a quad tuple: {src_ip, src_port, dest_ip, dest_port}
As such, the actual limit is outrageously high and isn't really a problem. Interestingly enough, the referenced Phoenix benchmark overlooked the dest component and hit a limit as they had only 40 or so machines to use in the test. Had they used multiple ips on the server it would have given them a lot more breathing room.
EDIT: Multiple IPs != Multiple NICs. You can assign multiple IPs to a single physical interface. Also, they could have used multiple ports to accomplish the same effect.
So they were limited by the testing cluster or the tested server?
For what it's worth, Win32 has offered a kind of green thread (fibers) for a very long time. Adoption has been minimal, even for other Microsoft products (e.g. .NET doesn't really support them properly).
As for the number of threads, when I use Boost ASIO with coroutines I often end up with multiple threads servicing a pool of coroutines, so if there is shared state between them then there is still synchronisation. I use channels implemented on top of eventfd to help with that.
When I converted some network code from callbacks to coroutines the code ended up about 1/3 as long and was far simpler and easier to understand. It also fixed several bugs I simply couldn't find.
The reality is that the callbacks are far more complex to use than the coroutines.
You can write concurrent code with pthreads. It may happen to execute in parallel but it doesn't have to. In fact, I believe the early SunOS/Solaris pthreads had a 'green' scheduler where it could all be done in userspace (this is from my own memory nearly 20 years ago). This made them basically "stack switchers" which is what some lightweight coroutine libraries end up doing.
Having written heavily callback-driven code. Threading contexts manually, thinking about object lifetimes, I can say that coroutines have a much cleaner flow to them and can make it easier to write and later read and debug programs than crazy callbacks and contexts.
Object lifetimes are one thing coroutines help with in a big way. I think it's why Rust is so attractive to so many.
Look at plan 9's libthread for example: https://github.com/9fans/plan9port/tree/master/src/libthread
It allows one to take advantage of nonblocking sockets as a single "kernel execution entity" mechanism across many userspace threads.
This approach works pretty well. The Mongrel2 webserver clearly borrows this approach in their task library:
https://github.com/9fans/plan9port/blob/master/src/libthread...
https://github.com/mongrel2/mongrel2/blob/master/src/task/38...
This is the biggest reason I vastly prefer using coroutines in my Javascript projects (by way of `redux-saga` if there's heavy lifting, or just async/await otherwise).
> Concurrency is about how you organize a solution really more than anything.
You hit the nail on the head here. IMHO, business logic should be expressed procedurally, whereas driver code in an event-loop driven environment (like V8) is more suited to a functional style. Coroutines let you do both, quite cleanly.
Thanks for the links, btw, I've been wanting to dig through other code like this in more established languages. :)
I needed to read some bytes, make a decision, read some more bytes, make another decision, etc. This started out as callback hell since every read was asynchronous, which was very painful, but after using the coroutine support I was able to write the code as if it was synchronous and the logic became a lot easier to follow.
And the coroutines weren't that hard to use, even lacking proper language support. I could basically write something like async_read(socket, buffer, yield_context);. The coroutine yielded behind the scenes and the buffer was populated once control flow was returned to the next line (from who knows where!)
Single Thread: 1 thread of execution, 1 execution stack
Coroutine: 1 thread of execution, multiple execution stacks
Multi threading: multiple threads of execution, multiple execution stacks
Coroutines are distinct from threads as they normally lack a scheduler (the decision to which coroutine run next is either made statically ot explicitly by the coroutine); coroutines meant to be used on top of an userspace scheduler are often called fibers (or simply userspace, or green, threads).
Imperative logic is equivalent to a state machine but again much easier for humans to reason about, that's why developers got so much more done in the past with synchronous execution and macros than they do today as we struggle to build even the simplest user interfaces. I'm hopeful that we can get all of the benefits of 1980s cooperative threading but that the decades of experience since allow us to avoid the pitfalls.
Co-routines and callback based programming are great for dealing with I/O and for offloading "work" to black boxes, those black boxes can very well be other threads, CPU cores, hardware, or another computer.
When the work is complete, your callback is called, or the co-routine continues.
It is perfectly possible to be juggling multiple tasks this way, but have all the results appear in sequential order on one thread, and not have to deal with the complexity of real multi-threading.
This only works for tasks that are easily offloadable, you need VERY clean data boundaries, minimal shared state, and well defined points where you hand off control.
In other words, a well engineered program. :)
For example, you could have coroutines that write to a shared resource (such as a socket) where interleaving the data from two sources would be a problem.
Choosing an approach in this field is not always obvious, though sometimes it is. E.g. many stupid server workloads are bound on I/O concurrency, so green threads are the best fit.
However, as soon as CPU scaling is required, multiple independently schedulable kernel tasks are required -- you either have to fork/CreateProcess or spawn threads. This does not necessarily imply that larger parts of the application have to be aware of this. E.g. it's entirely possible to plug a behind-the-scenes CPU thread into an event loop.
The Future/Task abstractions that are found in multithreaded programming languages (like C# or Java) also reflect this possibilities: E.g. they allow to specify on which thread a continuation should be run instead of just assuming the current thread. They also need synchronisation internally, because multiple threads might want to use the Future object.
From my personal experience the things that work well are pure singlethreaded models with an async style (like in Javascript/node.js) or multithreaded but pure synchronous models without callbacks (as popular in Go). The mixture (multithreading with callbacks, which might be invoked on any thread) often leads to confusion how an API should be used correctly.
https://github.com/wingo/fibers/wiki/Manual
Really neat, and the implementation is really a testament to the power of delimited continuations.
Not sure it can't be done, but it would require a good design. Personally I think that a libdispatch approach works better for C, particularly with the blocks extension.
[0] https://www.slideshare.net/SergeyPlatonov/gor-nishanov-c-cor... Interesting to note, slide #11 uses a tokenizer to demonstrate the usefulness of coroutines. IIRC, Rob Pike used a very similar (maybe his was a lexer/parser?) example in '14 re: Go.
[1] http://luncliff.postach.io/post/exploring-msvc-coroutine
[2] https://channel9.msdn.com/events/CPP/CppCon-2016/CppCon-2016...
[3]https://paoloseverini.wordpress.com/2015/03/06/stackless-cor...
--
Side-note : Love it or hate it, the C++ community is certainly moving at a vibrant pace. I don't write production C++ code anymore (and haven't for a long time) but I still go out of my way to watch the CppCon talks. IMO, they consistently produce best quality talks just because you have so many excellent programmers from various industries using select subsets of C++ in all sorts of different ways. The people writing games are focused on making sure they're portable 'enough' to hit all the major platforms, without getting too tied to platform-specific low-level calls, while retaining the necessary performance to get satisfactory models/meshes/lighting/collision detection/raytracing/dozens-of-other-things completed within that tiny 16.33 millisecond gap to complete the framebuffer and swap in the next frame. The academics who are doing vast numerical computations will be talking about their new MPI utilizations (and undoubtedly, next years talk will be re: a boatload of RDMA/NUMA optimizations). People complain that C++ is a mishmash of too many concepts (pun not intended). I.e., you can write it in the "C with Objects" style, or the "I use TMP so much my code is basically Haskell", and anywhere in between - but it's that heterogeneity that ends up yielding such high caliber talks.
I agree. Scala gets the same kind of criticism, but just like with C++, you can easily limit Scala to whatever subset suits you (e.g., pure OOP or pure FP). C++ and Scala are among my favorite languages to work with, so I might be slightly biased :P
Even if they kept the core simple, the complexity was shovelled into the libraries instead.
For example, I doubt there is anyone able to know the complete Python 3.6.1 documentation by heart, plus all major libraries, tools and implementations. Yet it is touted as a simple language.
(1). The language itself. (2). The tooling (IDE, build system, linters etc...). (3). The standard library. (4). The ecosystem of third-party libraries. (5). The developer's application code.
Moving complexity away from (5) might make applications smaller and easier to maintain, but it is done at the risk of increasing the training / learning burden on the development team.
If too much complexity ends up in (1), then we create a barrier to entry for new developers and hiring becomes harder.
If too much complexity ends up in (4), then we end up with endemic not-invented-here syndrome, because it becomes too hard to learn new libraries.
It seems to me that a lot of this is driven by learning and learnability -- any educational specialists around who want to wade in on this discussion?
Yes knowing how to play scales and reading notes might seem easy, but mastering an instrument and its variations (e.g. string instruments) takes a whole life.
Sigh. Go figure.
This means making use of library types for memory allocation, vectors and strings.
There is a place for old C style low level features, but those should come later, after the safer concepts are already ingrained.
Kate Gregory's “Stop Teaching C" talk at CppCon 2015 explains better than me.
https://www.youtube.com/watch?v=YnWhqhNdYyk
Get a nice IDE, QtCreator, Clion, VC++ Community, KDevelop, XCode.
Besides the "Tour of C++", check Stephan T. Lavavej talks at Channel 9.
"Core C++ Lectures"
https://channel9.msdn.com/Series/C9-Lectures-Stephan-T-Lavav...
"Advanced STL", these ones only when you already have a good understanding of STL as user
https://channel9.msdn.com/Series/C9-Lectures-Stephan-T-Lavav...
Also check video presentations from NDC, ACCU, CppCon, CppNow and C9 Going Native.
> First of all, and most important one, learn C++ idioms, not C.
I think it would be better to learn both. C programming really has its place, and it gives you a model to understand how the sausage is really made. I know lots of C++ programmers who couldn't even begin to understand how half the things in their language work, this leads to completely needless performance loss (like using vectors for statically-sized arrays of data, which I recently replaced with a C array in a piece of game modelling software for a 20-30% improvement in runtime).
The fact that you consider this the most important one is outlandish. I think it's easily more important to teach the tradeoffs and models involved in things like unique_ptr, rather than creating a cargo cult for abstractions that turn out to be useless half the time.
C++ programmers really need to learn C, at least after their first year of blissful ignorance. Don't stop teaching C.
C style code has only one place, unsafe C++ coding for optimization purposes, only after validated through the use of a profiler.
In 99% of the applications it is usually irrelevant for what the users are expecting, and there are even some nice talks at Nokia Wrocław conferences, where the presenter was able to generate better compacter code with modern C++ instead of classical C code.
It was the C mentality brought into C++, back when learning C before C++ was the only path as C compilers were adding C++ support, that brought the daily CVEs we nowadays endure in such codebases.
"There is a place for old C style low level features, but those should come later, after the safer concepts are already ingrained."
And I agree that teaching idiomatic C++ is important and C stuff is an extension.
If replacing vector with a C array gives you 20-30% my general assumption is that vector was used wrongly and there were too many copies happening and passing a pointer around avoided them.
And yes, when teaching pure C++ students will also easily create too many copies, but that's better than all the buffer overflows, null pointer dereferences etc. happening to students of C. Such bugs cause frustration, but students need results which give satisfaction.
See the idea is you can use std::array, so you can use modern idioms (and at() for bounds checked access).
No, not Xcode. Apple is not enthusiastic about c++ support and using Xcode is nice for debugging only, but write your code in something else. A lot of Mac devs will use Clion or for me, emacs, and then deploy or debug with Xcode when necessary.
Xcode doesn't format a lot of C++11 features properly and it's just a bad editor in general, even if you are using an "Apple-approved" language, which is why a lot of Swift/ObjC devs still use another JetBrains product, AppCode, for actual editing.
Also, the modern open source C++ stuff from Microsoft is also great.
Go for Chrome, V8, Dart, ChakraCore, the C++ part of DotNet VM.
But the only "problem" with them, is that they use their own sort of smart pointers, but if you know that they map 1-1 to smart pointers from std:: when you need to use the std ones, its not a big deal.
Chrome for instance have scoped_ptr = unique_ptr, scoped_refptr = shared_ptr, etc..
CppCon 2016: Kenny Kerr & James McNellis “Putting Coroutines to Work with the Windows Runtime"
IMHO, they are the right abstraction on top of event-loops. Every major server platform, especially - JVM, CLR, should support them.
I would be very much interested in context-switch data of server applications for Threads vs. Coroutines loads.
It's one of my favorite features of Lua and lets you do async-anything in a really nice and clean manner with minimal overhead.
IIRC, you're using a thread pool under the hood, and you sort of have to opt-in: if you're trying to write your own async code, for example, you need to schedule it in a way that's slightly more complicated than just saying `go myFunc()`, but it's possible.
As for the JVM, libraries like Akka and Quasar already provide this, although possibly not as performantly as if the runtime itself provided it?
Interesting time for all we C++ fans.
But seriously, the library implementation for coroutines is just a shallow wrapper around compiler builtins.
Therefore libc++ is probably bug free, but Clang certainly isn't.
EDIT: "probably bug free" == famous last words.
[0]: https://clang.llvm.org/cxx_status.html#ts [1]: http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-2017...