I see goroutines as a solid "next best thing" after BEAM Processes, both of which are miles ahead of async/await, which is admittedly an improvement over any lower-level thread manipulation.
Async/Await, asynchronous IO, none of this is really the same kind of “concurrency”… it’s why I wish I had more chance to use the Erlang/Elixir+Rust combo… low level safety and high level concurrency are a match made in heaven.
You cannot achieve this by tacking on half-baked support onto an already designed language. You can tack on Async/await, which is exactly why it's so popular.
- ports are subprocesses acting as erlang processes
- erl_interface is a more efficient version of the same as the communication uses BEAM's external term format
- C nodes are what it sounds like, basically a separate program acting as a node in a beam cluster
- port drivers have a shared library acting as a process, it's way more efficient but jettisons safety as a crash in the library kills the runtime
- finally NIFs are synchronous functions called in the context of an existing process, they are the fastest option but not only do they kill the emulator they can also screw up the VM state, and they can't be managed by the scheduler without their cooperation (which can severely degrade system stability), BEAM has a concept of "dirty NIFs" which are long-running, can't be suspended, and can't be threaded, if those are appropriately flagged they are run on dedicated "dirty schedulers", dirty schedulers have more overhead but less restrictions, in essence cgo is a more impactful version of that (although I believe dirty schedulers were invented a while after cgo existed, previously "dirty nifs" would just see you drawn and quartered)
Ports just wrap executables you can feed STDIN to and get STDOUT from in order to treat those streams as messages. You make one with one line of code, they behave just like any BEAM process. (you can message them, kill them, etc). If your goal is to use an existing DLL, this involves some C glue code, which looks like what you'd do if you wanted to be able to call the DLL functions in a bash script (take in STDIN, translate strings to appropriate types, call function, return strings to STDOUT).
The erl_interface option is using that library to replace the "translate to/from strings" task with "translate to/from BEAM types". Still a fair amount of glue code for "I want to call this DLL function", but I feel like it should be possible to codegen a lot of it. That might exist already, and if it doesn't it sounds like the kind of fun project I might pick up.
C nodes are using erl_interface to its fullest, defining a full blown BEAM node with one or more processes running on it in C. In practice, this means you can send messages to other BEAM processes, rather than going through an intermediary Port process. It's definitely the most involved option, but it's well documented (like everything else in OTP).
Port drivers free you from the concept of messages within the (even smaller) C code: You make a DLL (that links your target DLL) that provides some mapping information and some dispatch code, then in your BEAM language handle all the "send/receive messages" stuff associated with being a BEAM process. The BEAM node crashing if the library crashes is rightfully considered a significant issue, but it's worth noting that we only care about that so much because the BEAM spoils us with much better safety normally. In any other programming language, a crash in your linked library would probably be expected to cause your entire application to crash, whereas in BEAM land we can even mitigate this by putting our risky code on a different BEAM node, running on the same or different machine, to limit our blast radius.
NIFs allow you to present your C DLL function call as a normal, synchronous BEAM function call. They require the least C glue code, but if any of those calls take more than a millisecond or so you start getting into "thar be dragons" territory on the clean scheduler, or require the use of the dirty scheduler which slows everything else down.
Ultimately, the punchline to all this is that if you want to call an existing shared library from the BEAM, you're going to have to write some amount of C, ranging from "a couple lines per function you want to call" to "defining a small runtime that handles dispatch based on strings".
(That's partly why I find all the discussions about "async/await" "colors" silly: they aren't colors, they are types. You build types for other state machines, right? You don't complain about Regular Expressions [which are also often rewritten to simpler state machines] having their own types and that they can only embed other Regular Expressions and that consequent state complexity as having "colors", do you? About the only real difference between async/await and Regular Expressions is that the types for async/await implementations are all guaranteed to be monads in the async/await languages and might not be for Regular Expressions. Though monad Regular Expressions libraries exist.)
Whether a function is called asynchronously or not should be a decision of the caller, not of the function itself. Sometimes it doesn't make sense for me to continue when I don't have an answer yet. Sometimes it does. That makes it the caller's concern.
The solution to this in Javascript would be to simply make every single function an async function, but that introduces a bunch of clutter because async/await is a feature to be bolted onto an existing language, rather than a paradigm to build your language around.
In contrast, look at Elixir: No function coloring to be found. Want to run something asynchonously?
res = Task.async(&any_function_i_like/0)
\\some other stuff
out = Task.await(res)
out = any_function_i_like()
"But wait!" you might be thinking "What happens if the task fails?" Well, that's up to the caller, too. Maybe that only rarely happens, and there's no logical path forward from that failure, say, a very tiny meteor punches out the processor core that task is running on. That's okay, by default if the child process dies, so does the caller. This is passing the buck, but is the correct answer more often than not, so it doesn't make sense to make the developer do extra work to make it happen. But maybe a failure is somewhat likely, and we just want to keep trying until it works. That's easy, too! Just spawn a supervisor process that'll make a new copy of the called function if it fails. One more line of code relative to the async case. Maybe the function is just to make some side effect, and it doesn't matter if the parent crashes or not. Maybe it does.
There's a lot of different cases when it comes to concurrent programming. It does not make sense make those decisions unilaterally for someone using your library. Leave it up to the caller to decide whether your function is synchronous, asynchronous, long-lived, independent, dependent, etc. Unfortunately, that's hard to tack onto a language as an afterthought, so we get Async/Await.
I completely disagree with this, and I think that's a strong summary of why our opinions are so hugely diverged here. Async/Await was decades of programming language research in the making. It's "just" Haskell's do notation but with a lot of smart research into "okay, but how do we do that for beginners and junior developers". It may seem like an overnight success, but that's not because it is "tacked on", it's because it is well thought out, well tested, and well designed and therefore has spread to a number of programming languages. (I didn't say anything about Javascript. Javascript wasn't the first to add async/await and wasn't the last.)
I'm not sure it is a "failure state" of types, but I can agree it is a hard edge case, and there have been criticisms of it for decades. The same "why isn't every function in Haskell in the IO monad and written in do-notation" is the same "why isn't every function async/await" complaint. It's the same reason not every function in a language with iterators is written as a generator with yield instead of return. It's the same reason you don't solve every problem with a single RegEx. It's the same reason you use OO classes for encapsulation (or don't). Pragmatic programming languages are never just a single paradigm.
The callee knows what resources it is encapsulating and which of those need to be asynchronous. It knows better than the caller if there is going to be a suspend point what state it will need to transition into next and can prepare its internal state machine best for how it needs to operate. It's useful to bubble up that library knowledge into the type system and async/await gives a strong type safe way to do that (plus eases a lot of the trivia of building a state machine in languages that support async/await). Just like a generator function is a basic state machine for iterable results. Just like an OO library might encapsulate some amount of memory usage.
Your elixir example is a single function and you can do that in any of the async/await libraries to. The trick to async/await is that what you are writing in async/await isn't a single function but a complex state machine. Have you actually tried to rewrite async/await-heavy code to something like Elixir? It's not hard, but it is a lot like hand converting a RegEx to a DFA.
That doesn't mean that caller's "lose power", the power dynamic is shifted, but it isn't as deranged as you seem to think it is. async/await just defines the state machine possibilities, it doesn't know or care if the async/await state machines that have been defined complete synchronously or asynchronously. It doesn't care what threads/threadpools/green threads/greenlets/threadlets/whatever else those state machines run on, including the same thread as the caller. The state machines are generally agnostic to how their state transitions are triggered. Javascript doesn't give you a lot of power there, but that's because browsers have always been in charge of JS threading and that was true before async/await. That's not a deficit of async/await, that's a deficit of Javascript. (That you can't see past Javascript may indicate why you think it is a tacked on part of the language and not a success story from other languages modestly applied.) You can look up some of the tools that C# presents, for example, to let callers influence downstream scheduling. You can look up all the complaints that Python despite being the "there should be one clear way to do it" language decided that instead "explicit is better than implicit" beat out as the winning mantra and left scheduling to a handful of different libraries with different opinions of how async/await should be scheduled by callers, but the benefit is that apps have to explicitly opt in to one of those behaviors rather than a "works fine for most developers" out of the box default. The "power" remains you just have to learn new ways to apply it.