What Color Is Your Function? (2015)
journal.stuffwithstuff.com
journal.stuffwithstuff.com
I mean for languages that have event loops at their core, why isn't every function `async` by default? Let the caller decide how it wants to use the function.
`async` functions don't wait for a result of a `Future` unless `await` is used.
Instead of putting all that effort into putting `await` everywhere, why not invert that logic and make them `await` by default, and instead have the async-await syntax used by callers?
Like if one to were to write a transpiler to do this in dart, it might compile the following -
void main() {
Future<int> xFuture = async doSomething()
int x = doSomething()
assert(await xFuture == x);
}
int doSomething {
...
}
Future<void> main() async {
Future<int> xFuture = doSomething()
int x = await doSomething()
assert(await xFuture == x);
}
Future<int> doSomething async {
...
}
I looks an awful lot like what go does with its `go` syntax, but because you have an event loop with Future and Async Streams, you don't need to worry about dealing with CSP.
* Many popular languages today predate modern asynchronous computing. This makes async-as-default impossible because it's an afterthought
* Async computing is harder to learn and confusing for those just picking up the language. There are some pretty major ergonomics issues that you have to solve if you want anything more than a DSL to achieve noteworthy adoption levels.
Here's a rough sketch of how this can be achieved for a language that already has event loops -
Transpile the following -
void main() {
Future<int> xFuture = async doSomething()
int x = doSomething()
assert(await xFuture == x);
}
Future<void> main() async {
Future<int> xFuture = (() async => await doSomething())();
int x = await doSomething();
assert(await xFuture == x);
}
BTW that transpiler output is runnable dart code actually works.
I know this has tons of edge cases that this simple example doesn't capture, but it's definitely "possible".
//A async candidate
fun read_lines(file):
for line in file:
yield line }
let lines = read_lines("hello.txt").await //turn async
fun uppercase_lines(file):
for line in read_lines(file):
yield line
let lines = uppercase_lines("hello.txt").await //turn async, but also recursivelly read_lines? const name = await swear(fetch('/some.json')).json().user.name;
console.log(name); // Francisco
const error = await swear(readFile('./error.log')).split('\n').pop();
console.log(error); // *latest error log message*
While I do not use it directly since adding a library to make syntax slightly shorter defeats the point, I've included it into some of my async libraries:
• File handler `files`: https://www.npmjs.com/package/files
• Simple command runner `atocha`: https://www.npmjs.com/package/atocha
• Enhanced fetch() `fch`: https://www.npmjs.com/package/fch
[1] https://www.npmjs.com/package/swear
[2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
None of this is really problematic at the top level view of things. But when you need to compose libraries or applications that make use of these things - yes, even channels in Go - you can start running into problems.
Especially if you don't actually control the process you're running in. This is why promises and async/await really exist. E.g. if you have code you need to fork off the main UI thread so you don't block it, but need to re-capture the UI thread so you can call UI update code when you're done with your long running task. async/await is so much cleaner here.
Async/await allow for much more complex control of flow than simple goroutines and channels. Sometimes it's necessary. Sometimes it isn't. It was necessary in nodejs. It's extremely useful in applications where you have a "main thread" that drives your application that you don't want to unnecessarily block. Go doesn't solve this problem out-the-box. There's a reason why c# adopted async/await despite having futures and all sorts of other tools.
I don't really know how go UI frameworks work. How do you have multiple, preemptively scheduled goroutines on the UI thread but without critical sections? You have to use channels and message passing back to a main thread manager to handle this, yes?
I think Java's Loom would have the same issue but again, I don't really know. Perhaps worrying about a UI thread is 'fighting the last war' and we should work on new UI paradigms in these new language features.
I explored the landscape earlier this year when I was building a GUI version of a CLI tool written in Go. I was throughly disappointed by all the “native Go” and non-webview options — narrow selection of widgets with basic functionality missing, UIs ranging from lack of polish to horrendous looking (that makes Java GUIs look like godsend), lots of bugs, etc. I ended up using the Qt binding which, despite its own share of problems, at least worked fairly reliably and didn’t constantly get in my way in every way imaginable: https://github.com/therecipe/qt
First you'd probably call runtime.LockOSThread() to tie a goroutine to an OS thread if the native API you're coding against needs that (like Cocoa, OpenGL, etc).
To perform work on the main thread using closures is probably the most convenient way. So you just have a RunOnMainThread(func(){...}) that puts the closure on a channel to run on the main thread (or uses some similar feature from the underlying native framework).
It's not terribly inconvenient - it's similar to the old Cocoa / UIKit performSelectorOnMainThread: method except that you have closures to make things simpler.
There's probably a little syntactic overhead compared to if you had several async/await functions running concurrently on the main thread. But on the other hand it's probably a bit easier to reason about, since the flow of execution on the main thread is very straightforward (if you have multiple async/await functions running at the same time on the main thread, it seems like every time they await a result they'd have to worry about other code being run on the main thread, and make sure they release any locks, etc).
A simple example of where this might be useful is in UI code. Every UI framework I've worked in has a single UI thread, where all UI changes must go - e.g. adding a button, adding text to the UI, etc. At the same time, you don't want to block the UI thread, because that will cause the UI to become unresponsive.
A common thing you might do in a UI is: on button click, make an API call, then update the UI with that data.
The on button click will be called from the UI thread. You don't want to make the API call there though because that would block the UI thread. So you spin it off into a worker thread. But after that worker finishes, you need to regain control of the UI thread so you can update the UI.
The way you traditionally do this is you have some kind of main loop which is running the UI thread, which has (among other things) some kind of queue of functions it should call (I assume it would be a channel in Go). So your worker thread will hold onto a reference to this queue. When it finishes, it pushes a function call onto the queue. The main UI loop occasionally checks this queue and calls all functions in it. So eventually it gets called, allowing you to update the UI in the UI thread.
This can be fine for simple scenarios. But if you have deeply nested or complex interactions between multiple threads, the code can get confusing and turn into "callback hell". This is the problem that was ran into in early versions of nodejs. This is the problem async/await was intended to solve. It's a bigger problem in nodejs due to its design, but it can still be useful in other languages that don't share that design depending upon the application you're writing.
Channel is what it says it is: a way to send data between goroutines.
Also, Go works just fine for UI code, out of the box. See https://github.com/lxn/walk for one of many examples.
You just lock main goroutine with runtime.LockOSThread() to its thread and that's your UI thread and marshall code that touches UI to UI thread. Which is the same thing you must do in C# (or any other language). See https://github.com/golang/go/wiki/LockOSThread
In fact there are not function at all, just like generators are not.
Also coroutines can be called from functions and vice versa. You don't have to choose.
The problem is not the color. The problem is the default paradigm is to be blocking I/O, and you add non blocking on top. It's way easier to make everything non blocking, then add blocking things on top.
It's not a matter of syntax, but of core paradigm.
Go and erlang were designed from the non blocking perspective from the get go.
But even go add to use something to manage async control flow, so it uses channels, which you don't need if you have coroutines.
Erland went deeper and made everything immutable, and baked in actors in the runtime. To me that's more impressive.
I might use this test: When I wish to add a call to a function that yields control other than by returning to the middle of a regular function, so that the latter portion of the regular function can use the results of yielding, do I then have to change the function declaration and every single call site?
You clearly disagree. What test should we use?
Coroutines are not functions, they are coroutines. Different primitives.
A function has a single exit entry point and a single exit point and no state. If you call a function, you run its body.
A coroutine has several entry points and exit points, and an internal state. If you call a coroutine, you make an instanciation (the body doesn't run).
You can argue than using coroutine for async handling has drawbacks, but colored functions is not the proper analogy and make people very confused about the whole thing.
The fact the syntax to define them is so similar doesn't help, and I see most people difficulties comming from their attempt to reconciliate models that have no reason to be.
Just like a hashmap and an array are 2 different things despite you can index both, functions and coroutines must be understood as separated concepts.
Some programming languages have stackless async functions and generators that work the way you said, and some other languages have stackful coroutines. In the languages I use the most, functions that switch to other parts of the program are functions (according to the type system and the calling convention, but not in the sense that they have only one entry point and one exit point and no state). This is pretty great, because things are composable in precisely the way the article complains that they are not. Given that these things exist, it's fine to say "coroutines and functions are two different things" but maybe saying "a function which jumps to a different stack and which can then be jumped back into is not a function" will just make people very confused about the whole thing.
If you have this, uh, code fragment:
function read_five(socket)
local data, err = socket:recv(5)
if not err then
return data
end
return nil, err
end
> A coroutine has several entry points and exit points, and an internal state. If you call a coroutine, you make an instanciation (the body doesn't run).
Sort of, in some languages, but it doesn't have to work that way.
First, let's note that "blues" do have state, and they put it on the stack. So blues can only be entered once and store state on the stack. Reds store state in an object, and they can yield out of the middle and be reentered.
Then let's note that our top-level code has to be "red", or it would be impossible to ever run "red" code.
Now let's go on a journey.
1. Make it so calling a red from a red will start running the body right away, by default.
2. When calling a red from a red and running the body, we're already inside a coroutine instance. Instead of making a new one, keep using the same one. Grow it and store the new state at the end.
3. When calling a blue, if we're inside a coroutine instance, put the blue's stack inside it. And since our top-level code is red, the we always are inside a coroutine instance.
4. Since all our stack frames are safely stored inside coroutine instances, it's safe to call from a blue into a red! The entire stack can be saved for later, blue and red alike.
5. At this point the only difference between blue and red is that a blue function cannot yield, it can only have something deeper on the stack yield. For fun, you could make 'yield' into a runtime-provided function. Now functions written in the language are all the same. There is no difference between blue and red.
-
So there is still a distinction between "coroutines" and "functions". But in this setup, a coroutine is merely a container that you run functions inside of. There is only one kind of code you run, and it is a function.
Some language work this way. I wish javascript worked this way.
It cannot be applied to legacy languages.
Just like you could not add a borrow checker to C without creating 2 worlds in the C community. But you can design Rust with this in mind.
A design always has a context.
Also, make a giant coroutine has a performance price, because any line is a potiential context switching.
The reason you wouldn't do it in C is that there are so many implementations and only a couple of them will actually update. In most languages the extra difficulty for retrofitting it would probably be less than the difficulty of designing and implementing it in the first place.
> Also, make a giant coroutine has a performance price, because any line is a potiential context switching.
A notable one? You need a stack anyway, and you don't really have to change anything to make it switchable.
You have to avoid taking a mutex and then calling into arbitrary code, but that was already a terrible idea.
The thing is, Go doesn't have async functions, because it doesn't have await. That is why you don't have colored functions in Go: it doesn't support anything as advanced. Sure, the runtime does really cool things under the hood, but the Go programming model is more like Threads than async/await, because goroutines can't return data and they can't throw exceptions.
For passing information, you use channels, which can pass more than one value back to the caller.
The big thing is not running tasks in the background but the tight integration of channels and runtime scheduler that allows having an invisible event loop on top of what is synchronous programming.
Channels are nice, but they are not that hard to replicate in other languages (probably less efficiently). They have their uses, but they can't completely supplant shared memory or async/await - all 3 are nice to have in different situations.
var foo = await bar();
c1 := make(chan float64, 1)
go bar(c1)
foo := <-c1
Go seems better suited for high throughput message passing.
foo := bar()
That's syntactically different though, isn't it? Would the Go runtime lock you to that specific thread and block it or sleep that goroutine and move it to another thread when its no longer sleeping? Maybe in Go there's no difference in those concepts but in many languages certain threads are special. UI threads are often used for event serializiation but beyond that you might have a GL context on a specific thread.
How does Go manage that? How do you protect yourself from the runtime splitting your single thread work at a point you didn't intend? You probably have to make sure you only have a main goroutine that runs in a threaded way (as they do) and pass messages to it.
I can't really find an idiomatic Go UI example so I don't know what the answer is.
By default, the Go runtime does not guarantee any affinity between Go-level threads (goroutines) and OS threads. There's a way to forcibly pin a goroutine to a specific thread[1], but it limits your concurrency.
The Go approach does indeed cause problems wrt. OS-thread-affine state. It's a tradeoff. For RPC-oriented network services, the Go niche, OS thread state is very rare.
There are exactly the same reasons in Go to want asynchronous calls as there are in C# or Java or C++, except that performance of multi-threaded code (which is the semantics of goroutines) is much nicer in Go. Sure, channels can sometimes be a nice alternative to locking, if you can afford all of the copying.
But Go is stuck with only multi-threaded code + synchronous calls + locking/channels, whereas in C# or Java or even C++ you can chose between using that OR asynchronous code with futures.
Which isn’t the comparison I’m replying to here. Calling an asynchronous function with “await” forces it to behave synchronously. You would only do such a thing if you were operating in a framework or language with colored functions.
> in C# or Java or even C++ you can chose between using that OR asynchronous code with futures.
Java, C#, and C++ have channels?
I don't really think you are right. When you await an async function, you let the async function run asynchronously, but suspend your own execution until you can receive the result from that function (an actual return value, an exception, or simply termination for void functions).
Calling a function directly forces it to run on the same execution thread as you. Calling it with await allows it to run in any thread. This is the actual advantage, and Go doesn't have any equivalent construct that is as convenient for this use case.
Also, note that a function that expects to return data through channels can't be called in a sync manner in Go or it will deadlock. So in essence there is function coloring in Go as well.
> Java, C#, and C++ have channels?
Not out of the box, but they are easy to replicate if desired, wrapping a lock in a send/receive interface (you can add a buffer as well if desired). It is probably not as efficient, but it may not be vastly different either.
OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by running that coroutine in Thread B?
One potential answer is that while Thread A is blocked by “await”, it can context-switch to a different coroutine. You can effectively do similar things in Go if you want to. But doing so abandons the guarantee that Thread A will pick up where it left off as soon as Thread B is finished.
> Also, note that a function that expects to return data through channels can't be called in a sync manner in Go or it will deadlock. So in essence there is function coloring in Go as well.
Is this a popular or idiomatic interface for Go library code to the same degree it is for “async” libraries in other languages?
In isolation I find it more understandable to do channel writes as an explicit side effect than to manage futures but maybe that’s just my brain.
> Not out of the box, but they are easy to replicate if desired
I’m pretty sure you could implement futures and async/await using Go channels too if you wanted to.
If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things in parallel with that run, and now that we need the result, we can block.
For example:
myHttpClient.StartReq1()
myHttpClient.StartReq2()
auto Res1 = await myHttpClient.WaitReq1()
auto res2 = await myHttpClient.WaitReq2()
Given the lack of generics, you would get a much worse interface. Btw, here is what a non-buffering channel would look like in Java:
class Channel<T> {
T value;
void publish(T value) {
synchronized(this) {
this.value = value;
try {
this.wait();
} catch (InterruptedException e) {}
}
}
T consume() {
synchronized(this) {
this.notify();
return this.value;
}
}
}So in Go that’s just a blocking channel receive.
Go and channels allow you to block many threads with little overhead and little syntax but at the cost of not being able to easily target a single thread without manual orchestration through channels.
Consider a scatter/gather algorithm. N async tasks, N coroutines. Start N from the main thread and block until all complete. Simple right?
Its not!
Imagine there are GPU commands that must be grouped or even logs where you want some of the subroutine's logs to be grouped.
With async/await you have the verbosity to synchronize tasks to thread contexts such that you know chunks will not be run in parallel. Its easy to control when execution leaves the current thread. It can ensure that it both does or does not yield execution. You can easily switch between synchronized main thread execution and parallel chunks without any top down design.
With goroutines you most likely would write the ordered results to a channel that was passed around. I'm not sure if channels support N inserts in an atomic fashion out of the box but if not it must be a channel of arrays or maybe some kind of control channel as well. Hopefully you have access to every piece of code you need to synchronize. This all assumes you can get away with a single main goroutine. If you need a single special OS thread for interop I'm sure its more complex. Its not just a blocking channel.
Essentially the two paradigms can do everything but they both seem to excel in certain cases.
Yes, `await Bar()` is equivalent to ` <- ch`, but only if there is some goroutine that actually writes something to the channel. So there is no getting around the fact that there are colored functions to the same extent in Go as in C#/Java, in regards to asynchronicity.
And I maintain that Go does not have colored functions. Colored functions are when you hijack function call semantics to do asynchronous programming. Go doesn’t do this; the asynchronous behavior has to be done explicitly. Even though it’s possible, it isn’t idiomatic for a function to return a channel that you have to try and receive later, the way async functions have to be awaited before you actually get the return value.
The most correct way of looking at this is that async fucntions return a channel that you can read data from, and await is exactly like a channel read. The magic in async/await comes in the function calling 'await', which is suspended and scheduled to continue later. So, just like in Go, you could create a function that needlessly uses channels but is otherwise synchronous, or you can have a function that uses channels because it actually is doing something in the background.
> Colored functions are when you hijack function call semantics to do asynchronous programming. Go doesn’t do this; the asynchronous behavior has to be done explicitly.
Colored functions are a very general concept. They refer to families of functions which perform the same computation but that need to be written differently for syntactic reasons. Famous examples that have nothing to do with async include C++ const-ness, where you have to write a const and a non-const version of a function to to be able to offer the same functionality for this and for const this.
Go has this with functions which return data versus functions that push data on a channel. If you want your function to be callable in a sync manner, it must return the data directly. If you want it to be callable in an async manner, it must take a channel and push its result on that channel. You can't launch a sync function as a new goroutine and get the result back. You can't call an async function as a regular function and read the result back. So, they are two colors of functions in Go.
More concretely, say you are writing an HTTP client lib. If you want to allow your users to start multiple requests in parallel in a nice fashion, you must write your request function to take a channel and push its response there. If you want to make it easy for your users to just call your function in a sync manner, you must write a function that returns the response as its return value.
Of course, you can implement one in terms of the other, as demonstrated earlier. But you still need to add that wrapping, so your functions have a color.
Futures, maybe (though without generics you'll be either very dynamic or write a new future for each struct, of course).
But async/await is a syntactic feature and can't be implemented in a language without macros and/or continuations. Basically `await` is a keyword which returns a future that will execute the rest of the function as written. Something like this is relatively easy to implement:
async Task<int> Foo() {
int i := await Bar();
return i + 1;
}
func Foo() func()(int){
var reply chan int
go func() {
reply <- Bar()
}
return func()(int) {
i := <- reply
return i
}
}
async Task<int> Foo() {
auto sum = 0;
foreach (auto x in someEnumerable) {
switch(x) {
case 1:
sum += await Bar(x);
goto case 2;
case 2:
sum /= await Bar(x);
break;
}
}
return sum;
}
To show a more realistic comparison, let's asume Bar() is not "async ready" in either Go or C#. Then, the C# code would look something like this:
async Task<int> Foo() {
var task = Task.Run(() => Bar());
int i = await task;
return i + 1;
}
func Foo(ret chan int) {
task := make(chan int, 1)
go func() {
task <- Bar()
}
i := <-task
ret <- i + 1
}
And again, in both languages, there are clear demarcations between async functions and blocking functions. In C#, async functions return `Task<T>` and are usually marked `async`, while in Go async functions have one or more `chan T` arguments and don't normally return values. If you want to call an async function from a non-async function or vice-versa, you need to use special constructs (e.g. Task.Run, Task.Wait or Task.Result for C#, and `go func () {...}` and blocking channel reads/writes in Go).
Yeah they do but without green threads the implications are a bit different. You can use channels and OS threads though most code bases don't.
var foo = await bar();
foo := bar()
const foo = await Promise.all(randomArray.map(bar));
foo := bar()
auto foo = bar();
It is true that, due to the runtime implementation,
go foo()
new Thread(() => foo());
But if you want to start multie things in parallel and await all of them, or if you want to create asynchronous workflows that modify shared resources without locking, you don't have any Go library or construct to help, which was precisely what async/await gives you.
Golang functions only have one colour. This colour may be closer to "red" than "blue", but at least you never need to concern yourself with this as the consumer of an API, and you can count on the rest of the language having been designed around what is possible with these "purple" functions, as opposed to JS where the situation is that the language provides you with plenty of nice features to structure your code on paper but the design of existing code you need to interface with prevents you from effectively using them.
Neither does the C# snippet. In fact, if you want to extract the result of the function call as well, you can still keep you API as is in C#, and use Task.wait() or something similar to block execution until the task is finished, and read its result.
In Go there is no such alternative: if you have a value-returning function, and want to run it asynchronously and continue when the value is available, you have to re-write the function into a void function and add as many channel parameters as return values it held (or wrap the original function with something which calls it synchronously and then pushes those values on the channels),or use some shared memory to store the results. And make sure to also catch any panics the function might raise, if you were planning to handle panics.
I would not be surprised if some future version of Go introduces async/await to handle all of this complexity for you, except that Go developers usually hate making things easier for their users.
Edit to add: channels in Go actually tend to impose colors on functions just as much as async/await in C#. You can't call a function which returns its result in a channel synchronously, you must start it in a separate goroutine. Unfortunately, the compiler doesn't know this and will let you create a silly deadlock that way.
You can't if you want to free the OS thread while waiting for result.
Frankly, I have a hard time imagining too many seasoned developers clutching channels to their chest as something superior to async/await. Async code with channels is not simple, you have in-channels, out-channels, and channels-over-channels. It's a pretty niche instrument working best when you only have uni-directional communication in your application.
For example, I've worked on channel code where I'd rather be debugging mutex deadlocks.
But... don't bother too much trying to explain this to ppl who don't intuitively grok it right away... some just seem to never get it no matter how hard you try and explain it to them, it's like their brains are "wired differently" when it comes to reading and understanding code, they don't get the advantage and meaning of unification / universality / "one solution for many problems" etc.
Go is NOT my favorite programming language, but there's a stroke of simple genius in it that probably only got materialized bc its creators were left alone to work on it their way inside Google and just implemented their solution to things without being bothered by "language experts"...
EDIT+: not saying "colorless + channels" is always better or anything like that, all's tradeoffs... probably async/await code is much more readable than channels hell for many/most cases (but because it's less powerful - no true parallelism possible).
2018: https://news.ycombinator.com/item?id=16732948
discussed at the time: https://news.ycombinator.com/item?id=8984648
a bit more from the same day: https://news.ycombinator.com/item?id=8982494
* spawn/1 becomes fork()
* send/2 becomes kill()
* receive F -> ... G -> ... end becomes sigaddset(&s,F);sigaddset(&s,G);sigwait(&s,&r)
There are of course other issues, like there aren't that many signals, or that memory management is hard, or whatever, but these are solved by other languages as well, so you can imagine this is at least straightforward to implement.
The downside is that it means you can have functions that you can't write. That can be a big bummer to a lisp programmer, but I don't think python, javascript, etc, programmers care about that, so I think history will judge the trend to colour functions as lazy and shortsighted.
Thinking in terms of functional programming / category theory, this coloring seems to boil down to working with different arrows. The coloring in this case would correspont to signifying the target category of the arrows `arr/lift` function (For example async functions in JS should be morphisms in smth like the Kleisli category of the `Promise` functor).
The issue now seems to be that there are no nice native language constructs in most languages to work with arrows. What's missing are arrow comprehensions - the bridge between the compositional notation used in functional languages and the more traditional way of writing things like `const x = ...`. The vanilla functions without `async/await` are arrow comprehensions for the identity monad.
The reason why the `async/await` syntax works so well as compared to using `.then` is that it is basically arrow comprehension for the promise monad.
However, if you can avoid the additional arrow entirely: even better!
I mean Haskell of all languages has green threads and doesn’t need the entire promise shenanigans.
Even in Haskell I sometimes I build very specific DSLs for my domain to be interpreted in a separate step so I can freely fix my pure and impure code.
I don't think it's a "good thing". I think it's a "cost". Even in nice functional languages, it's a cost, in that even with the nicest syntax there is you still get color cascades when you realizes deep in the middle of some deep function that you're in the wrong color.
There's a great case to be made that said nice functional languages, through a combination of ameliorating the costs and successfully obtaining benefits from the separation, mean that it's a good tradeoff.
Conventional imperative languages do not successfully obtain enough benefits from the specific color of "asynchronousness" (functional languages have many more colors than just that) to make it worth while. Covering it over with syntax wouldn't be enough, because it's not just a syntax problem. What you really need is to not have this particular color in the first place. (While arguably lacking other colors that you really need, but that's a story for a different day.)
Unless you refuse to pay even a little bit to avoid that color problem (which is a valid choice in some circumstances but I would submit a bad default choice for most code and most coders), we can increasingly see that there are languages that do not have the async color issue at all, and pay only modest performance prices to do so. As languages like Go and the Beam VM continue to progress, and are joined by other up-and-coming languages, even the problems with being unable to interact with C libraries fade as you can find something that does what you need for the most part. (And besides... personally I "blame" C rather than the languages trying to do better than C and then having trouble interacting with such primitive code.)
I wanted to use @usefultools/monads to have Maybe types, etc, but it's easier to just stick with promises throughout the chain. Essentially working with functions that return promises (like database calls with Mongoose) makes you convert your functions to also use promises. As far as I can tell there's no way to unwrap a promise (similar to Rust) without the await keyword inside a try block.
I don't know if this is exactly the best approach, but it's significantly less verbose than .then().catch() callbacks. I also learned about Promise.all() and Promise.allSettled() recently here [0].
I wish I could stick to futures with Rust or goroutines with golang. Bleh.
His write up on the process is also interesting:
http://journal.stuffwithstuff.com/2014/11/03/bringing-my-web...
For example, something as simple as:
const result1 = promise() // start this promise first but don't await it yet.
// ...
const results = await Promise.map([promise(), promise(), result1])So instead of a.map(v => f(v)) ... or Promise.map(a, f) ... or a.map(f) ... it's just f(a).
If you then want the value, in the end, if at all (e.g. the web framework understands Promise and you return a Promise in your controller and not a value, you don't need await at all).
You'd need some syntax for Promise.all(...) though to "join" concurrent executions again.
The language should probably have some syntax for multiple writers, e.g. and atomic compareAndSet.
shared v = { .... }
v.change(v, f)
But your functions would not have colors.
I'm not sure I agree with the conclusion: I find using futures with very carefully controlled threading a preferable paradigm (for how I structure my programs in C++, at least for my text editor: https://github.com/alefore/edge).
In my experience, using sync code looks more readable on the surface but just kicks the can down the road: you'll still need to deal with the complexity of threading, and, in my experience, it's going to be waaay uglier. What you gain with your "superficial" simplicity, you pay a hundred times over with the algorithmic complexity of having to use threads with shared state. Every time you're troubleshooting some weird race condition that you can't easily reproduce you'll be wishing you had just used futures.
What I do is that the bulk of my processing runs in the main thread and I occasionally dispatch work to other threads, making sure that no mutable state is shared. When the "async" work is finished, I just have the background threads communicate the results to the main thread by setting a future (i.e., scheduling in the main thread the execution of the future's consumer on the results). Async IO operations are modeled just the same. In the beginning I had used callbacks spaghetti (mostly writing things in continuation passing style), but I started trying futures and found them much nicer.
I'll admit that on the surface this makes my code look slightly uglier than if it was directly sync code; however, it allows me to very safely use multiple threads. I think not having to make my classes thread safe (I typically stop at making them thread-compatible) and not having to troubleshoot difficult race conditions (and not having to block on IO or being able to easily run background operations) has been a huge win. If I had to rewrite this from scratch, I'd likely choose this model again. My editor only needs to use mutexes and such in very very few places; it suffices to make my classes thread-compatible and to ensure that all work in threads other than the main thread happens on const objects (and that such objects don't get deallocated before the background threads are done using them).
I rolled out my own implementation of futures here: https://github.com/alefore/edge/blob/master/src/futures/futu... (One notable characteristic is that it only allows a single listener, which I found worked well with "move" semantics for supporting non-copyable types that the listener gets ownership of.)
Here is one example of where it is used: https://github.com/alefore/edge/blob/5cb6f67e1e0726f8fbe12db...
In this example, it implements the operation of reloading a buffer, which is somewhat complex in that it requires several potentially long running or async operations (such as the evaluation of several "buffer-reload.cc" programs, opening the file, opening a log for the file, notifying listeners for reload operations...). It may seem uglier than if it was all in sync code, but then I would have to either block the main thread (unacceptable in this context) or make all my classes thread safe (which I think would be significantly more complexity).
I think this is cleaner than callbacks spaghetti because the code still reflects somewhat closely its logical structure. For loops I do have to use futures::ForEach functions, as the example shows, which is unfortunate but acceptable. In my experience, with callbacks spaghetti, it is very difficult to make code reflect its logical structure.
So first you think there is a problem, but there are not any, but by solving the imagined problem you do actually create it. You now have generator functions and "async" (with the async keyword) functions that always returns a promise, and normal functions.