Futures in Go | Hacker News Reader

Futures in Go

blog.charmes.net

47 pointscreack14 comments

Futures are not necessary in Go, since they are a hack around not having the concurrency that Go supports natively. This is why the Go community never talks about "futures"... as in this example, they are a regression vs. the already-supported primitives, not progress. Channels are already a more general, more powerful, and most importantly, more composable version of the "futures" idea that is what most people consider "futures". (I caveat this just because there are many such ideas which have different characteristics, but it's not exactly what the JS community means by the term, for instance.)

You should return a chan. In this case, all crawlers should be returning along the same chan, errors or results, rather than creating one per request. Then you A: don't have a problem where you could be processing result #5 but result #3 is blocking right now, increasing the total latency, B: having multiple consumers using the same channel is trivial and at the user's discretion, and C: the resulting chan can participate in a "select" of the user's choosing, unlike a function call. There is no way in which using this approach is superior to using a chan.

If you look at the wikipedia page for "futures", which has a lot more than just any one community's idea of "futures" in it, you can see that a channel that you expect to read one value from is, if not necessarily exactly any one of the given definitions, certainly a family member. It isn't in the type system, but it is still the case that if you have a need for a "future" in Go you're better off with a chan with that constraint documented, rather than returning a function that can be called, because of the inability to select on such a function. http://en.wikipedia.org/wiki/Futures_and_promises

I haven't looked closely at the original post, but there are a few semantics that are useful for promises that current channels don't have (I'm more familiar with core.async than go). Clojure is working on promise channels: https://t.co/Z3TQbQMnRb

Agreed. Passing around channels (they are values anyway) makes a lot of concurrency problems simpler.

Note that when designing servers at Google in Go, we strongly discourage the use of concepts like futures and promises. They exist as abstractions that work around heavyweight threads. In Go, you have lightweight threads which remove the need for futures.

Create a new goroutine and block on the action. You can continue working in another goroutine and communicate between them using channels.

The advantage of this programming model over futures/promises is straight-line code that is easier to read and reason about. For example, when bugs appear you get a nice stack trace that reveals the state of the call. Future leave you in a callback-like mess.

See http://blog.golang.org/pipelines for more details.

I don't think I've ever heard it clearly explained why you rarely need futures in Go. (The blog article doesn't explain it either.)

It seems to be because each goroutine has a single parent (which launched it), so there is typically a single reader for its return value.

I'm not sure that's the case. Erlang has public-addressable processes: you can pass a process's handle around, and then anything with that handle can direct messages to that process—including messages containing their own process handles, allowing for responses; built on top of this, there's a global service registry, allowing anything to direct messages to a given process. Despite this, Erlang still doesn't need futures/promises.

Generally, futures/promises just solve the problem of how to simulate blocking within an event-loop-driven (i.e. cooperatively-scheduled) platform, e.g. Node. When actor-processes are cheap, your consumer can be an actor which just actually blocks on a result—and then continues execution in the same linear block of code when that result arrives.

To put it another way: in non-actor languages, you'd see a block of code like this...

    do_random_other_work();
    bfn_args = some_setup();
    blocking_fn(bfn_args).then(function(result) {
      console.log(result);
    });
    more_random_other_work();

...while in actor-modelled languages, it'd go more like this:

    do_random_other_work();
    spawn(function() {
      bfn_args = some_setup();
      result = blocking_fn(bfn_args);
      console.log(result);
    });
    more_random_other_work();

Note how you have a closure in both cases, but they contain different parts of the code. The actor-modelled closure contains the setup work of sending the message, as well as the work of dealing with the reply—which must be linear with respect to one-another, but which can all happen concurrently to the random other work.

As well, notice that the actor-modelled version of blocking_fn is actually a plain old blocking function—a synchronous-IO read, for example. The blocking_fn in the promise-modelled code, on the other hand, is a special function which must be defined using a promises library, and must do the work of ensuring its asynchronicity at definition-time, whether or not a given consumer wants to call the function in an async way relative to its own thread of execution.

In short, idiomatic actor-modelled code puts the consuming programmer in control of which tasks are linear and which are concurrent, which usually makes for succinct, reusable plainly-synchronous functions that Get Things Done, and higher-level glue code that represents all the policy of what concurrency happens where. Promises throw out all these advantages and mix policy with mechanism in a way that increases verbosity and decreases reusability.

I think that explains why languages like JavaScript will have many functions that return Promises and Go doesn't need that kind of API. Often in JavaScript, there will only be one reader for the result of a function call, but you'll use a Promise anyway since there are no threads.

But there are cases where you really want multiple readers. For example, on a cache miss, you want a single goroutine to do the work to fill the cache, but there may be multiple goroutines waiting on the result. In this case it might seem natural to model the cache internally as a map from keys to futures, even if it's not exposed?

Your actor-modelled example doesn't touch the idea behind promises, ie the background operations should return a result that is usable by the "main thread". In your example, that would be "result" being passed in some way to "more_random_other_work".

The way to do it would be closer to something like that:

    do_random_other_work();
    actor = spawn(function() {
      bfn_args = some_setup();
      result = blocking_fn(bfn_args);
      emit(result);
    });
    result = wait_for_result(actor)
    more_random_other_work(result);

This error pops up for me:

    This site or app is sending too much traffic to   
    rawgit.com. Please contact its owner and ask them to use 
    cdn.rawgit.com instead, which has no traffic limit.

It saddens me that this tool does not fail elegantly. It should deny access to the resource rather than mess with the browser.

This isn't a Future. A real Future would allow multiple goroutines to block waiting for the value to arrive. When it arrives, all the goroutines wake up.

In Go, each value on a channel goes to a single reader and it's rare to need multiple readers. One way to do it might be to return a channel that receives an infinite stream of the same value.

This is terrible. Each result blocks in the first example (e.g. if the first url in the array returns in 3s and the second url returns in 3ms - the routine will wait 3s then deal with both instead of dealing with the 3ms result immediately it returns). The second example blocks until all the results are in. This might be want you want, but if so then you might as well run them serially. The advantage of concurrency is that it can deal with the results as they come in.

The correct way to do this is to return results on a chan of (error, response) regardless of how much you "don't like it" for unspecified reasons.

    var (
            ch  = make(chan *http.Response)
            err error
    )

    ....
 
    return func() (*http.Response, error) {
        return <-ch, err
    }

It seems to me like calling that returned function is destructive -- if you block on it more than one, it would block forever. Is that right? It seems like a different contract than the usual Future / Promise shtick.

Yes, i wondered the same. The trick is that it relies on creating a waitgroup prior to that, and passing it as a parameter to the "future". You wait on that group, and only then will you be able to be sure that you're not going to wait when looking at the result of the promise.

Reply on news.ycombinator.com