Futures for C++11 at Facebook | Hacker News Reader

Futures for C++11 at Facebook

code.facebook.com

Rats, the terminology is backwards from the E/JavaScript promise/future distinction: https://en.wikipedia.org/wiki/Futures_and_promises#Read-only...

There's been a push to standardize "Promise" to be the read-only side and "Future" to mean the resolver/promise pair, but Folly Futures use the opposite convention.

Agreeing on terminology is hard. :)

Edit: I think it's fine that Folly adopted the C++11 convention. I'm just whining in the open that that promise vs. future may never be agreed upon.

I fucking hate the terms future and promise so god damn much. The names are beyond stupid. They're just confusing as hell. No one can read the names and make an educated guess as to which is which it what they do.

The following is my favorite interview response of all time.

Q: what's a quaternion? A: I don't know... But I bet it has four parts.

Genius! An honest answer followed by a wise educated guess. Now, what's a future and what's a promise? How can you remember the difference? Good luck!

Those terms should be retired forever.

A promise what you get when you can't get a result immediately: a promise to deliver a result later. "Future" is nonsense terminology.

You could as easily say that a promise is a promise you've made to produce a piece of data in the future. That's the problem with the term -- it's equally easy to interpret it as the consumption or production side of the equation. As an exercise in skeuomorphism, it's poor, since just like in real life, you can equally give and receive a promise.

Or, you know, a promise is another name for assert.

Except of course it isn't. Oops.

That's exactly what a "future" is in trading terms. I think the problem is conflating domains. I'm pretty sure most people will interpret future as "the future" and not as a "future contract".

Microsoft uses "Task". I like that way better.

and also the aptly named "TaskCompletionSource", which would be the promise equivalent (using Facebook's terminology)

That is, their terminology for this project. In facebook's Bolts framework for iOS they use Task and TaskCompletionSource https://github.com/BoltsFramework/Bolts-iOS#tasks

That term actually makes the subject much more understandable.

It doesn't. "Task" has a notion of side-effect and no notion of yielding a value. Except a task can yield a value and have no side-effect.

Make a header file for your project that exports the Facebook headers and typedefs the names to something that makes sense. I like source and sink. Producer and consumer are fine too. Emitter and receiver are fine as well. And writer/reader. If you do this, then you only have to make sense of which side is which once, then forever after you'll have sensible names for the things (at least in your project).

The important thing is to always treat the name the same. Generally, I treat types as nouns. So an emitter is something that emits something. It's passed to the routine that consumes the data. On the other hand, a consumer consumes, so it's passed to a routine that emits something for consumption.

I've always found "promise" intuitive personally; it returns a guarantee about a future action. "Future" gives me a vague idea that it has to do with something later, but isn't great.

Learning these terms isn't that tough, either. I don't find "promise" any less intuitive than "statement", "expression", "parameter", or "argument". I'm just used to the latter ones.

Scala also takes the same way C++11 does, and to my mind it makes more sense this way. So from my perspective Javascript is the backward duck here. I agree it would be nice if we could all just settle on something

E came out in 1997. From my perspective, Scala and C++11 are the backward ducks (unless you can point to something earlier using the Scala/C++11 nomenclature).

I agree the Scala/C++11 way might make a little more sense intuitively, but it really doesn't help flipping the definitions around.

It's much older than that. See

https://en.wikipedia.org/wiki/Futures_and_promises

The distinction between futures and promises in the wikipedia article matches Scala's terminology exactly.

Yes, Wikipedia's terminology exactly matches that of Scala, but that's because Wikipedia references a Scala SIP ;)

It's always bothered me that Baker and Hewitt felt the need to coin the term "future" when they were clearly aware of "promise" and "eventual". Their main distinction seems to be that a future is a 3-tuple (process, cell, queue). One can argue that their future meets the "read-only view" definition because cell is only to be written to by the same tuple's process. But they never explicitly distinguish a future as "a read-only placeholder view of a variable," and E was the first publicly-available language to make that very distinction (please correct me if I'm wrong), only they happened to use "promise" in that regard.

In any case, I trust your judgement on the matter more than my own, just wish everyone had picked a nomenclature and stuck with it.

There is another fine post about this project at http://instagram-engineering.tumblr.com/post/121930298932/c-....

It was posted as https://news.ycombinator.com/item?id=9746522, but we merged that thread with this one so as not to have two on the front page.

On a related note, MS has PPL on Windows: https://msdn.microsoft.com/en-us/library/dd492427.aspx

Very similar to C#'s TPL.

And VS2015 has experimental support for async/await in C++: http://blogs.msdn.com/b/vcblog/archive/2014/11/12/resumable-...

There's also pplx, which is a cross-plattfor implementation of ppl: http://microsoft.github.io/cpprestsdk/namespacepplx.html

How does this project relate to Facebook's other futures/promises library, Bolts?

https://code.facebook.com/posts/225525624316574/building-and...

They might look related in terms of various "idioms/building blocks" used but they cater to two completely different programming languages i.e. Objective C/C++.

I'm totally on board with futures, IF your application needs to scale to the point that blocking, threaded computation is infeasible. However, I honestly don't think that's the case for most folk. Blocking, direct-call computation is way simpler than futures.

Or use fibers (lightweight threads) and enjoy the best of both worlds: simple, blocking code with the scalable performance of async.

Fibers are fine as an idea. But all the C++ implementations I'm aware of require the programmer to mark up their code for cooperative scheduling. That's a non-starter for me.

So writing their code completely differently is a less-costly alternative?

If I compare the three alternatives:

    - Standard blocking code in OS threads.
    - Future-based code.
    - Coroutines with explicit annotation.

The latter is probably only viable for very large organizations in terms of engineering cost. The other two are doable for smaller organizations.

Can you explain why? Futures seem to be much more intrusive than the other two.

They are intrusive in the sense that the code looks much different. On the other hand, I'd guess they are less error prone. Marking up syscalls would be a constant cognitive load, since you have to remember to do it each time (or else maintain an abstraction layer). But once you've decided to use futures, that's just how you do async.

I think the language and framework are also important in deciding to use them or now. In modern C# with MVC/WebAPI it feels wrong to me to NOT use async/await/Task (the .Net version of Futures and Promises). For fairly little change in coding style you can get some big performance wins on IO code.

Unless you're writing pretty perf intensive code, part of me doubts you're going to see huge wins on something like async/await.

Futures model async tasks, there's no reason you couldn't write an executor which uses OS threads using this framework (as they say in the post...)

>we provide a flexible mechanism for explicitly controlling the execution context of callbacks, using a simple Executor interface

Am I the only one who thinks ".then(...)" code is also unreadable? Composability with ".then(...)" looks like a workaround for C++ limitations.

Why can't we have this instead:

  fooAsync(input) {
    //...
    USING_B_THREAD_POOL
    //...
    USING_A_THREAD_POOL
    //...
    USING_B_THREAD_POOL
    //...
  }

What are the disadvantages?

.then() is a hack as much as the entire async paradigm is a hack to make up for the weight of OS-supported threads used in their intended way. We have to step back and ask: what is the ideal mode of development? What does one want to do but can't? The ideal answer is to continue developing in the traditional, stackful, RAII-made-easy way where the operating system can provide a suitable natural API and runtime environment (i.e threads and kernel scheduling), and the programmer can state their intent as clearly and naturally as possible.

The std::future interface is good, but it has no truly suitable implementation. I'm not familiar with FB's Folly but at a glance I don't see it solving the fundamental problems with both std::future and callback-hell; .then() seems to just move callback-hell into one place rather than having it spread out -- it's still hell. std::future on the other hand is tied directly to the OS thread interface. Are you able to .wait() on say, 1,000,000 items at once? You'd either need 1,000,000 threads or you're bound by the slowest waiting entity in the current thread. It's definitely flawed.

The solution is to once again break down the execution context. Like threads did with processes, a new division is necessary in userspace based on stackful context switching (think boost::context crossed with boost::asio using the yield_context feature). Userspace contexts can "go to sleep" and "wake up" while being agnostic to the bounds of the thread-pool, and working fine on even a single thread. This model allows synchronous-looking programming while really acting in an asynchronous way -- which is what the obsoleted OS process/thread system offered.

"Async" and "callbacks" are really just a model of no-model -- it hasn't inverted the stack, it's basically eliminated it -- and I treat it as nothing more than temporary.

Fibers which could do async IO coded in a blocking style (co-operatively tasked) would rock so hard that one day it has to work without sucking...

I'm a huge fan of Finagle futures, which this project seems to draw a lot of inspiration from. My biggest challenge is actually the fact that Folly as a dependency is so heavyweight memory wise: I have trouble building anything on my laptop with 4 gigabytes of memory. Most of the Folly code is template code - I guess lots of template code leads to large memory compile time footprints?

Anyway - great project. I'd love to use it sometime maybe when I get a bigger dev machine.

Yes, templates are expensive at compile time (memory and time both). folly/futures only depends on a few pieces of folly/, you may find that you can use the futures headers just fine. If building libfolly is the showstopper, you might be able to comment out all the non-dependent files.

Excellent library and writeup !!

This is very similar to HPX, the general purpose C++ runtime system for parallel and distributed applications by the Stellar Group - https://github.com/STEllAR-GROUP/hpx.

I recently saw the excellent video presentation by Hartmut Kaiser on this https://www.youtube.com/watch?v=5xyztU__yys and a lot of concepts in folly futures are quite similar. However the most striking thing in HPX was that all the building blocks are serializable, and the presenter mentioned that it is so because you could serialize and move a thread to a different machine and run it there.

Kaiser also gave a pretty good keynote at Meeting C++, "Plain Threads are the GOTO of todays computing:" https://www.youtube.com/watch?v=4OCUEgSNIAY

Same talk maybe?

> Threads are heavyweight — switching threads is inefficient, they have considerable memory overhead, and the OS will bog down if you make too many of them.

I didn't realize Posix threads were that inefficient?

First, I can't see why the memory overhead would be anything more than having a separate stack and entry in the TCB?

Second, can you not just save the stack pointer, program counter and registers into the TCB, then do the reverse when restoring a thread?

Finally, I wouldn't have thought you'd have too many TLB misses either, thus leaving the only expense being trapping to the kernel to switch between threads?

Can anyone explain?

I can't explain completely, but check out http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-ma....

Relatively speaking, I don't think context switching is _that_ expensive compared to other areas you can focus on like doing smarter memory management.

I don't know exactly what they mean by threads having heavy memory overhead, but possibly they mean cache interference mentioned in that link? I'd be curious if there is an actual large memory chunk other than stack/scheduling details in play here too.

On modern cores too, there are a decent chunk of registers, including floating point registers. I've looked into timings before for embedded applications and the performance hit here isn't trivial when looking at interrupts and the like; but I'd be surprised if it was that overwhelming on servers.

Futures sounds a lot like Promises to me. Or am I wrong?

Futures can be created with promises:

- http://en.cppreference.com/w/cpp/thread/future

- http://en.cppreference.com/w/cpp/thread/promise

- https://en.wikipedia.org/wiki/Futures_and_promises

- http://docs.scala-lang.org/overviews/core/futures.html

Senders hold promises, and receivers hold futures. A promise represents a "promise" to deliver a value at some point to the holder of the future.

Yes, the two terms are unfortunately both synonymous (comparing different implementations of the pattern in different languages), and usually have specific meanings in the context of any given implementation. Often, as here, Promise is the write handle and Future is the read handle.

I've never programmed seriously with futures, and I'm a little apprehensive.

For example, let's say I type the string "123". If there's a future / promise / async whatever anywhere, we risk processing the characters out of order. We may not catch that in testing, because it's usually fast enough that it comes out in the right order.

> It is more efficient to make service A asynchronous, meaning that while B is busy computing its answer, A has moved on to service other requests. When the answer from B becomes available, A will use it to finish the request.

What if the first request is "set sharing permissions to private" and the second request is "upload this compromising photo?" Obviously it would be bad to process these out of order. How is this handled?

Are you asking "how do you do concurrent programming?" These problems aren't specific to promises.

I don't think so. Here's a concrete example of what I mean.

Say we are making a Todo list app. We have a list of Pending and a list of Completed todos, and when the user completes one, we move it from Pending to Completed. But how do we ensure other threads can't see the transient in-between state? Traditional concurrent programming might solve this with a lock:

    lock()
    Pending.remove(todo)
    Completed.add(todo)
    unlock()

This problem is very well known, and any discussion of threads will spend a lot of time on locks, queues, serialization techniques, etc. for avoiding races. Now, with futures:

    Pending.remove(todo).then({Completed.add(todo)})

We've got the analogous race condition, even if we're single threaded. But articles on Futures never seem to discuss techniques for mitigating this. Why not? Is there a Futures equivalent for a lock?

Futures are used when you want to do something asynchronously -- that is, you want the action to happen in the background and be notified after it completes. If you want actions to take place atomically, then why make them asynchronous? Nobody ever said that all code blocks of the form "A; B" should be replaced with "A.then(B)".

If you're writing in C++ with shared data structures accessible from multiple threads, you need locks anyway; futures don't change that. If you're writing in Javascript, your code is inherently single-threaded and no locks are necessary.

> If you want actions to take place atomically, then why make them asynchronous?

This is usually up to the API you are using. You may not have a choice.

> your code is inherently single-threaded and no locks are necessary

Locks are necessary in the single threaded case. See my example:

    Pending.remove(todo).then({Completed.add(todo)})

Nothing prevents another operation from executing between the remove() and add() calls, and seeing the transient state. You need the analog of a lock to prevent that. What is that with Futures?

Sorry, maybe I was unclear. Let me try again from a different angle: It's up to an API designer to come up with a sane API, including sane usage of futures/promises only where it makes sense.

In the browser world, futures or promises are an abstraction over some operation that doesn't block the UI thread, and therefore can allow some other work to happen in the meantime. In your example, if Pending and Completed provide an interface to some remote API, then a lock provides no benefit because making separate RPCs can't possibly be atomic anyway. If they're operating on local data or the DOM, then making them return futures is pointless because the work will happen on the same thread, and no other code could possibly observe the intermediate state anyway. (This is a core principle of the browser event loop: Javascript code is never preempted, it can only yield control by running to completion. Apologies if I'm repeating stuff you already know.)

In other languages, futures are more flexible because they can contain CPU-bound work that operates on shared memory. In that case, futures don't magically absolve you of the need to protect that shared memory. But if you have multiple operations on the same shared data, it once again doesn't normally make sense to decouple them with futures in the first place.

That makes sense. Thank you for your thoughtful reply.

A future or promise is not a replacement for locks and mutexes. They are intended to abstract async operations for lazy evaluation and intend to fix the problem of "callback hell" that occurs when chaining async operations and response handlers.

Futures and promises make more sense in functional languages because they don't have a shared memory model, which avoids having to share resources between parallel tasks using locks. Google dataflow programming for more insight into how lazy evaluation and referential transparency parallelize work when there's no mutable state.

> What if the first request is "set sharing permissions to private" and the second request is "upload this compromising photo?" Obviously it would be bad to process these out of order. How is this handled?

Upgrade the user wetware. Have them wait until confirmation that sharing permissions are now private before beginning upload.

I would never use your system. I expect a system to behave as advertised , not debug its internal state transitions

Futures in Scala are composable, so this becomes:

    val upload = for {
      p <- Future(setPerms(private))
      u <- Future(uploadPhoto(photo))
    } yield u

    upload onComplete {
        case Success(uploadStatus) => // do stuff
        case Failure(why) => // do stuff
    }

These will not happen concurrently unless the futures are declared outside of the for comprehension.

I guess this should be

     u <- Future(uploadPhoto(p))

right?

I'd be interested in seeing why instagram didn't implement the suggested user service as a batch process that populated some KV store. Online evaluation is great for guaranteeing up-to-date results, but the tradeoff is that you now have to bring up and maintain a fairly critical production service.

> The SU service fetches candidate accounts from various sources, such as your friends, accounts that you may be interested in, and popular accounts in your area. Then, a machine learning model blends them to produce a list of personalized account suggestions.

> This enabled us to reduce the number of instances of the Suggested Users service from 720 to 38.

We've seen huge gains from making the system as realtime as possible. Consider a new user who hasn't followed anyone yet--each new follow is super helpful in informing the recommendation system.

But how does one elegantly stop a chain of futures from executing when one is no longer interested in the final outcome of this chain of futures?

In microsoft-land where I reside, they are called tasks and we cancel tasks.

Technically all tasks can be collected in a list and then you just cancel everything in the list but I have yet to find a scenario where I've needed this.

Folly Futures support interrupts and cancellations. See https://github.com/facebook/folly/blob/master/folly/futures/...

com.twitter.util.Future supports raising interrupts that propagate 'backwards' through the chain (and across network boundaries.) This is used to implement cancellation, among other things.

https://github.com/twitter/util/blob/master/util-core/src/ma...

This is quite nice. I still wish C++11 would add some CSP style channels into the core language, with appropriate elegant syntax (maybe they could steal Go's syntax for channels).

I'm sure there is a good performance reason they haven't done this yet. I don't know enough about the design of programming languages to know all the complexities involved. I'm just idly wishing.

Can someone explain to me what does it add over http://www.boost.org/doc/libs/1_58_0/doc/html/thread/synchro... and pros/cons ? Thanks.

It is along the same lines as boost's futures implementation. We have a different mechanism for expressing thread management, born out of trial and error and Facebook engineer feedback. At the time we set out to write this boost futures were slow and buggy (1.53), and C++ standard monadic futures proposals were in very early stages (it now appears that there will be monadic futures in C++17). I do not know if boost futures are now more robust and/or more performant in 1.58.

thanks for the explanation.

Since it seems like you're at FB, do you know if anyone's working on an equivalent of FB's Swift but for C++ instead of Java? If so, I'd love an email to express interest in such a thing. Thanks in advance!

Swift uses annotations (reflection) and generates bytecode on the fly. This fits much better with modern Java development practices and tooling than source code generation. Is such a thing feasible C++, especially in an idiomatic way? Or perhaps I am misunderstanding your question.

To my knowledge, our C++ Thrift code is here: https://github.com/facebook/fbthrift

(I work on Presto and occasionally Swift at Facebook, but am not at all familiar with modern C++)

Thanks for the reply! You're right, I should have been more clear - the goal would be to autogenerate Thrift IDL at compile-time from annotations in our C++ source, to avoid having to hand write the IDL. So more in the sprit of Swift rather than a direct parallel.

They seem to be more robust, but boost::future::then() has weird design that makes it all but useless for real-life use. They return the same kind of future that std::async() does, i.e. its destructor blocks until the future is ready. In other words, you can't write "fire and forget" code:

    void run_and_report()
    {
       something_long_running()
       .then([=]{ report_results() });
    } // blocks here until the then() block executes

Has anyone used the Reactive Extensions for C++? I've used RxJs fairly extensively, but I haven't tried it any other languages. Rx solves a similar problem, but observables are more general than promises.

https://github.com/Reactive-Extensions/RxCpp

Futures/promises seem like they're basically just a limited subset of FRP (Rx, RAC) observables/streams - such that they either "next" once and "complete", or "error" without "nexting".

I suppose they also have caching built in, but that is easy to build on top of FRP.

Yeah, I think you're right on the money. Matt Podwysocki has a chart explaining how promises fit in to all of this but I can't seem to find it. Here's what he says in the RxJs documentation.

"One question you may ask yourself, is why RxJS? What about Promises? Promises are good for solving asynchronous operations such as querying a service with an XMLHttpRequest, where the expected behavior is one value and then completion. The Reactive Extensions for JavaScript unifies both the world of Promises, callbacks as well as evented data such as DOM Input, Web Workers, Web Sockets. Once we have unified these concepts, this enables rich composition."

Indeed, any good intro to Rx points this out: Observable is the plural of future/promise.

When people are trying to do fancy streaming stuff with futures and come to me for help, I generally recommend they do the streaming with Rx instead. (eg rxcpp from Microsoft)

Total strawman on the code example. They don't really need the mutex and all the complexity, but they add it so they can overstate their point -- make the most complex code you can and compare it to a simple example you code. Blah.

Another option that has been around for a number of years is: http://doc.qt.io/qt-5/qtconcurrent-index.html

Promises, C++ style?

That's what I surmised as well.

Names go against C++ convention, should be. Future<T> => future<T> makeFuture => make_future

In game development we don't use futures or similar mechanisms. We use job systems. They are much more powerful for serious muktithreading requirements. I'm not saying futures are useless at all, but we wouldn't use them where performance matters.

Reply on news.ycombinator.com