Comment by AaronFriel | Hacker News Reader

Comment by AaronFriel | Hacker News Reader

This is very, very familiar. I'm working on a library in which I'll be implementing something quite similar very soon. It actually reminds me quite a lot of the actor model - as each event loop is essentially an actor which receives and sends messages through channels. Background follows:

I'm the author of a Haskell library[1] for interfacing with HyperDex[2], a distributed database produced by some folks at Cornell.

The client-side library for HyperDex uses an event loop, and is "thread-safe" to call into as long as you synchronize access to the pointer. This is an awkward area to work with in Haskell-land, as the code I write has to deal with the intersection of the three uglies:

* Foreign function interfaces (and marshalling)

* Mutable resource management (allocating and freeing C-structs)

* Concurrency (synchronizing access to an object)

I've tried a variety of implementations. To speed up testing, I wrote a "fake HyperDex" client and test harness in which I can insert chaotic behaviors to test resilience.[3] While the code isn't as clean as I like right now, it lends itself to the smallest, most compact implementations of what I want. Each HyperDex connection pointer is paired with an event loop which receives requests for calls into HyperDex and processes them. When a response is demanded, the loop begins a busy-wait (will soon be replaced by an epoll/select on an fd) on results from the database. Each asynchronous request is an event loop too - a free floating closure sitting in memory that sends off a message and waits for responses.

GHC's garbage collector will determine when waiting on an MVar or Chan is deadlocked, and keeping everything in a ResourceT monad ensures that the whole system gracefully closes when portions fall out of scope and are unreachable.

The approach has a certain elegance to it. I am curious to hear what other people's thoughts are on such implementations though, because there are naturally performance implications.

[1] https://github.com/aaronfriel/hyhac

[2] http://hyperdex.org

[3] http://lpaste.net/101084 - a hodgepodge of code I wrote to test implementations - ill-documented, this is just for personal exploration and testing

> The client-side library for HyperDex uses an event loop, and is "thread-safe"

That is a common misconception. Or rather it is a tautalogy. No threads = thread-safe. But, it is not concurrently-modifying-data-structures safe -- which is the main painful point.

One can get just as easily tangled over a set of callbacks.

Here is a set of callbacks all started from some select/poll/epoll loop. Some call it a reactor (namely Glyph's own Twisted Python).

cb1 -> cb2 -> cb3|eb3 then cb3->cb4 and eb3->cb5

Processing starts with cb1 and ends with cb4 or cb5. Notice at some point cb2 function could result in generating an errback (eb3) which then ends up calling another callback cb5.

Understanding that the above, in a large system is just a messier, uglier concurrency structure than a thread/goroutine/task/actor is crucial.

It doesn't necessarily save you from simultaneous access to same shared data.

Imagine processing starts at cb1 and by the time it reaches cb3 (say cb2 calls some io or sleep operation), cb1 gets called again. cb1 through cb3 end up modifying some shared data (hey no need for lock, we are using callbacks remember!). Now there are two callback chains modifying shared data.

Yes you need locks and semaphores with the above just as you do with threads

For example this exists -- Twisted's own Sempahore:

http://twistedmatrix.com/documents/10.1.0/api/twisted.intern...

I had to use it, and not just for throttling concurrency, but also to protect critical data from being modified concurrently.

Asynchronous/callback/promise/future based concurrency looks really good in small examples. In large application they get messy quickly.

Threads/actors/goroutines etc are still nicer from a logical, application point of view. You can even build them on top of the same epoll/select/kqueue system calls if the language can support some kind of a coroutine structure (which for example python gevent/eventlet) is doing.

I am unsure how to map your reply to my concept of HyperDex's event loop (and C API). You are of course correct, you have to synchronize access to some object which is a pain point.

What I am doing in my implementation of a thread-safe wrapper around HyperDex is similar to what you describe at the very end of your post.

> I am unsure how to map your reply to my concept of HyperDex's event loop (and C API).

I just generalized about "thread safety" and async programming. So it was nothing specific to your particular code.

The guarantee you get when using an event loop (or some other cooperative concurrency model) is that inside a contiguous block of code you don't have to worry about any other code accessing "your" data: until you yield control, the only code that will execute is your own.

Obviously if you call a function you need to be aware of what it might do to any state you grant it access to, but even then it can only mutate state either before it returns (as in most popular programming models even without concurrency) or, if it registers a callback, at some point after you yield control.

It takes a little bit of getting used to, but this effectively makes mutual exclusion blocks the default unit of execution - and in doing so eliminates a lot of the care needed to write correct code in a preemptive model. Take a look at a well written Node.js or Twisted app - neither Javascript nor Python has concurrent datastructures in their standard library, but it just isn't an issue in most cases.

All of that being said, I think that the failure of this model is that the reason programmers tend to yield control is not because they no longer need it, but rather because you they must do so in order to avoid blocking the event loop on I/O or a timer. In other words, they need to model arithmetic, for example, fundamentally differently than they model loading a configuration file. Things get even wonkier when you start specifying interfaces without knowledge of the implementation: at some point almost any operation could require IO, even if the default implementation doesn't, so you end up registering callbacks on every line. In addition to wreaking havoc on your mental model you have to begin worrying about stack sizes, etc.

In this sense, I think well written Go has a certain elegance. The programmer knows to treat shared state as a special and dangerous case. Instead, you tend to program around a produce/consume model using channels with near zero shared state. You don't need to worry about whether a call yields to the scheduler or not, or even how many threads your goroutines map to - you just write little blocks of code that operate on easily comprehensible state.

Reply on news.ycombinator.com