Master Concurrent Processes with core.async | Hacker News Reader

Master Concurrent Processes with core.async

braveclojure.com

131 pointsswannodette17 comments

Just one caveat to the use of core.async as a processing pipeline (as in section 6: Callbacks), make sure you're taking extra care to handle exceptions. Since the exception occurs in a separate thread, it will happen silently and will cause your pipeline to stall.

Some recommended reading for how to handle such a scenario:

http://martintrojer.github.io/clojure/2014/03/09/working-wit...

https://github.com/zachallaun/async-pipeline

https://github.com/ztellman/manifold

Exception handling is currently my biggest complaint with core.async, and I've gained a fair amount of sympathy for the idea of encapsulating asynchronous operations inside an object that can accommodate failure states, like a future (though Clojure futures don't).

There are Clojure systems with good support for handling exceptions. Zach Tellman's has an event driven channel/pipeline system in Clojure called Lamina. It offers good support for monitoring and recovering from exceptions:

https://github.com/ztellman/lamina/wiki/Probes-and-Instrumen...

Any system that is concurrent needs first-class support for handling exceptions that occur in a thread pool. I appreciate that Lamina puts an emphasis on this.

This seems a good reason to consider the Erlang model, of processes that fail early and exit on any error, with monitor or supervisor process to restart it.

Are there any Clojure libraries that do something like this? I think Akka supports this kind of design.

This is the essence of the non-problem. core.async by not choosing a default and acting as substrate allows developers to choose between a supervisor-like model in which go processes can just die and be restarted or an exception oriented one that propagates errors through channels.

This means it's less "easy" since you have to think about this up front. But it also means you are not pigeon-holed into working around canned defaults when constructing the appropriate solution for the problem at hand.

I get this viewpoint, and it makes surface sense to me. That said, essentially all of the non-hello-world CSP code, in go or clojure, that I've seen ends up trying to implement call/cast/monitor at a minimum, and frequently gen_server/gen_supervisor/gen_fsm. Seems like a problem.

I've seen plenty of Clojure core.async code that doesn't look like this at all. Also this discussion isn't considering the other reason for core.async's existence - ClojureScript - the patterns are completely different.

I think Pulsar supports this model: http://docs.paralleluniverse.co/pulsar/#error-handling.

Incidentally, in the javascript port of core.async, we are also trying to come up with a good error handling story [1]. One approach we are considering is (sort of) adapting the Erlang model, by treating goroutines' return channels as their identities. I'm researching error handling in CSP literature and prior implementations as well.

[1]: https://github.com/ubolonton/js-csp/issues/14

I absolutely love @nonrecursive's writing style. This is one of those books that are a pleasure to read and as such it's very easy to learn (and keep) the information given.

Once this is out in dead-tree form I will definitely get a copy or three (there are geeks around me who will enjoy it too).

This isn't just a very good introduction into Clojure. This is a real piece of art.

This made my day. Thank you :)

Good timing - I'm halfway through the core.async chapter in Seven Concurrency Models in Seven Weeks, which is well worth a read too...

Had no clue about there's actually a book on this topic! No more googling around differences between actors/goroutines/threads. Thanks a ton.

I'm currently also at reading it. It's definitly worth it, gives a good overview on different concurrency solutions.

However if you want to have a really deep dive into actors/core.async/threading-primitives then more specialised books might be more helpful.

If you're trying to learn clojure, read this book. Maybe even first.

Agree with what others have said, Clojure for the Brave and True is a fantastic book, was excited when I got the leanpub email this morning alerting me to a new chapter.

I tried to follow this (as a Clojure newb) and got stuck on the second code segment. Apparently nothing can find a "chan" function...

You'll need to make sure you require/refer the core.async namespace according to the preceding code block. If you have the code there, make sure you re-evaluate the file. Here's a resource on require and refer: http://www.braveclojure.com/organization/

If you still have trouble feel free to leave a comment on the site or tweet me - I'll be able to respond more quickly that way :)

Or just use Elixir.

Reply on news.ycombinator.com