Hacker News Reader

Comment by tombert | Hacker News Reader

I'm a bit surprised that there didn't appear to be any mention of Clojure's built-in concurrency support outside of basic immutability. core.async gives a nice channel-based system, agents give an actor-ish system, and STM/atoms let you mutate the variable safely, without having to manually work with locks.

This is definitely a good high-level article, just something I was surprised by, since core.async is what drove me (and several other people I know) to start using Clojure.

EDIT: Just a note that I know core.async isn't built in, that was a mistake. It is a first-party library, however.

View on HN

Yes, core.async should definitely be mentioned. It's also much more than a nice channel-based system, but you have to dive in to appreciate how good it is. For example, you have to write real systems to appreciate the fact that core.async channels can be used both from "go-threads" and real threads. That's really useful when your async work involves both quickly sending responses to clients (where lightweight go-threads shine) and doing heavy I/O for database updates (where you want a thread to pick up the work). You use the same channels for all types of work, which is really neat.

It’s also worth noting that almost nobody uses agents or STM (except for some highly specific use cases, but I’ve never seen them in years), and core.async is a library, not a part of Clojure (which is a good thing, because it promotes choice and keeps the language small).

I did use agents in the past, but stopped since core.async became available: it addresses the use cases for agents in a much more flexible way. Also, I found that handling errors in agents is difficult.

As for the general use case, atoms and core.async are great tools, and I haven't needed to use refs (aka the STM) for years.

Don't the core.async go-blocks have issues if you need to block on IO heavy stuff, as in the thread-pool will be blocked until the side-effects are done? Agents don't have that problem, at least I don't think they do.

I believe all I/O is blocking unless you use something that wraps NIO. This is definitely true for core.async threads, but I'm not sure why it would any different for agents.

This is also a bit of a departure for people coming from Go or Erlang/Elixir.

Agents are run in a separate pool, and no one agent is processing at a time, so I suppose they're not "parallel", but concurrent still. I think the agents are preemptive, but I'm not 100% sure on that.

I found this comment enlightening: https://clojuredocs.org/clojure.core/send-off#example-593472...

It seems like there are a lot of parallels between agents and core.async.

Agents, like go blocks, run in a thread pool. Any blocking IO should be pushed off to a separate thread outside of the thread pool. In the case of agents, there is (send-off). In the case of core.async, there is (thread)

Yeah this is true, typically you want to keep blocking functions outside this. By default, core.async allocates as many threads as cores in your CPU.

In addition to this, I personally find core.async to deal poorly with flow control (e.g. slow subscribers slowing down the entire flow for every subscriber), and seems to not have a lot of telemetry / it’s difficult to find out what’s going on in the thread pool behind it.

I’ve personally settled on Zach Tellman’s Manifold library, which provides a much more sane abstraction and is fully compatible with core.async.

For some addition thought and material on these topics, I highly recommend watching this talk by Zach, “Everything Will Flow” https://youtu.be/1bNOO3xxMc0

Hmm, interesting — but manifold is clj-only, and I use core.async both on server side and client-side in ClojureScript.

All of the core.async operations having blocking versions (eg. >!! vs. >!) which means you can use your own threads/thread pools for any work, if needed. Basically, you create your own threads instead of using go blocks and then use the blocking operations, >!!, <!!, alts!!, alt!!, etc.

Well, they don't have "issues", they are simply not intended for I/O — but you can simply do your I/O processing in a (thread) block instead of a (go) block and use <!! instead of <!. The beauty of core.async channels is that one end of a channel can be used in a go block and another end in a thread block, and things will be fine. I use this a lot in practice.

Agents also have a fixed thread pool, and you should use send-off if you intend to do I/O.

I actually use the agents semi-often, partly because I come from an Erlang background and they're relatively easy to work with.

I mentioned in an edit that I realized core.async is a library. It's still first-party, so it's still somewhat idiomatic.

I used agents for things like polling (combined with a scheduling library) because the additional plumbing with core.async wasn't worth it. But it's pretty niche.

It was not built-in because it WAS possible, unlike other languages. So I think the argument in this specific case is mute.

Indeed, Clojure also has unfettered access to Java's concurrency primitives.

STM in Haskell is really nice to use also.

52 major open issues, 2 critical issues, and 38 others

major issues list memory leaks. https://dev.clojure.org/jira/secure/IssueNavigator.jspa?rese...

first-party?

Core async is maintained by Cognitect (maintainers of Clojure). It is very commonly used, and it has been upgraded to mesh well with the rest of the language over time (namely support for transducers).

I don't really know what that has to do with anything; every piece of software has bugs. I haven't really encountered any issues with core.async yet. I suspect you could find similar bugs in nearly any popular concurrency library.

By first-party, I mean it seems to be officially part of the Clojure ecosystem, the docs are hosted on the main site: https://clojure.github.io/core.async/

Every commonly used project has a ton of open issues. Look at React, VS Code, Docker, etc

It doesn’t mean it’s not production ready.

Reply on news.ycombinator.com