Hacker News Reader

Comment by armitron | Hacker News Reader

core.async in Clojure is not equivalent to Go's asynchronous support. Even worse, core.async is the simplest & least-performant way to implement asynchronicity: It's just using an internal thread pool, meaning that go blocks in Clojure are not at all like goroutines in Go since they are state-machines produced by a macro that can block the entire thread when doing blocking calls. So, one has to manually "park" them by using core.async channels.

Summarizing: core.async is macro sugar on top of java.util.concurrent.

Go has actual lightweight processes that are scheduled on top of OS threads. Big difference in both design and runtime performance. Erlang too has m:n threading and offers a simpler and safer programming model (each Erlang process has its own heap).

View on HN

Re "core.async is macro sugar on top of java.util.concurrent" - yes, but also no. Java does not (yet) have a way to spin light-weight processes that can park without consuming a thread OR a way to select/alt over multiple parked ops, so being able to do that in Clojure is actually novel (and enabled by having macros that can rewrite code as a state machine).

There have been a few projects that did this in the past on Java using runtime bytecode modification but none of them are widely used afaik.

This will change once Project Loom lands in Java and provides m:n fibers.

Yes, yes in the future at some undefined point java will be better.

Meanwhile go works very well today. And you don't need to be saddled with the JVM and the very bloated java ecosystem.

>very bloated java ecosystem

Why waste time reinventing what java has already solved?

Life is too short to waste on Java? Yes, some of us really find Java that bad.

Plenty of other languages have solved the same problems in much better ways (as demonstrated by this very discussion re: asynchronicity). I'll pick Go over Java, Erlang over Go and wouldn't pick Java (incl languages running on top of) for anything today.

Why not?

The java ecosystem has solved so many problems already.

Don’t you want to just focus on the business problems?

I don't solve business problems, I solve my own problems and try to have fun along the way.

The business problem still can include things like "concurrency", which is just -painful- in Java, especially if I/O bound.

I much prefer languages that make easy stuff harder, but hard stuff easier, than languages like Java that make easy stuff easier, and hard stuff harder.

Java doesnt solve business problems.

It solves enterprise ones.

Or at least claims to.

Enterprises do business last time I checked.

The Java ecosystem has solved many business problems, for me and plenty of other people I personally know, both in small startups and in large companies. Just because you are personally prejudiced against Java for some reason, doesn't mean that its not a good ecosystem, because it is. I don't like Java the language (and certainly not the crazy frameworks), although it has improved, but the ecosystem, especially from Clojure, is fantastic.

I usually find immutability to be more important than this particular detail. Because of this my preference is usually Elixir/Erlang, followed closely by Clojure (for the potential interop with a huge ecosystem), with languages like Go and Java fairly close to eachother but pretty far down list.

I would love it if Clojure on BEAM were more mainstream.

https://github.com/clojerl/clojerl ?

You are confusing Java with the JVM.

No I'm not, there is no escaping Java when you work with Clojure, since you'll be using Java libraries and working with Java APIs all the time.

Depends on what you're doing.

Sure, sometimes a simple, crude runtime will do, but if you need really good performance and low-overhead deep observability, you might want to be saddled with something that's more state-of-the-art.

Regarding Java, sure. But in the context of Clojure, it's here right now as well, no need to wait for anything.

I also don't really understand the "bloat" comment. What do you refer too? Are you referring to the fact the JVM supports too many features?

Go's concurrency primitives are a less expressive version of concurrentML, which imo has been the nicest way of writing concurrent (but for a long time not parallel) applications the last 30 years.

Guile scheme has a parallel version of it that is implemented as a library: https://github.com/wingo/fibers/wiki/Manual

Comparing concurrent systems is hard. I would suggest implementing selecting with negative acknowledgement (nack). If I can do that nicely I know I will probably like the system.

I am confused. If I remember correctly, go's runtime uses a thread pool as well. It's not like go somehow invented a runtime that does not rely on OS threads. So in the end of the day, what difference does it make?

If you ask me, java.util.concurrent is a much better and battle tested concurrent infrastructure than the simplistic implementation in go, because java.util.concurrent is actually written by the world's foremost concurrency expert, Doug Lee. I will take that over someone else's hand rolling concurrent runtime any day. God knows how many bugs there are in your home grown concurrency code.

It makes a huge difference. You can't use blocking calls in core.async, since they'll block the entire thread. There is no switching when you do a blocking call, the thread your code runs on will be out of commission until the blocking call returns. So this means that blocking APIs are out of the question and this translates to you not being able to write linear, straightforward code. You have to use non-blocking APIs and callbacks, and rely on core.async channels to force switching between blocks (core.async doesn't have lightweight processes). This is a total nightmare to debug and forces you to bend your programming logic to support this non-optimal programming model.

Constrast with Go goroutines or Erlang processes where you write linear, synchronous code that executes as part of lightweight processes. The built-in scheduler will automatically schedule these processes on real OS threads, and switch between them when they block (either through channels or blocking calls). OS thread utilization is far more effective, debugging is a piece of cake and the code that you write is easy to read and understand.

I'm bypassing your "world's foremost concurrency expert" comments. I suggest you use the concurrency offered by Go and Erlang by spending a few hours to do tasks you've done with core.async. You will learn something and you will see how Clojure falls far short of the optimal.

Hum, what you're saying seems to contradict a lot of what I've read. We're talking about IO here. So something has to block unless we're using non-blocking IO. I think what you mean is that Go will detect the block, and will yield your execution, and manage the IO in a non-blocking way automatically for you?

Under the hood, go still relies on non-blocking IO, epoll and all. And for disk and other serial IO I think it actually allocates real blocking OS threads as well.

I agree, the automatic management is convenient, but I fail to see the difference from core.async and the direct use of async IO or blocking OS threads when that's not available like for disk IO.

Go scheduler is sophisticated and it indeed intercepts all blocking system calls.

Essentially Go runtime will run a different goroutine, if current goroutine is blocked on: blocking syscall (for example opening a file), network input, channel operations, primitives in the sync package.

http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_Desh...

I would always prefer the runtime to do this job instead of having to scan code under a microscope across functions, libraries and module boundaries to see what blocks and what doesn't.

http://danboykis.com/posts/things-i-wish-i-knew-about-core-a...

If I wanted complete control, I would use C++

I don't disagree about the convenience of having the Go runtime handle this automatically for you. I do wish core.async could as well.

That said, all your prior comments seem misleading, at least they confused me because it sounded like you were implying something that didn't feel accurate. Core.async has lightweight processes. The core.async Go macro can yield to other Go blocks and it will be really cheap to do so. You can have hundreds of thousands of them concurrently parked and cooperatively switching between each other, etc.

In effect, it means that you can achieve the same scale of concurrency in Clojure as you can in Go.

Yes, it will be trickier in Clojure, because you have to manage the async IO on your own, and you have to delegate an OS thread on your own when blocking IO is needed. In ClojureScript it won't be as much an issue since all IO is async, but still not as nice as auto-yielding by the runtime under the hood as Go does.

core.async doesn't have lightweight processes, for any sane definition of these terms. A code transformation spat out by a macro, subject to all the serious drawbacks I've already mentioned, is not a lightweight process.

But looking at the number of downvotes I've received, stating the obvious no less, I think I'll stop here. I will however take the completely nonsensical, misleading and downright erroneous responses illustrated in this thread as signaling me to steer well clear of Clojure and its community. Let's not forget that this thread started with someone claiming core.async is better than Go's goroutines.

> But given the number of downvotes I've received here

If I were to guess, I'd say it's because your comments feel like they come from a place of bad faith.

I'll give you the benefit of the doubt here.

> core.async doesn't have lightweight processes, for any sane definition of these terms

We can argue semantics if you want, but that leads to nothing constructive.

I understand very well that the underlying implementation is different. Core.async is a very clever code rewriting machinery. Go actually has a user level thread implementation, with each Go block its own stack, and the Go runtime schedules those over real OS threads. I know all that.

The implementation doesn't matter for what I'm saying though. At the end of the day, it means you can pause and resume multiple logical code blocks and cooperatively alternate between which one is given another chunk of CPU time. All while keeping memory footprint low, and with very fast context switches.

Thus you can achieve a similar scale of concurrency, and even though they achieve this with a different implementation, Go and Clojure both achieve the same end result.

While I totally acknowledge as well that the implementation Go uses allows for a more ergonomic programmer experience.

Reply on news.ycombinator.com