Hacker News Reader

Comment by abraxas | Hacker News Reader

I fell in love with Clojure and fell out of it. It is a great language. Lisps have this inherent elegance to them especially in data transformation pipelines.

That said I found its performance still a bit on the wanting side particularly if you avoid type hints. For many tasks the performance is probably acceptable but the backend work I tend to do makes the abstract style get in the way of trying to figure out performance issues.

Also I'm coming around to the imperative style more and more. Sometimes a for loop is all that's needed and makes for a more understandable code than all those hipster maps and folds.

View on HN

What kind of backend work do you tend to do?

I am curious because I am doing real-time financial data processing with sub-millisecond latencies just fine in Clojure. Admittedly, there’s no heavy number crunching in my use case.

I work with service to service integrations. Data gets ingested into Kafka, transformed and delivered to target systems. The initial prototype used Clojure but a plain Java reimplementation ended up much faster - 2-4x difference.

I also found that Clojure is full of weird edge behavior. Sets are unsafe, and might change the type on you. Seq leaks like a sieve, so there are all kinds of places where you have to map seq, and type hinting is the worst. The moment you type hint a Protocol, all kinds of weird shit starts to happen, including the need to import the type hinted classes in the consumer (except Java.util.*), even if you don’t reference it. Oh, and if you type hint protocols wrong, it doesn’t error, but your performance remains bad.

At least it did in 2017. I haven’t used it since then.

I don't know what "sets" refers to here - idiomatic Clojure does no such thing (there are some facilities with the word "set" in them but they are all rarely used). seq is not leaky at all imo, it's a model that choose to use (https://clojure.org/guides/faq#seqs_vs_colls). Or you can choose not to use it now with transducers and use any of the seq functions and collect into a specific collection type. I don't know what you're talking about with protocols, typically you don't need to type hint them, and I think there may have been an issue with unqualified imports on referred functions but that was fixed several releases ago.

Set as in the collection. Sets are idiomatic, they even have syntax sugar for their construction: #{}.

Seq is extremely leaky, as whether or not a built in function takes a seq or a specific collection is undocumented and arbitrary. Hashmaps for example do not accept seqs when using “into”, it accepts vectors. So my code was littered with (into {} (map vec <thing>)). That’s a leaky abstraction.

You need to hint protocols if you want to go fast.

I have trouble understanding almost all of these complaints. I have been developing Clojure for two years now at two different shops and haven't encountered the issues you describe.

I have to concur that it's not easy to gather what's idiomatic and why.

It took me watching the "transducers", "inside transducers" and "reducers" talks several times, before i realized that `into`, `transduce`, `eduction`, etc are the same family of functions, providing transducing contexts.

Despite that, I have no idea what are those issues in the critique above.

To me it smells like trying to do something forcefully, instead of stepping back and approaching the problem differently, in simpler ways.

Other way to avoid such issues is to formulate the problem clearly and ask it on a couple of forums, where more experienced Clojurists can answer it. This of course takes some time, sometimes hours, but it yields great results. I've received very insightful responses directly from authors of libraries on Github issues or Clojurians Slack and from Cognitect too about some advanced Datomic query technique.

I've noticed that some ppl, instead of asking for help, sweat out some convoluted, inconvenient way to solve a problem, then decide it's the only way to solve it in a specific language or with a library in question and they just keep copying that solution and teaching it to others too and constantly complain about it... It feels to me that OP might be in a similar situation to some extent.

I worked in Clojure for 2 years in a company that had people who’d used Clojure for 4-5. I spoke at the Conj before I got a full time job in Clojure. Of course we researched and experimented, and I find the suggestion that we didn’t do our due diligence extremely patronizing.

I’m more than happy to explain the issues I had, but don’t imply that the problem is that I don’t know any better and I’m doing it wrong.

I can recommend YourKit. It's a paid tool but if you really need it, the small cost is totally worth it.

Usually the bulk of your perf. problem comes from a few hot places that are called often and the JVM needs to do reflection on those every time. Type-hint those and the performance usually goes through the roof.

I use Jprofiler and I'm no stranger to using profiles but some issues with Clojure's performance aren't fixable with a profiling session.

Any specific issues you've encountered?

A lot of issues centred around persistent immutable data structures that Clojure uses for just about everything. Yes those data structures are faster than any previous attempts at language level immutability constructs but are still much slower than their mutable counterparts. Yes, you can use in Clojure Java Collections through the interop interface but that mostly defeats the point of Clojure.

In the end I retreated back to Java but even that is wrought with a number of performance issues due to its lack of contiguous arrays of complex types. I’m toying with Go as it’s a simple language that has far fewer performance surprises than most other popular high level languages. If that turns out disappointing I’m going back to where I started 24 years ago which was C++.

Or you can also use the transient/volatile versions of data structures and functions, but I wouldn't know where to learn about these in depth. I've seen them being mentioned briefly in some official docs on clojure.org and seen them in implementations of basic functions, like `reduce` and whatnot, but I'm still not very confident using them... Also, I had no need for more performance than what I get with the immutable data structures.

Reply on news.ycombinator.com