Why did Clojure gain so much popularity? | Hacker News Reader

Why did Clojure gain so much popularity?

quora.com

182 pointsyogthos266 comments

I work at Walmart Labs. Clojure is super popular[1] in Walmart Labs. Clojure is fun and importantly it is a very productive language to work with. This gain in productivity comes from less state, thus easy to debug and fix, and functional approach, so the functions do what exactly what they are supposed to and there are no hidden surprises.

Less state or immutability means concurrency is easy. Easy concurrency means we can scale[2] applications with ease. I mean ease of development. It takes less work, less mental effort to take something that works for a single thread and make it work in a multithreaded and/or distributed manner over hundreds of thousands of cores. Debugging is easy too. P1 incidents for a microservices written in Clojure takes less time to debug compared to those written in other languages. So no wonder most of our mission critical services are written in Clojure.

For a complex environment where thousands of microservices are deployed on hundreds of thousands of computes, Clojure has been super successful.[3]

[1]: https://github.com/walmartlabs?language=clojure

[2]: http://blog.cognitect.com/blog/2015/6/30/walmart-runs-clojur...

[3]: https://clojure.org/community/success_stories

One big reason why running on the JVM was a success factor for Clojure was that in Java, concurrency is hard.

Concurrency is hard anyway, but in Java it's considerably easier than it could have been (I'm thinking C++) given you know that java.util.concurrent exists and have a handy reference to the relevant book. Even having 'synchronized' as a primitive keyword has its benefits... But Java wasn't born with java.util.concurrent, Clojure had the benefit of being able to follow (IMO) one of Perl's laudable principles of "easy things should be easy, and hard things should be possible" and build the language with the assumption of being on top of a rather high quality set of existing concurrency primitives.

Concurrency plus mutability is hard. And idiomatic Java works by mutating state. A language where immutability is idiomatic can be appealing even to somebody who knows that java.util.concurrent exists and has a handy reference to the relevant book.

Stealing a line from from https://github.com/l3nz/SlicedBread - "..the over 400 rich pages of 'Java concurrency in practice' show how hard it is to write and debug a good-mannered multithreaded application in standard Java."

Heh, I wasn't even thinking of that one as the relevant reference, though it's valid. I was thinking of "Concurrent Programming in Java" as it comes from the horse's mouth, but that one is also close to 400 pages.

I don't even try to think about concurrent designs too much at work and sometimes feel guilty about it. Occasionally I'll use or suggest someone use a CompleteableFuture, but there is so much code and irrelevant detail compared to Clojure's "wrap body in (future), done". On the other hand some more junior devs have expressed lack of interest in studying concurrency at all since "we don't really work on concurrent or distributed systems", at which point I have to point out that as a company we serve a giant web application with API entry points from multiple bits of hardware with clients all over making concurrent requests... The road is long.

I highly recommend java devs use the concurrent collections and avoid using the synchronized keyword.

This, using synchronized is a mistake unless you really need to use some exotic collection without an implementation in java.util.concurrent

Not every set of operations you need to synchronize can be squished into a collection operation (add, get, etc.). If all you need to do is make sure things can get and add stuff to a queue or list concurrently, well yah, that's definitely a good place to be. Can also just shove it all off on your DB

You guys hiring?

Could you speak more on the use of macros and bottom-up programming as well as REPL-driven development at Walmart Labs (if applicable)? Would be interested to know if those lisp-like attributes play as important a role as functional programming and the JVM in general.

Not OP, but at my work, they definitely play a large role. A lot of the productivity boost comes from REPL driven development and Macros.

That said, Clojure has to be seen as the sum of its parts. It's the coming together of Lisp, FP and the JVM which truly gives it its strengths.

I love how you mentioned that Clojure is fun!

We use Clojure at my work as well, and ya, everything else you said afterwards is true as well. Clojure is a great productive language. It can scale, is very comparable to Java in performance, simple to maintain, has a complete ecosystem, is batteries included, etc.

But most convincing! It's just so much fun!!

Being fun is an undervalued quality of a language in my opinion. That joy you get from using it actually translates in the quality of the work you do. So it's not an intractable thing, the fun is good for you and for the product!

This can't be said enough. At the end of the day programming is done by human beings, and it needs to be treated as a human endeavor. If I enjoy what I'm doing I'm going to do it better, it's really that simple. There is actual science to back up this idea as well http://neverworkintheory.org/2014/05/01/happy-sw-devs-solve-...

Fun is important, different language in my case (Kotlin) but learning it has reminded me what it was I liked about programming.

Day job is great but spending more than half of programming time modernising some of the worst PHP I've ever seen gets old after a while.

Kotlin is actually a pleasure to use so I look forwards to using it.

Clojure is particularly well suited - at least Rich Hickey would claim - to real world applications with unsafe or unstructured inputs. I.e. complex programs that receive or pull data from external and loosely-typed sources. A lot of web programming falls into this category. Why is it well suited to these? Because it is dynamic, and offers rich runtime solutions to validation (Spec), has immutable data structures, safe and relatively simple concurrency primitives, and has the expressivity of a lisp. It also has access to the vast array of JVM libraries which helped early adoption (but possibly hindered it in the longer term?!).

Although I love working with Clojure more than any other language it's relationship with its host is somewhat schizophrenic. Calling Java classes from carefully-crafted functional code is like pouring tomato ketchup over a lemon sorbet. Thankfully there are about 23,000 pure Clojure libraries at clojars.org to help my Clojure code stay functional.

It depends on what your requirements are. Through the years, I found it easier to write reliable code by just skipping the Clojure wrapper libraries (e.g. using Kafka directly rather than a wrapper): it's easier to "escape" into functionality the wrapper doesn't offer, and it's typically more performant as well.

To be perfectly honest, Clojure's biggest mistake, from my point of view, was to be so heavily JVM-dependent.

Yes, in certain environments Java is king and a ton of people use it, it's battle-proven, etc. But so are other stacks.

Clojure, as a programming language, sparks a lot of joy and is a lot of fun to use and reason about. Yet in reality, when working in a production environment, you're basically fighting the always-memory-hungry JVM.

Just give me Clojure running on BEAM and I'll take it over anything in this world.

https://github.com/clojerl/clojerl

Clojure for the Erlang VM

1. clojerl language: https://github.com/clojerl/clojerl

2. kapok language: https://github.com/kapok-lang/kapok

Thought the first was clojure on perl for a minute

I also had to think for a moment until I realized what these links represent. More explanatory text would definitely have helped.

ClojureScript can run on Node.js and the Browser.

"can run" is much different from "the default environment". There's a big noticeable difference.

It's not "can run." ClojureScript is a fully supported and viable way to run Clojure in production.

> But so are other stacks.

Not really. Few if any platforms compete with the JVM on performance and observability (monitoring/management) and ecosystem. It's technologically head and shoulders above pretty much all other runtimes (e.g. BEAM has excellent observability, but loses big time on performance and ecosystem). Also, if you're OK with the JVM running at BEAM speeds, you can give it far less memory.

> Also, if you're OK with the JVM running at BEAM speeds, you can give it far less memory.

I'm curious about this. How do you do it in practice? Doesn't it just crash with OutOfMemory errors when it runs out of memory?

The JVM doesn't use more memory than other virtual machines, the problem is that it doesn't release memory back to the operating system, which is why it is being configured with a maximum heap size. Also compacting garbage collectors need more memory available for de-fragmenting the heap.

I've been running a simple JVM web app on my own personal server with an allocated 128 MB of RAM and a lot of that is actually unused. You can do a lot with a heap of just 64 MB.

People aren't aware, but when running your average PHP app you need more than that with just a couple of requests done at the same time, whereas my JVM server can easily handle 1000 requests per second. Similarly Ruby or Python apps can be big resource hogs, especially when you end up setting up multiple parallel processes.

I was also surprised to find that compilation to "native" isn't necessarily efficient in terms of used memory. I've built a similar app in Haskell and the used memory ended up being higher and more unpredictable. If something like Go fares better, it's not because it is compiled to native.

All in all the JVM is actually quite efficient in terms of used memory, for a garbage collected language ;-)

> The JVM doesn't use more memory than other virtual machines, the problem is that it doesn't release memory back to the operating system

This is untrue. The default configuration is just to do it very reluctantly, but this can be changed.

I like how you think 64MB is small

I mean I have 64 gigs of ram on my personal desktop. RAM is cheap, and there are a lot of problems that can be solved by throwing more RAM at it.

Considering Hetzner (hetzner.com) offer VPSs with 2Gb RAM for 2.99 Euros per month I think it's fair to say we've left the 90s behind as far as RAM is concerned.

I'm intrigued. Was your "simple JVM web app" written in Java? What kind of JVM optimisations were used and do these also apply to Clojure?

You get an OOM if your working set is larger than the heap. Otherwise, the GC just works harder, reducing your performance. Generally, the JVM uses as much memory as you let it.

.NET has come up on par, or slightly better than, the JVM in recent years in terms of performance. Ecosystem may not be as big but its pretty well fleshed out.

At my last job, I was a C#/F# dev. Recently changed jobs and learned that even in 2018, .NET is a dirty word at many companies.

.NET-the-technology is pretty good, but for many developers, .NET is synonymous with expensive licensing and tooling (and you pretty much have to use Windows, ick).

The JVM and .NET are presently about the same capability-wise, but Java is (or at least, is perceived as) much easier and cheaper to freely experiment with. Unless you need to use C# for some deep Windows integration, this makes Java the obvious choice for anyone without sunk costs in the .NET ecosystem.

You are aware of .net core right? We're deploying services using linux docker containers. And I'm having a blast writing my macos utilities in f#

.NET is the only runtime that comes close. Unfortunately, some early design decisions (reified generics) make language interop on the CLR much more cumbersome than on the JVM. Clojure in particular makes heavy use of the fact that a Clojure map/list/set is a Java map/list/set. This is awkward on the CLR (where you must specialize the collection).

ClojureCLR exists as well, to target the .net CLR VM.

It's an official dialect: https://clojure.org/about/clojureclr

See the interop issues (https://github.com/clojure/clojure-clr/wiki/Calling-generic-...) I alluded to here (https://news.ycombinator.com/item?id=19729618)

Initial implementation of Clojure targeted CLR and JVM. Urban legend says: Rich Hickey named the language "Clojure" because it has 'c', 'l', 'r' and 'j' in it.

There's also [MAGIC](https://github.com/nasser/magic) compiler that Ramsey Nasser of Arcadia fame is working on.

I have found that quite hard to get working in various aspects (from runtime installation to package management and compatibility with the rest of the ecosystem).

> Just give me Clojure running on BEAM and I'll take it over anything in this world.

Isn't it more or less what Elixir is?

Isn't Elixir more like Ruby on BEAM?

Substantively, to me, it seems more similar to Clojure than Ruby, though it has a superficial syntactic resemblance to Ruby.

It's more like Erlang with syntactic sugar and macros. Phoenix and Rails have more in common than Elixir and Ruby do

As a Ruby dev turned Elixir dev: no. I mean, they both have `do..end` blocks, which I don't love in either language. But the similarities pretty much end there.

Here's the view of someone who went from Erlang to Elixir: https://www.theerlangelist.com/article/why_elixir

The biggest disadvantage of Elixir today - it doesn't have its own thing comparable with Clojurescript. Clojurescript is not just extremely nice compiler that gives you syntactic sugar, you can totally share code between JVM and JS and that's incredibly awesome.

I think it's the other way around: Clojure gained more exposure than past Lisps because it runs on the JVM.

But it's still definitely niche and hardly used anywhere.

ABCL runs on the JVM and is about the same age as Clojure, and isn’t anywhere near as popular.

Kawa (https://www.gnu.org/software/kawa/) is even better, and I only ever came across one other person who has ever heard of it.

Right, I'd forgotten about that! I used it as an extension language for a Java program once. I didn't think it even had a compiler, but that was 20 years ago, and it appears to have one now.

It has always been a compiler. The REPL works by compiling expressions when you type them.

Kawa was my first Lisp / Scheme and I dug it quite a bit. I wrote some wrappers for Processing called ProKawa which probably doesn't work with the latest Processing releases, but it was all very fun. Quil on Clojure is pretty good, and in general I'd say I've gotten more milage out of Clojure for sure.

A lot of credit for that goes to Rich Hickey - from what I can tell, he did a very successful job early on of 1) designing a Lisp with a unique combination of ideas and 2) publicizing it in a way that resonated with a people.

I agree, Rich has done a great job promoting and maintaining Clojure. BTW, he used to use Common Lisp. I first got to know him (via email) because of his wonderful Java <--> Common Lisp bridge.

re: ABCL: this is really a great project. I don't use ABCL very often but when I need it I am grateful to the developers.

Not just that. Clojure actually has a lot of features. It's not just "yet another Lisp on JVM". here's an excerpt from Joker page (Clojure linter written in Go):

Protocols, records, structmaps, chunked seqs, transients, tagged literals, unchecked arithmetics, primitive arrays, custom data readers, transducers, validators and watch functions for vars and atoms, hierarchies, sorted maps and sets.

Clojurescript was much easier to make because it was written in Clojure, not Java. IMO if we ever see Clojure that is hosted on other runtimes it will be written in Clojure and probably still will have to depend on JVM.

I think this is accurate (in terms of exposure, not that it's hardly used). As soon as it was possible for other languages to run on the JVM, the huge amount of developers working around JVM infrastructure that didn't like the sometimes painful experience of working with Java, started looking at alternatives.

Groovy, Scala and Clojure were the early contenders that people were exploring and Clojure seems to have won a lot of mindshare.

The thing about Java isn't the language or the JVM it's the App Servers. Enterprises have invested heavily in the App Server infrastructure for deploying JVM based code from monitoring, training, etc.

Throwing all that away to use a non-JVM language is a REALLY hard sell. Those app servers are very well done.

Running on the JVM made using Clojure, Scala and Groovy possible in places where other languages wouldn't have been. In the early 2000s, it was Java and everything else.

The .NET ecosystem was awful at that time. The programming world was essentially, Java, Perl and PHP 4 in terms of enterprise usage. People said the words "Enterprise Java Beans" without a chuckle.

People were HUNGRY for better options.

Now you've got a solid C# ecosystem, Elixir, Go, a dominant Python ecosystem, significantly more polished PHP, K8's & Docker that make automating the infrastructure around all different languages consistent...it's just a different world.

>Groovy, Scala and Clojure were the early contenders that people were exploring and Clojure seems to have won.

I love Clojure and hope it gains more mindshare, but I don't think this is accurate. Scala currently enjoys quite a lot more industry use than Clojure does, for instance there are currently roughly 15x as many postings mentioning Scala as there are Clojure (source: Linkedin.ca).

You're right. I updated my comment.

Same in London. Search indeed.co.uk for "title:Clojure" and there are on average no more than 6 unique results. Outside London the rest of the UK shows next to nothing. About 6 years ago Clojure dojos and meetups were well-attended but even then the numbers were a fraction of the equivalent Scala events. Nowadays you're lucky to get a handful attendees at London Clojure events which is a real shame as it suggests Clojure is disappearing into a very small niche.

Yes, it’s niche, but remove “title:”.

According to all the recent JVM surveys Clojure has roughly the same usage as Scala:

https://snyk.io/blog/jvm-ecosystem-report-2018

https://www.jetbrains.com/research/devecosystem-2018

It's entirely possible that companies tend to hire for Scala via Linkedin a lot more than for Clojure.

In the early 2000s, it was Java and everything else.

It still kind of is in a lot of places. My country barely bothers to even teach anything else at a lot of schools, and enterprise is almost entirely dominated by the JVM.

You are exactly right. Having worked in Clojure and talked to a lot of other devs here and across Europe, most everyone got into Clojure or Scala because management and cultural complaints left them stuck with the JVM, and at least it beat building more EnterpriseWidgetFactoryGeneratorPatterns.

Oracle and Java are the new IBM, and just like IBM, no one's really happy with them but management yells if you use anything else.

> As soon as it was possible for other languages to run on the JVM, the huge amount of developers working around JVM [...] started looking at alternatives

This is true, but...

> Groovy, Scala and Clojure were the early contenders that people were exploring

... Jython and BeanShell were much earlier than those three.

Jython, JRuby, Rhino, and Clojure are JVM languages that are syntactic copies of existing non-JVM languages, and it seems JVM developers didn't want these. Clojure has won within that group.

But the other group is what developers really wanted, i.e. languages with a Java-like syntax but with extra features added. Beanshell came first with dynamic typing. Then Apache Groovy cloned Beanshell and added closures, and Beanshell never kept up. Scala had all that but also had inferred static typing, which Groovy later added but was too late to the game to have much impact. Kotlin then came and merged the best features of Scala and Groovy, and became successful on Android.

Yeah, it's only used at weird startups like Walmart, Apple, Amazon, Cisco, etc.

To be fair, at companies that size basically every programming language is used somewhere, by some team.

However, they don't just use any language for large scale mission critical projects https://www.youtube.com/watch?v=av9Xi6CNqq4

Used like "someone used their pet language for small side project" or used like "these companies would crush if Clojure compiler suddenly stopped working tomorrow"?

I recall that Walmart Labs uses their clojure graphql server library in production, for some definition of "in production".

I don't know if JVM is a problem; the issue that caught me in my few-weeks of experimenting with the language was the lazy semantics of sequences.

I know the problem, and I've danced around this same issue in things like Haskell with strictness annotations, and I've run into the same problem in R, but I'd like to just be able to read and parse a ~1GB CSV file without blowing out RAM with a giant list of thunks.

This was many years ago now, so maybe they've fixed it, but I'm tired of fighting that particular battle.

Ya, this has been completely fixed.

There's now fully lazy as well as fully strict variants of all sequence functions.

What you want to look into are transducers: https://clojure.org/reference/transducers

Also, the lazy operations were made batching. So they are better optimized now in memory and performance.

> To be perfectly honest, Clojure's biggest mistake, from my point of view, was to be so heavily JVM-dependent.

IMHO, Clojure's use of the JVM was something of a Faustian bargain. (But net worthwhile).

Reasonable people might disagree, but there is a huge amount of value in having access to a good JIT, GC, and standard library - not to mention the ecosystem around the JVM. Keeping in mind that Clojure started out as a one-person, self-funded project, it was a very reasonable thing to borrow all of this from an existing well-established language. Maybe these things could've been reproduced, but not in the approx. two years that were initially available, and it's not at all clear that building a JIT, GC, etc. would have been the best road to making a useful impact.

What is BEAM?

Erlang VM: https://en.wikipedia.org/wiki/BEAM_(Erlang_virtual_machine)

yeah BEAM didn't google well

The same can be said about WebAssembly. But we needs to first figure out how to implement GC.

Alas, it looks like it hasn't had much love since 2014: https://github.com/the-concurrent-schemer/scm

If we look at it from a historical perspective, Clojure came about at the end of the Java and Ruby eras where running big long-lasting programs was the norm. It was about 2 years later that Node.js, Go, and then Rust came about. These languages specialized in faster startup times, which made them more amenable to a variety of use-cases (where Java and Ruby were mostly used as web servers).

If we just look at developments of the last 5 years, and in particular serverless, it's been about taking advantage of those short startup times to use in a server context. When you take that into account, these languages become even more in the lead.

I think the next lisp that has a chance to really take of will be one built on wasm.

Couldn't you just compile cljs=>js=>wasm? cljs is mostly the same language right

You absolutely can, and you can compile JVM Clojure to native binaries via GraalVM. For example, somebody wrapped up my Migratus library into a standalone migration tool recently https://github.com/leafclick/pgmig

Don’t really agree with the history.

MRI ruby had always had a fast startup time and was not originally created for the web... it’s more like a “better” Perl cum toolkit for C/unixy stuff- ironically what python became more popular for despite python not originally being written for that purpose.

Also in the old days of the web.. CGI was literally spawning a process per request.

Regardless it’s bizarre to think nodejs pioneered startup time... Ruby, lua, python, and good lord Perl would really like to have a word with you for starters.

Java was really more the exception.

And I’m not sure it’s true that these serverless platforms are spawning a new process per request ... but wow what is old is new again. Sounds like you’re describing inetd (look it up). Nothing of what you’re talking about is 5 years. Try 30-40.

You're misreading my comment. I'm talking about general trends in programming. The 2000s were dominated by bulky server environments like Java, IIS (for .NET) and Ruby Mongrel. These were not things you started up just by running a script.

The 2010s the trends changed towards languages that specialized in faster startup times. That led to serverless.

No one said that these are new inventions. Like all trends its cyclical. I'm just pointing out the trends in order to better understand where Clojure fits in.

No not misreading. Ruby mongrel was the antithesis of a bulky server environment, it was a single threaded worker host generally spawned behind something like Apache or nginx or lighttpd. The need to keep mongrel workers around was more a function of the slow startup of rails then ruby or mongrel itself.

Like I said the Java (and .NET) were the exception. As far as trends... java and .net dominate enterprise and they still do. The trend is very slow in the grand scheme.

Frankly your comment seems to be conflating language issues with framework issues with “serverless” and even scripting. I don’t think there is a cogent argument to be had about overall trends with all that. Yes lambda and serverless are a thing but half the whole world still runs on JavaEE or ASP behemoths

I always believe the exact same thing.

Immutability and dynamic typing make it perfectly suited for BEAM.

BEAM is also more suited for Clojure. Less powerful CSP and awkward component would be replaced by the actor and supervision tree.

As someone who has done some Clojure, including some smaller projects in production at a big company, Clojure running on BEAM would have been a complete non-starter.

It's really easy to introduce Clojure into a Java shop, because if things go south, you can throw away a bit of code and replace it with Java. Heck, people on other teams don't even have to know this started as Clojure code.

Asking people to learn both a new way of developing and a new VM? Too much all at once, which is why Clojure spread as far as quickly as it did.

> To be perfectly honest, Clojure's biggest mistake, from my point of view, was to be so heavily JVM-dependent.

With Clojure being a hosted language, there's nothing stopping it from adopting a new platform or expanding to other platforms or languages. It has actually already done this with Clojurescript (run in the browser or Node.js)[1] and Clojure CLR (run on Microsoft's .NET)[2].

[1] https://clojurescript.org/ [2] https://clojure.org/about/clojureclr

> Just give me Clojure running on BEAM and I'll take it over anything in this world.

That already exists: http://try.clojerl.online

It's a community maintained dialect. Its maintainer, Juan Facorro is very active. You can see him giving a talk about it here: https://youtu.be/Ow8o9Mm_N7M

I'm sure all the project is in need of are more adopters to really get it off the ground. Check it out!

JVM has two new open source garbage collector implementations that are designed for TERABYTES of heap space with 10 ms GC pause times. (yes you read that correctly)

ZGC from Oracle -- but open source

Shenandoah from Red Hat -- also open source

I know of no other managed runtime platform with the GC options and tunability as JVM.

Also JVM has not one, but TWO JIT compilers, C1 and C2. Your JVM byte code starts out interpreted. It is constantly dynamically performance profiled. If it is getting above average CPU use then it is quickly compiled by C1 into native code, and scheduled to be recompiled soon later by C2. When C2 comes along, it takes its time recompiling your code into highly optimized native code.

Suppose that YOUR function A calls MY function B. And C2 aggressively inlined my B function into your A function. Now suppose my code is dynamically reloaded within the running environment. Now your function has a stale version of my code inlined within. JVM will instantly de-optimize your code back to JVM byte code interpretation so that soon, if it is still a cpu hot spot, it will get recompiled again by C1 and C2.

Let me know of any other runtime platform that even comes close to the things JVM does.

> Let me know of any other runtime platform that even comes > close to the things JVM does.

Some of the more advanced Lisps would qualify.

No Lisp runtime currently has this quality of JIT compilers, GCs, and observability.

You should check out SBCL, it's competitive.

It isn't. Perhaps you haven't seen what the JVM can do lately. It's got JIT compilers that do this[1], GCs that are getting close to 1ms max pause times on terabytes of heap [2], and low-overhead, in-production deep profiling [3]. Not to mention all of the classic capabilities of the JVM like hot code redefinition and exceptional monitoring. Lots of applications can certainly manage well without some or even all those things, and some want other tradeoffs (like no warmup or lower RAM footprint), but these days it's hard to compete with the JVM where it's strong. The technological gap between it and other runtimes is only growing.

[1]: https://twitter.com/ChrisGSeaton/status/619885182104043520

[2]: http://cr.openjdk.java.net/~pliden/slides/ZGC-Jfokus-2019.pd...

[3]: https://craftingjava.com/using-java-flight-recorder-with-ope...

And yet it still can't even do TCO

Right. Different platforms have different priorities. But tail calls will be coming to the JVM eventually.

This is the dumbest thing in the universe (in context of a Clojure discussion) that I keep seeing parroted over and over. So you have to type "recur <args>" instead of the actual function name + <args> for the exact same effect. If you can get over that aesthetic "sin" then you have your TCO in effect. I don't even find it mildly inconvenient...and even prefer the "shortcut" syntax vs explicitly repeating my function name.

Do you have appropriate benchmarks? I would appreciate seeing the comparison.

Yes, benchmarks.

But it's not just benchmarks. It's capabilities.

He should please provide references that compare other runtime platforms' capabilities side by side with JVM's capabilities.

> provide references that compare other runtime platforms' capabilities side by side with JVM's capabilities

How many weeks work do you imagine that would be?

My point: if someone is going to assert that another platform is competitive with JVM, then not only is performance important, but capabilities as well.

While no language / platform / os / etc is perfect, JVM as a managed runtime platform is hard to beat. More than two decades of research in its JIT compilers, and multiple GC implementations with various knobs and dials for tuning and monitoring. Battle tested.

Someone mentioned SBCL. While I am a fan of Common Lisp, and generally all Lisps, I would cringe to see SBCL with terabytes of heap and hundreds of CPU cores running a serious workload -- and see how well it holds up.

Just an example: You can Google for this, but in 2012 Twitter switched from Ruby on Rails to Java. They have YouTube videos explaining their change over. Basic reasons: performance and scalability. They have to handle BILLIONS of tweets per day and route each one to multiple places with notifications in near real time.

> I would cringe to see SBCL with terabytes of heap and hundreds of CPU cores running a serious workload

That's nothing SBCL can do, but one can save an executable and start that in a subsecond without the need for a terabyte of RAM.

Generally there is a lot of stuff in the JVM which makes it less attractive as runtime for Lisp: more complex code loading via 'class loaders', lack of support for memory representation of Lisp's data types (like CLOS objects, which are more dynamic than Java classes), lack of TCO, lack of easy AOT compilation, lack of easy image dumps, lack of resumeable exceptions, ...

And in 2019 something like Eclipse still is a bit clunky to run on my 8 core Xeon - part of the reason could be a less than great GUI implementation. In IntelliJ I need to restart my IDE for every simple plugin... probably features like live-updating haven't made it yet into popular apps.

> but one can save an executable and start that in a subsecond without the need for a terabyte of RAM.

A few tens of magabytes maybe and the JVM runs a complete Hello World in about 60ms, but yeah, Clojure does a lot of work on startup. It's more of a Clojure issue than a JVM issue, though. It's true that HotSpot is mostly designed for long-running applications. Other JVMs -- like Substrate VM (AKA Graal native image, which isn't quite a JVM yet) and Excelsior JET give you AOT compilation (no warmup).

> makes it less attractive as runtime for Lisp

I think Clojurists (which probably outnumber all other professional Lispers combined several times over) would disagree. I'm sure there are use cases where you'd like to use Lisp and HotSpot is not the right choice, but it seems like those cases are vastly outnumbered by the cases where it is. Domains where HotSpot is certainly not the right choice are usually either client-side web or domains where C/C++ dominate.

> lack of TCO, lack of resumeable exceptions

Delimited continuations and eventually tail calls are coming as part of Project Loom (https://wiki.openjdk.java.net/display/loom/).

> lack of easy image dumps

HotSpot's monitoring, management and observability are almost unmatched (maybe Smalltalk is somewhat better in some regards, and BEAM in some others). Whatever you may be lacking on that front is probably offset by something probably even more valuable (these capabilities have been designed over the past decades to suit the needs of millions of developers running huge backbones).

I'm not saying that HotSpot is always the best tool for the job, but it's the best tool for the job for a huge portion of the software industry.

> And in 2019 something like Eclipse still is a bit clunky to run on my 8 core Xeon - part of the reason could be a less than great GUI implementation. In IntelliJ I need to restart my IDE for every simple plugin... probably features like live-updating haven't made it yet into popular apps.

You don't need to use a Java IDE if you want to write Clojure, although the Java IDEs are also pretty unmatched -- a matter of getting used to.

> Other JVMs

So I need to switch to exotic JVMs to get better performance?

> I think Clojurists (which probably outnumber all other professional Lispers combined several times over) would disagree

Does the number matter, since most of them have a) never used Lisp nor b) do they know anything about Lisp implementation? Things like 'no TCO' in the JVM are no question of the number of people using it, it's simply a fact.

If the JVM would be a better match, then Lisp implementations on it would have a better start-up time - like most Lisp runtimes have.

There are many reasons to use the JVM, but good support in the JVM for Lisp implementations is not one of the strong points.

> Delimited continuations and eventually tail calls are coming as part of Project Loom

Which means, after reading the linked doc, that neither resumeable exceptions nor TCO are in JVM, neither currently nor in the near future. Now the usual arguments about TCO are: a) one does not need TCO b) it makes tracing difficult c) it's a security problem d) we can do recursive self-calls e) a future version of the JVM will provide annotatations for tail calls f) using TCO would provide interop problems with JVM code... etc etc.

The fact remains: TCO is not supported by the JVM, while this capability is a part of many other runtimes.

> probably even more valuable (these capabilities have been designed over the past decades to suit the needs of millions of developers running huge backbones).

That's all great, but I don't develop huge backbones all the time.

> I'm not saying that HotSpot is always the best tool for the job

Your original claim was this: 'if someone is going to assert that another platform is competitive with JVM, then not only is performance important, but capabilities as well.'

We are now talking about capabilities like full TCO and the JVM simply does not provide it. It currently provides no TCO at all.

> You don't need to use a Java IDE

It's not about what I need, it's that capabilities like extending the IDE on the fly are still not available in an IDE like IntelliJ - why? After downloading a plugin, it wants to restart the IDE. Something which a typical Lisp system on a Lisp runtime brings for free, because of its capabilities.

> So I need to switch to exotic JVMs to get better performance?

Depends what you mean by "better". HotSpot is optimized for peak performance (which is higher than that of other Lisp runtimes), and to suit the needs of server-side development. Other VMs target other uses -- fast startup, embedded etc.

> If the JVM would be a better match, then Lisp implementations on it would have a better start-up time - like most Lisp runtimes have.

It's already a better match, because that's what most Lispers prefer. If you mean it could be even better than it is, then I agree. I also agree that it's not the best match for, say, Racket. Now, it's perfectly fine if you prefer Racket over Clojure, but most people prefer Clojure to Racket, partly because overall the runtime gives them more of what's most important to them. Would it be cool if the JVM had more features that would allow it to be a better match for Racket? Sure.

> Does the number matter

Yes, because Clojure is the most popular Lisp, the features it has are probably the ones most important to Lispers.

> Things like 'no TCO' in the JVM are no question of the number of people using it, it's simply a fact.

Yes, but how much not having TCO matters can be answered by popularity. That people use Clojure much more than they use Lisps on runtimes with TCO suggests that other things matter to them more.

> neither currently nor in the near future

Currently, no. Near future -- probably. I'm the technical lead for that project, and it's very likely that delimited continuations would arrive soon (in fact, you can already build the prototype today, and early access builds will be available in a couple of months).

> Now the usual arguments about TCO are ...

Well, as it's being addressed only now rather than, say, 10 years ago means that it's not a top priority, but judged to justify some reasonable effort at this point in time.

> TCO is not supported by the JVM, while this capability is a part of many other runtimes.

True, but the fact remains that the totality of capabilities of the JVM is considered more attractive by more people than that of most other runtimes.

> That's all great, but I don't develop huge backbones all the time.

I am not saying that you, personally, should definitely use the JVM, only that it seems that the JVM is a great platform for many languages, including Lisps, and that the use cases it targets are very desired.

> Your original claim was this ...

I believe I also said that I mean capabilities for the domain HotSpot targets. Obviously, other things may be more attractive for, say, embedded software.

> It currently provides no TCO at all.

There are other things it doesn't provide. But by comparing capabilities I did not mean one by one, but the totality of those of one runtime vs. those of the others.

> It's not about what I need, it's that capabilities like extending the IDE on the fly are still not available in an IDE like IntelliJ - why? After downloading a plugin, it wants to restart the IDE. Something which a typical Lisp system on a Lisp runtime brings for free, because of its capabilities.

No, that capability is definitely there. I'm guessing the IDE authors just didn't think it's worthwhile to use. NetBeans, for example, doesn't require a restart after downloading plugins (or, rather, rarely does).

> I'm the technical lead for that project, and we're very likely to have continuations soon.

What relationship have continuations to making Java exceptions resumeable?

> But by comparing capabilities I did not mean one by one, but the totality of those of one runtime vs. those of the others.

Not sure how you compare a totality, other than actually having a list and compare them one by one.

My feeling: you make a claim, but you have never used one of the Lisp runtimes like Allegro CL and you simply don't know what they provide and when it comes to specific capabilities, they are not important to YOU, because JVM users don't use them - given that they have no choice.

The JVM isn't the predominant enterprise platform because it is the best Lisp platform. Very few people in companies care whether it runs Lisp well or not. People have chosen it as their platform for their niche language, because of its domination in the specific enterprise market - not because it is technically or capability-wise a good platform for their niche language. Actually even though the JVM now exists for a long time, none of the dynamic languages have their best / most performant / dominant implementations on the JVM. Not Smalltalk, not Lisp, not Scheme, not Python, not Ruby, ...

> What relationship have continuations to making Java exceptions resumeable?

We don't want resumable exceptions for Java. But if you want to implement a Lisp that has them on the JVM, you'll be able to use continuations for that.

> but you have never used one of the Lisp runtimes like Allegro CL and you simply don't know what they provide and when it comes to specific capabilities, they are not important to YOU, because JVM users don't use them - given that they have no choice.

I have never used Allegro, and, as I said, some capabilities that are important to some people may be missing on the JVM. But it seems that most Lispers prefer what the JVM has to offer, which is high performance, low-latency GC, low-overhead monitoring, and a huge ecosystem.

> not because it is technically or capability-wise a good platform for their niche language.

True, but most people don't care about what's good for their language, they care about what's good for their program (and users). It seems like more people prefer Lisp on the JVM, even if they have to give up some features (and they'll need to give up fewer features with Loom).

> Not Smalltalk, not Lisp, not Scheme, not Python, not Ruby, ...

Python and Ruby are actually most performant on the JVM nowadays. Look up TruffleRuby and GraalPython. They're just very new. Even TruffleJS is almost as good as V8.

> We don't want resumable exceptions for Java.

So the Java libs I'd have to use would not support them. Not so great.

> I have never used Allegro

So the totality of your opinion of the JVM being better for Lisp comes from exactly what knowledge? Oracle marketing?

> But it seems that most Lispers prefer what the JVM has to offer, which is high performance, low-latency GC, low-overhead monitoring, and a huge ecosystem.

You mean Clojure users recruited from Java? I have never seen many Scheme, Common Lisp, Javascript, Julia, ... users on the JVM.

> Python and Ruby are actually most performant on the JVM nowadays. Look up TruffleRuby and GraalPython. They're just very new. Even TruffleJS is almost as good as V8.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Really? Looks to me that it's not particular fast.

  [GraalVM CE, Java 1.8.0_202] on darwin
  Type "help", "copyright", "credits" or "license" for more information.
  Please note: This Python implementation is in the very early stages,
  and can run little more than basic benchmarks at this point.

Haha, basic benchmarks, okay. Size '178939568' okay.

Probably the largest Python implementation in the world...

> Not so great.

Well, resumable exceptions are not so great... We really don't want them in Java (remember that Java is a language originally designed by Lispers).

> So the totality of your opinion of the JVM being better for Lisp comes from exactly what knowledge? Oracle marketing?

I've used Racket (and PLT Scheme before it) quite a bit (I was a Schemer when Java barely even existed), and Guile, and, of course, Clojure. Also, Oracle marketing is quite terrible.

> You mean Clojure users recruited from Java? I have never seen many Scheme, Common Lisp, Javascript, Julia, ... users on the JVM.

I started using Scheme in the mid-nineties. Unfortunately, I haven't seen many Scheme and Common Lisp users anywhere. Clojure is the first in over three decades to be used on any noticeable scale in industry. There are plenty of people that run JS on the JVM (probably ten times all non-Clojure Lispers), and Julia is still new.

> Really? Looks to me that it's not particular fast.

It's very fast. Here's some benchmarks with early prototypes: http://thezhangwei.com/documents/oopsla113-zhang.pdf

Beats PyPy.

> Well, resumable exceptions are not so great...

I thought they are?

> We really don't want them in Java (remember that Java is a language originally designed by Lispers).

Who of the Oak team were Lispers? Not even Gosling has ever seen or used a competent Lisp implementation - he wrote a 'Mock' Lisp without lists ;-) for an Emacs written in C on Unix. One of his many projects. Stallman later implemented a slightly better Lisp using Goslings C code as a start.

> I've used Racket (and PLT Scheme before it) quite a bit (I was a Schemer when Java barely even existed), and Guile

None of them have especially interesting runtimes. Racket now gets a new runtime based on Chez Scheme from Cisco. Chez Scheme is actually great. You should have studied that. Well, source code is now available from Cisco. It has a nice AOT native compiler.

> I started using Scheme in the mid-nineties. Unfortunately, I haven't seen many Scheme and Common Lisp users anywhere

You were too late. SUN had its own Lisp offerings in the 80s. Like DEC/DIGITAL, IBM, Apple, HP, Apollo, Texas Instruments, Xerox, ...

Yes, I know Chez Scheme, and I know about the excitement around Lisp in the 80s, and now Clojure is giving Lisp a bit of a resurgence. I assure you that Lisp still has fans in the Java team today. I also know about various cool features in various runtimes -- we monitor the field closely and learn from everyone, but as those who've built the most successful programming runtime in the history of computing, which is also one of the most successful and capable Lisp runtimes in the history of computing, and also made not one but several breakthroughs in JIT compilation, garbage collection and low-overhead monitoring, the Java team knows a thing or two, as well, especially about which features have a big impact.

FWIW, there's the language shootout game (which is downright awful, but maybe better than nothing, especially if we just want a ballpark): https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Java outperforms SBCL by a very wide margin -- 1.25-4x faster -- except in the short-running benchmarks (and except the spectral-norm test, where SBCL is 5% faster), as the game does not perform warmup, and includes Java's compilation time (you can see that no Java result is under 2 seconds).

"which is downright awful" just because you say so, and yet apparently better than anything else you know-of.

> you can see that no Java result is under 2 seconds

That's because the workloads were chosen to push Java results above a couple of seconds.

> as the game does not perform warmup, and includes Java's compilation time

How many 1/10ths of a second do you imagine that takes?

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I don't know of any other SBCL benchmarks, so, as I said, it's probably better than nothing.

I have looked at some example programs in the shootout game where different implementations use completely different low-level algorithms, including code that's very unidiomatic, so those benchmarks are of low quality. And yes, Java does have a disadvantage here (I work on OpenJDK, BTW, so I don't need to imagine).

Anyway, I’m not sure which side you’re arguing, because the benchmark game did show a rather drastic win to Java (TBH, I would have been surprised if that were not the case). I simply qualified that win because the shootout game is not a high quality benchmark.

> completely different low-level algorithms

Which programs?

> including code that's very unidiomatic

Which programs? (And don't you expect code written to be fast to be different?)

> I work on OpenJDK, BTW, so I don't need to imagine

So why haven't you told us how few 1/10ths of a second?

> I’m not sure which side you’re arguing

I don't care one way or the other, I'd just like your comment to do better than name-calling.

Alright, so first of all, to qualify my "name calling" let me say this: the benchmark game does have a lot of valuable data, that can be useful for all sorts of things. In particular, it can be used to study which low-level algorithms and language specific optimizations make a difference when comparing different programs in the same language. Of all the things the data can be used for, one of the least useful ones -- to the point of being nearly meaningless -- is comparing the performance of different languages to each other, the thing that the game's presentation emphasizes most. Why is it nearly useless for that? Because the programs were written by different people and use different low-level algorithms; sometimes the benchmarks measure compiler quality, sometimes library quality, and most often it measures the optimization skill (and patience) of the various program authors. Also, usually comparisons are made along one axis -- same program, different compilers; different languages but same algorithm and author; different authors but same language and compiler; here, all factors change with every test. Here, we have two programs presented next to each other, with differences along three or four variables, each can impact the result by tens or even hundreds of percent, and then we see a difference of, say, 40% between the two. What does that even mean?

All these things make the inter-language comparison as presented nearly useless. Crucially, the site itself doesn't even state what it is that it purports to measure or compare, like a paper without any hypotheses, claims or exposition. That alone is enough to place it in the low-quality benchmark category (regardless of the merits of the problems themselves, which I haven't looked at). But that's where most non-professional benchamrks are. ¯\_(ツ)_/¯

Now, I would guess that there is some meta-hypothesis to the game pertaining to markets, but that's not what's being analyzed or even presented.

> Which programs?

I tried to look at programs that report big performace differences and I'm looking at C++'s regex-redux, and I see it uses a library that isn't in the standard library. Or look even at "Java" vs "Substrate VM" pidigits. These are two VMs for the same language, yet the programs are completely different. I'm not sure what I could learn from that about HotSpot vs Substrate VM.

Or, the example that caught my eye when I reached the game when looking for SBCL benchmarks -- SBCL's reverse-complement, that is reported to outperform C by more than 10x. Well, not only does it use a completely different low level algorithm (doesn't use parallelism), it doesn't outperform C or Java at all, but crashes, and what appears to be some junk result is reported. Had it not crashed, it would still have used a very different algorithm.

> So why haven't you told us how few 1/10ths of a second?

Because that would be misleading. Now, I admit that what threw me off at first was Java's performance compared to SBCL's reverse-complement, but that's just a mistake and has nothing to do with compilation. Real programs can take many seconds to compile, but looking at the benchmarks, they do look quite small, so you are correct that they will be compiled rather quickly (less than a second). However, I see that Java is run with tiered compilation, and obtaining good optimization, even for such small programs, can still take many seconds (say 10 or 30). As a general rule of thumb, we always warmup for at least a few seconds, even for microbenchmarks.

It is true that the difference would often not be large compared to the inter-language differences reported by the benchmark -- and as I said above, it's unclear what it is they compare -- but they can certainly be in the 5-10% range, and maybe more. This is a big difference for what I call "high quality" benchmarks, benchmarks intended to really give a precise performance measurement. For example, Twitter has a whole machine learning system that tunes and re-tunes their JVMs just so they can get a 7% performance improvement. Even the game's own examples that you linked to and seem to say, see, warmup doesn't matter, report differences of 10-200%.

> Which programs?

Now I'm looking at Haskell's fannkuch-redux

> And don't you expect code written to be fast to be different?

Yes, and if the comparison was for the same language (and compiler) then this would be interesting, but what does this tell you when comparing different languages?

P.S

Browsing some more I see that the binary-trees programs for Java and Substrate -- again, same language -- use completely different algorithms. Not to mention that just the "Java" programs alone use such a weird range of parallelism constructs, that I have no idea whether there was any reason behind it, the programmers simply didn't know about proper parallelism constructs, or the program is so old that it was written before newer constructs were introduced.

Currently, 5 Java binary-trees programs are shown.

The program with the fastest measurements on OpenJDK did not have the fastest measurements on Substrate VM.

I understand that, but it still makes the comparison between the two pretty useless. Those are literally two compilers for the same language. I see a comparison of two compilers compiling two very different programs. What does that mean?

It means "The program with the fastest measurements on OpenJDK did not have the fastest measurements on Substrate VM."

Someone with sufficient curiousiy might choose to investigate — Why?

> It means "The program with the fastest measurements on OpenJDK did not have the fastest measurements on Substrate VM."

Does it? How do you know that the same programs were run?

Simply by looking!

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

You're right. Picking at random, "Substrate #3" is indeed the same program as "Java #3".

1) Of all the things the data can be used for, one of the least useful ones -- to the point of being nearly meaningless -- is comparing the performance of different languages to each other, the thing that the game's presentation emphasizes most.

Is that what the presentation emphasizes most?

"Will your toy benchmark program be faster if you write it in a different programming language? It depends how you write it!"

"Which programs are faster?"

"Which programs are fast?"

"Non-motivation: We are profoundly uninterested in claims that these measurements, of a few tiny programs, somehow define the relative performance of programming languages."

2) Or look even at "Java" vs "Substrate VM" pidigits.

Currently 2 Java pidigits programs are shown.

The program with the fastest measurements on OpenJDK uses GMP and something different needs to be done to make that program work on Substrate VM.

3) …but crashes, and what appears to be some junk result is reported.

What is reported? Measurements at the largest workload both programs worked.

4) examples … you… seem to say, see, warmup doesn't matter

See, for programs with runtimes of 0.24s, 1/10th of a second is a lot!

See, for programs with runtimes of 4s and 20s, 1/10ths of a second… !

5) Now I'm looking at Haskell's fannkuch-redux

And… ?

6) … if the comparison was for the same language (and compiler) then this would be interesting, but what does this tell you when comparing different languages?

That seems to be a subjective statement about your interests.

Someone else might think that neither "tell you" much until you investigate Why?

> Is that what the presentation emphasizes most?

Yes, because that's what the site shows most prominently: comparisons between languages.

The fact that the explanations to the benchmarks basically say that these comparisons mean nothing doesn't make the comparisons mean much more than nothing, especially as there's no explanation to what the comparisons do (or are supposed to) mean. So in addition to having no hypothesis, no analysis and no exposition, there's a notice saying, don't use these comparisons for so and so. Then what are those comparisons for?

Again, I don't think the game's comparisons are worse than many other benchmark comparisons out there, but that doesn't make it good. What's unfortunate is that unlike those other benchmarks, they have a lot of data that could be used for some really cool stuff (not to compare languages).

It's possible that the language competition is meant to encourage more people to submit and show how their favorite language wins, but there's no information about the market aspects.

> See, for programs with runtimes of 4s and 20s, 1/10ths of a second… !

I run a toy benchmark that's the same size as the game's, and HotSpot's C2 gives me a 30-50% boost after ~10s. It really varies by program (not by problem). This tends to be worse for programs that use a lot of parallelism and aren't configured to run compilation in blocking mode, because then the application threads compete with the compilation threads, but again, there isn't really a good rule of thumb here. In any event, it is simply wrong that C2 would never yield significant performance boosts after a few tenths of a second, even on small programs (it may be true for some specific benchmark entries and maybe even all current ones, it may vary by JDK version, and it definitely varies on whether HotSpot uses Graal or C2). Although, given all the other serious problems that render the comparisons mostly meaningless (different algorithms), my guess would be that this effect is smaller by comparison.

Still, there's a lot of cool stuff the game could do with the current programs (and historical ones), but comparing languages is not really one of them.

> … what the site shows most prominently…

Program after Program after Program.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Of course, it's really not possible to show any computer program which is not written in a programming language.

> … basically say that these comparisons mean nothing …

Obviously not: please don't put your words in others' mouths.

> … a lot of data that could be used for some really cool stuff…

Take the data and do that really cool stuff.

https://salsa.debian.org/benchmarksgame-team/benchmarksgame

> … but there's no information about the market aspects.

What do you mean by "market aspects" ?

> …it may be true for some specific benchmark entries and maybe even all current ones…

So it would be wrong to try to dismiss those measurements, based on nothing more than extrapolation about how some other toy benchmark program might perform.

> … cool stuff the game could do …

You're allowed your own personal opinion about what's cool and what isn't (and no doubt others will have their own personal opinion).

> Obviously not

It's certainly not obvious to me what they comparisons mean (is it obvious to you?) and the authors of the site don't even state, let alone support, a hypothesis. They just say what the comparisons don't mean. So unless it's a work of art, I think observers can be forgiven for thinking it doesn't mean anything.

> What do you mean by "market aspects"?

I mean a hypothesis about "free market competition" driving people to write really fast programs.

> You're allowed your own personal opinion about what's cool and what isn't

True, but I meant something that could be used in a more rigorous/meaningful way (e.g. comparisons where only one variable changes). Meaning is perhaps also subjective, but I don't even know the subjective opinion of the meaning behind the game, because the site doesn't say.

> https://salsa.debian.org/benchmarksgame-team/benchmarksgame

Cool.

Anyway, now that I've taken a slightly closer look, I realize I spoke too soon when I said that the SBCL vs. Java benchmark is "better than nothing." If, say, Java uses parallelism while SBCL doesn't, then if Java is 4x faster that may not be a "better than nothing" indication.

> not obvious to me

Program X measured less than Program Y.

> … comparisons where only one variable changes…

Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK).

> I spoke too soon

You chose to summarize data: "Java outperforms SBCL by a very wide margin -- 1.25-4x faster".

Do you mean that you made mistakes when you made your summary.

> Program X measured less than Program Y.

That's like asking what does a program do and being told, "it does what the code says." Technically true, but not what we mean when we ask. Also, I don't think that's really what the site's authors thought. After all, when I click on the Haskell page, I don't see a comparison of Haskell programs, but a comparison of the fastest Haskell program with the fastest Java program. It's not obvious to me what that means, or even that it means anything at all.

> Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK).

That's exactly right, but because that's rather tedious, I would present just meaningful comparisons rather than non-meaningful ones, so no inter-language comparisons. I'm not saying they're always meaningless, but they can't be made with the data currently available on that site. Unfortunately, we cannot manufacture (non-fictitious) meaning out of data as we like.

> Do you mean that you made mistakes when you made your summary.

Yes. By "better than nothing" I overestimated the significance of results, at least in this case. After a closer look I realized that different programs use different algorithms, and so the comparison tells me nothing about the relative speed of those languages.

> That's like…

No it is not. Program X may not have measured less than Program Y.

> … meaningful comparisons…

Earlier you complained there was "code that's very unidiomatic" but now you say "writing all the Lisp programs as if they were being written in Java" would be exactly right?

1) You seem to be contradicting your own argument.

2) Your desire for "rigorous" comparison is in-conflict with your desire for inter-language comparison.

> … different programs use different algorithms…

For some programs, for some definition of "algorithm".

> … the comparison tells me nothing about the relative speed of those languages.

What is the speed of a programming language?

> Earlier you complained there was "code that's very unidiomatic" but now you say "writing all the Lisp programs as if they were being written in Java" would be exactly right?

I may have misinterpreted your comment. By "as if they were written in Java" I meant using a similar algorithm. Alternatively, there could be a rule about idiomatic code (enforced by experts), which would compare very different things, but would also be at least somewhat meaningful.

> For some programs

I don't know. Whenever someone refers me to a result on that site that looks weird, I see very different algorithms.

> for some definition of "algorithm".

For a very well-accepted definition of an algorithm in the context of real performance comparisons.

> What is the speed of a programming language?

I'm not sure (although perhaps I could come up with some definition given enough thought), but I didn't make a website that compares languages based on performance. In fact, I said that the website is totally meaningless. But the people behind it may have something in mind, and if they do, they're not telling us. All I can do is think of ways to use the site's data in ways that can at least be meaningful.

> … a similar algorithm.

Will someone else look at those similar algorithms and "see very different algorithms"?

> … a result on that site that looks weird, I see very different algorithms.

Do regex libraries for different languages in-fact always implement the same algorithms?

> What is the speed of a programming language? I'm not sure…

You did say "Java outperforms SBCL by a very wide margin -- 1.25-4x faster".

Saying "… so the comparison tells me nothing about the relative speed of those languages" suggests you are looking for something that tells us about "the relative speed" of those languages?

> Will someone else look at those similar algorithms and "see very different algorithms"?

I would think so.

> Do regex libraries for different languages in-fact always implement the same algorithms?

I don't know. But I do know that when something is presented as an answer, it's good to know what the question is.

> suggests you are looking for something that tells us about "the relative speed" of those languages?

I can think of a couple of ways the two can compare. One is the relative speeds (and other metrics) of programs of a similar nature, written simple idiomatic code. This gives a general "feel" for what kind of workloads different languages may be suitable for. Another is comparing the performance of the compiler/interpreter on some microbenchmark of the same algorithms, preferably written by people who coordinate, and preferably without mixing too many variables (e.g. measuring parallel scheduling separately and regex separately). Either one of these is of some interest.

The benchmark game does neither. If there is some idea behind the site's comparisons, it's certainly keeping it a mystery.

> … relative speeds (and other metrics) of programs…

So when you say "the relative speed of those languages" you actually mean the relative speed of programs?

> … programs of a similar nature … preferably without mixing too many variables…

Seems vague, no-longer rigorous.

We can make mystery for ourselves — no one can force us to understand.

> So when you say "the relative speed of those languages" you actually mean the relative speed of programs?

No, I mean the relative speed of programs with some carefully chosen relation between them. For example, there is a big difference between "relative income of individuals" and "relative income of social groups," even though the latter could be reduced to the former. Whether a comparison of individuals is meaningful as a comparison of groups depends on how the individuals are chosen and aggregated. Clearly not every choice of individuals would make for a meaningful comparison of groups and not every choice of programs makes for a meaningful comparison of languages. The choices made by the game do not seem at all meaningful for the comparison of languages, but if there is some rhyme or reason to the choice, the website doesn't say what it is.

> We can make mystery for ourselves — no one can force us to understand.

True, but when a website chooses to present data in some aggregate way that seems to defy any familiar forms of aggregation and reasoning without so much as stating what the meaning of the presentation is, it is itself mysterious. I agree that it is possible that the comparisons are not worthless, but without any statement of a hypothesis, it is the most reasonable conclusion. I am sure that the authors of the website would agree that if they don't even attempt to say what the meaning of the comparisons is, even in their own minds, it is reasonable to assume it means nothing.

And 2 days later, way down-thread ( https://news.ycombinator.com/item?id=19766017 )

> > So when you say "the relative speed of those languages" you actually mean the relative speed of programs?

> No, I mean the relative speed of programs with some carefully chosen relation between them. For example, there is a big difference between "relative income of individuals" and "relative income of social groups," even though the latter could be reduced to the former. Whether a comparison of individuals is meaningful as a comparison of groups depends on how the individuals are chosen and aggregated.

You can choose which program measurements to compare.

> > We can make mystery for ourselves — no one can force us to understand.

> True, but when a website chooses to present data in some aggregate way that seems to defy any familiar forms of aggregation and reasoning without so much as stating what the meaning of the presentation is, it is itself mysterious.

You have not shown that is what that website does.

I agree that it is possible that the comparisons are not worthless, but without any statement of a hypothesis, it is the most reasonable conclusion.

Yet another tendentious assertion!

Program X measured less than Program Y.

I am sure that the authors of the website would agree that if they don't even attempt to say what the meaning of the comparisons is, even in their own minds, it is reasonable to assume it means nothing.

Once more, you are putting your thoughts and words in others' mouths.

> You can choose which program measurements to compare.

Yes, but the website also chooses, and it is the website's choice that I think is bad. Also, it doesn't seem like the game even has comparable programs in some/many/most cases.

> You have not shown that is what that website does.

How can I show what a website doesn't do? If you think they say what the hypothesis is, show me. I haven't been able to find it.

> Program X measured less than Program Y.

First of all, when something means nothing other than itself, we say it has no meaning. If the comparisons on the website are meant to suggest nothing more than just their own definition, then they are meaningless.

Also -- come on. The website presents comparisons for specific choices of X and Y. What is the meaning of the choice and the resulting comparison? Your position that it's not meant to mean anything (other than itself), sounds disingenuous because the website still makes some very specific choices that seem to hint at a meaning (e.g. why not compare the second fastest programs between languages instead of the fastest?) yet the meaning is never explained.

Anyway, we're going in circles.

> … comparable programs…

For someones definition of "comparable".

> How can I show what a website doesn't do?

You can try to show your claim — "… chooses to present data in some aggregate way that seems to defy any familiar forms of aggregation and reasoning…" — has some basis in reality.

> … when something means nothing other than itself, we say it has no meaning.

That may be what you say. Show why we should think that's what people interested in "Why did Clojure gain so much popularity?" would say.

> … why not compare the second fastest programs between languages instead of the fastest?

Inexplicable! Or obvious: there would need to be two programs for each, rather than one.

> Anyway, we're going in circles.

I agree, you repeat your assertions without establishing anything.

http://www.chiark.greenend.org.uk/doc/cmucl-docs/compiler-hi... is mostly applicable to SBCL last I checked. It gives an idea of what kind of cleverness the compiler can undertake.

Then why is Clojure implemented on the JVM?

As Rich Hickey says: because it has _reach_

The JVM is widely used.

ClojureScript on JS is because JS also has reach.

ZGC is targeting 1ms max pause time now.

From what I recall of Clojure's launch and the reactions I heard from people, the JVM aspect might've been 1/3 to 1/2 of the attraction to CL people: Java was more widely popular at the time, and Clojure said "here's our angle to get more people paid to Lisp".

My self-interested perspective on Clojure today (as an expert in a couple other Lisp niches) is that I'd be happy to learn Clojure if someone would pay me to, but it's not relatively attractive to me now, in the ways that a Lisp is more attractive than a non-Lisp. (Nor in the way that Rust is in some ways more attractive than C and C++. I've been a senior developer in those, and Rust is sufficiently different, and might open some new doors.)

I’ve played with LFE (http://lfe.io) and found it quite enjoyable, but it isn’t Clojure and requires some understanding of Erlang-and some getting used to.

Interestingly, Clojure is associated with the highest salaries worldwide[1], and is used by developers with the most years of professional coding experience[2], according to the StackOverflow Developer Survey 2019.

[1] https://insights.stackoverflow.com/survey/2019#technology-_-...

[2] https://insights.stackoverflow.com/survey/2019#developer-pro...

So what you're saying is, if you look at a language that is only used by people with many years of professional experience, then salaries will be higher compared to a language used by many entry level learners?

I think Clojure is awesome and have used it for some projects but I also think that causal relationships in the SO survey are often completely misunderstood.

> So what you're saying is, if you look at a language that is only used by people with many years of professional experience, then salaries will be higher compared to a language used by many entry level learners?

I think in practice most less experienced developers cannot justify investing their time in such a niche programming language as Clojure. It's not like Clojure developers are paid best because they are the most experienced. It's because they could afford the time to master a programming language with very little demand on the job market.

These surveys seem (increasingly) biased towards certain types of development. Specifically, web development. I couldn’t find a single language I regularly or irregularly use on this page (the closest thing I could find is Emacs). Also the sample size for Clojure developers is smaller than other languages. It’s not obvious to me why the bar should be raised at a certain point (e.g. letting in slightly niche Clojure programmers but not other more niche things).

I don’t want to imply that these surveys are rubbish, just that I do not think they give the whole picture

The Quora article mentions that clojure took off in the fintech industry. People get paid a lot in this field, so if there is some way to control for this niche, the stats will be more accurate.

Well, they are listing what are obviously the most widely used languages in the industry - C, Java, C++, SQL, JavaScript, R, HTML&CSS, Python, C#, Go, Bash, Assembly; and a few of the better known but more niche languages - Ruby, Swift, Objective-C, F#, Erlang, Rust, Kotlin, TypeScript; and still a few others.

Together, these cover cloud programming, web, embedded, desktop, mobile, OS, networking, games, machine learning, data science and probably quite a few other industries.

The only languages I can think of that see quite a bit of use and that are missing are Lua and TCL; and some of the more obscure ones that would still have been interesting are Ada, some MLs, Haskell, and CommonLisp.

Why do you believe that this list is biased towards web development?

And, out of curiosity, what languages are you working with? By your mention of Emacs, I'm thinking maybe some kind of Lisps?

Absence of syntax! I really wish every language were a lisp. Like, Rust, the lisp, Swift, the lisp. Who needs rules of precedence and associativity and statements vs expressions and all the other stuff that languages that aren't lisps impose on you? Just stick an expression in some parentheses! And then put that in another expression!

Absence? There's absolutely syntax in Clojure.

1) Lexemes have a specific structure.

2) Core data types have a specific syntax (representable with a grammar).

3) The data types used to represent programming constructs also have a specific set of syntactic rules by which they are assembled.

It's a different kind of syntax than the usual - more raw/direct, perhaps - but it's a misconception to say there is an absence of syntax. A big part of the reason I mention this is that the nature of Lisp's syntax is one of its biggest strengths.

Fair enough. I was aiming for a non-technical use of the term "syntax" more or less tracking the cognitive burden of remembering and parsing out the relationship between symbols for the programmer. Like, putting s-expressions in the front seat feels like a much lower cognitive load to me than, for example, all the stuff you have to think about writing, or, god help you, reading, pointfree haskell or line noise perl

Yes, there is syntax, but there is a lot less noise and syntactic bureaucracy compared to other languges.

Yes please. Like the sibling comment says, it's definitely a syntax, but what a lovely syntax it is! Like a master key unlocking program structure. When I'm studying other functional languages, I find that the best way to make sense of their syntax is to figure out where the parentheses should go.

(find (most people) difficult (without (infix operations) (to expressions write)))

There’s definitely a learning curve, but it’s not a terribly steep one, and most minds with programming aptitude don’t seem to have much trouble with it after that.

It’s similar to learning declensions in Russian/German/Latin: weird and seemingly pointless to English speakers, and then usually fully internalized after a few months, with better intuition for costs and benefits of syntax vs cases.

I’d argue that the Clojure learning curve is significantly shorter than that also.

The best criticisms of Clojure (and I’ve written a lot in it) have nothing to do with parentheses, but rather the difficulties of grokking other people’s dynamic, untyped code.

Worse than untyped code is undocumented code.

Some programmers think types obviate the need for docs. Some programmers think concision obviates the need for docs. Except in the most trivial cases, they're both wrong.

My gripe with other people's code in Clojure is less understandability in terms of types and more in terms of style. A lot of major clojure libraries seem to use a really indirect style that brings a bunch of OO-style concepts back in. Like, I'm thinking of libraries like Component and Mount whose authors seem to have been aiming for a dependency injection kind of experience and end up creating a bunch of free-floating state, where, for example, you use a template to scaffold a simple webapp up, and it's surprisingly hard to figure out where in the code it's talking to the database. This actually caused me to leave the clojure ecosystem eventually.

If you use "The Pure Function Pipeline Data Flow",it only need Unix-style pipes and functions. you won't have to use concepts or patterns of OO&FP, and the parentheses are less than any programming language ;)

[The Pure Function Pipeline Data Flow](https://github.com/linpengcheng/PurefunctionPipelineDataflow)

Hi Lin Pengcheng, do you have an example of making side-effect calls with Pure Function Pipeline Data Flow method? e.g. database query.

A series of functions in a ->> block can only have one function with side effects and can only be at the end.

Exception handling is included in the function of side effects, and the exception is handled as actively as possible. In management, if an exception is encountered, always throw it to the superior, who will be fired:-)

you can do it like ring, treat client and server as db, query each other.

yes but (takes only (a (of bit practice)))

In 99% of programming languages, infix operations are only used for arithmetic -- and more than 99% of the programming I do doesn't involve that. Even the numerics software I've written is >90% non-numeric parts.

Arithmetic doesn't even look right in programming languages with infix operators. Go to the "Chalk is still one of the primary tools of mathematicians" article here, and try typing the inequality in Java. Mathematicians aren't writing "Math.floor()" and "% 2" on their blackboards.

I think you're optimizing for the wrong thing. Arguing that we need infix operations for 1% of our programs, so that arithmetic looks like how we entered it on a TI-81 or Apple II, is a false economy. It was a good tradeoff 25 or 50 years ago, but the majority of programming has moved past writing programs in terms of arithmetic on registers.

That said, there is a nifty macro [1] which adds infix support to Clojure, if you really want that. It even supports symbols like "√", unlike any other language I can think of!

[1]: https://github.com/rm-hull/infix

> In 99% of programming languages, infix operations are only used for arithmetic

I've been spoiled by Scala, but certainly most languages allow a "subject.verb(object)" style for general-purpose code, and that reads a lot better (particularly for those looking from the business domain) than "(verb subject object)".

> It even supports symbols like "√", unlike any other language I can think of!

Heh, again I'm spoiled by Scala, where √ is just another method/operator (there's no distinction) name if you want it to be.

> most languages allow a "subject.verb(object)" style for general-purpose code, and that reads a lot better

Why do you think this?

Programming isn't natural language, and I don't even know what the "subject" or "object" is in most cases.

> Programming isn't natural language

It's not, but it should be close; readability is important, and particularly when you want to express domain logic it's important to be able to hew closely to the actual language of the domain. Arithmetic is the most obvious example of this but it's just as true for other domains.

You're right readability is important, and Scala fails terribly in that department. I worked with Scala for around 6 months or so at one point, and I was still constantly looking things up after months of using the language. There's simply too much syntax and weird quirks in the language that it becomes impossible to keep it all in your head.

Not my experience - I found Scala's syntax is a lot more consistent than most mainstream languages (though, yes, it certainly doesn't go as far as lisps). Operators are just normal methods. Rather than complex rules about which constructs require braces, everything permits an expression or you can put a braced block of statements anywhere you could put an expression. Rather than a bunch of different special case syntaxes for collections / async / possibly-absent values, you have a single lightweight syntactic sugar (for/yield) that translates directly into plain old method calls. There are some libraries out there that make poor choices of method name, but that's about all.

My experience with Scala matches this pretty closely https://www.reddit.com/r/scala/comments/4246qc/heres_why_sca...

That's a strange set of complaints for a clojure advocate, since most of what they're complaining about is not too much syntax but too little - no mandatory braces, mandatory brackets, mandatory types. Indeed "a -> b" for "(a, b)" is not syntax or even a macro, but just a function.

_ is genuinely costly syntax, as are by-name parameters (though less costly in an age of IDEs). I consider _ to be worth its while (indeed I miss it a lot when writing lambdas in any other language); if you're going to have syntax in your language at all then it's one of the most general-purpose, effective constructs I've seen.

The problem isn't whether there's too much or too little, it's that the syntax is incredibly inconsistent. There are a million ways to do anything, and you have to juggle it all in your head. Consider stuff like this:

https://twitter.com/pembleton/status/1116343189437394944

A new line can be significant syntax in Scala!

With s-expressions, all of this mental overhead goes out of the window. I have regular boring syntax where pretty much everything works the same way. It's incredibly liberating. On top of that, s-expression syntax facilitates structural editing, where I'm manipulating the code tree semantically as opposed to thinking about moving lines of text around.

Scala has three styles, OO, FP, Python. When learning, people can find their comfortable style. But at work, any style of preference must endure the other two styles.

People use non-arithmetic infix operators in Haskell constantly. It's great.

It's also pretty easy to add prefix operators in e.g. Haskell or ATS (for something like square root)

This sounds horrifying.

Same here, after working with s-expressions I realized that you don't need to drown in syntax to have a flexible and expressive language.

Complex syntax introduces a ton of mental overhead, and it's a constant source of weird and hard to debug errors because code often does something subtly different from what you think it's doing. S-expressions enforcing a consistent and explicit syntax frees up a ton of mental space for me.

Amen. S-expressions allow you to navigate the code through your editor, and everything is consistent. Semicolons, commas, and curly braces are major cognitive overhead.

Or, rather, a single simple syntax that applies throughout the language.

Did it though? Most of the time when I mention Clojure, people think I mean “closures”.

Our whole stack is Clojure/Clojurescript. Clojure has helped reduce bugs and improve stability across the board.

I enjoy writing it everyday and wish more people would give it a shot.

Compared to lisps before it, yep it did.

I don't think Clojure needs to boil the ocean to be successful. We already have a community of many thousands of developers with lots of very smart people in it. At this point there's enough mind share to keep it going indefinitely.

At the same time Clojure benefits from being a hosted language. We don't have to build everything from scratch, instead we leverage two of the biggest ecosystems and reap all the benefits for free.

It's still a very niche language to be sure, but from what I've seen there has been a lot of commercial adoption in the last couple of years. There are new a few consulting companies, such as JUXT and Metosin, that are focusing exclusively on Clojure consulting. A lot of the libraries in the ecosystem are now maintained by these companies as opposed to individuals. There are also efforts like Clojurists Together for funding Clojure projects sustainably. These efforts would not have been possible without growing commercial use with companies willing to donate money.

So more companies using Clojure directly leads to a better ecosystem, and that in turn makes the language more attractive. The state of tooling and libraries is dramatically better today than it was even a couple of years go. I'm excited to see how the ecosystem grows this year.

One of the projects I'm most excited about is https://cljdoc.org/ which aims to provide a one stop shop for polished Clojure library documentation. I just saw a presentation about it at Clojure North, and I was very impressed with where it's at already.

Thanks for your reply. I am very glad to hear this! Cljdoc looks great as well - much needed :)

Btw, thank you for all your open source projects and comments on various GitHub repos.

Thanks, glad to hear it's coming in handy. :)

Clojure builds on Lisp that its creator wanted a modern functional programming language that runs on the JVM. Clojure also has immutable data structures Link to Githut programming language popularity on Github, compare the number of open issues per repository for different programming languages. Then you will find that Clojure, Lisp and Haskell has relatively fewer issues compared to other more popular languages. https://githut.info/

Lisp is used in AI. https://medium.com/ai-society/the-lisp-approach-to-ai-part-1... https://ai.stackexchange.com/questions/2236/why-is-lisp-such...

> But that's not how programming being taught and mentored today - it's more like "steal it from StackOverflow, adjust it to your needs. Do what everyone is doing. It's okay to follow the hype".

Oof. So true, I hate this field. But I guess you could say similar of almost anything. Maybe I just hate humanity.

Soft-misanthropy is healthy enough, just as long as you don't go full-misanthrope. I hope.

You left out the immediately following bit, "Experienced software developers don't chase the hype". I cringed. But maybe if my interpretation of the overall point is correct, I can agree with a point that Clojure attracts grumpy programmers who have Seen Things and gives little lip service to beginning devs.

Why cringe? Hype creates burnout. The churn is absolutely untenable in the UI space right now and hasn't shown signs of slowing down anytime soon.

Where I work we have 3-4 different ways of accessing our monolithic backend, and on top of that 3-4 different ways of accessing and manipulating application state.

It drives me up the fucking wall.

I'm not sure I agree with the little lip service statement. I feel like the feelings are most likely mutual.

Stable systems that don't need constant heroism and don't diverge much from original choices made at the architecture level don't make for good places for juniors to make mistakes and learn from them.

I cringed because it's not my experience that experienced devs are immune to hype, it's an orthogonal characteristic. I also find it somewhat logically inconsistent for there to exist a push for teaching/mentoring The Way of the Hype if seniors haven't bought into it themselves.

For me it was the draw of Datomic and DataScript. Clojure is a neat language, but I feel like how we store, retrieve and transmit data is vastly more important.

> ...how we store, retrieve and transmit data is vastly more important.

Agreed. Data storage and transmission formats have the potential to dramatically outlast the modules where they originate. This is in large part because these things represent commitments to others, where the internal design (or implementation language) of a specific module really doesn't. So, these days I tend to be much more interested in the design at the interface level than anywhere else.

This works both for and against languages like Clojure. It works for it, to the extent that fewer people have to care what you use when you're building the innards of something. Against it to the extent that it can put a potentially relatively low bound on the gain that can be realized through the use of better (?) tooling like Clojure.

(There's also something to be said for the fact that languages like Clojure tend to snowball in my experience better than other languages. A single program written in Clojure is nice, but it's in the ecosystem/network effects where it really pays off... and that takes a lot of time to see. More than most people take.)

Take a peek at Crux from Juxt. Really exciting times happening on the data front.

link to the repo https://github.com/juxt/crux

One day old, 4 comment HN discussion of it: https://news.ycombinator.com/item?id=19714073

I'm doing a series on concurrency models at the moment for personal professional development, and I recently reviewed Clojure and its native ability to separate identity from state. It's a very properties-based and specifications-based language, and the thing about properties and specifications of languages are that they're nigh impossible to retrofit at a later point in time. Other languages need to have external tooling or frameworks on top for things Clojure supports out of the box, and even then those frameworks may not be as reliable as something guaranteed on a language level.

It's actually a pretty sizable moat. Python broke encodings between 2 and 3 and split the community for at least a decade. Some companies will likely never be able to migrate from Python 2 to Python 3. Now imagine breaking now `dict` works.

https://bytes.yingw787.com/posts/2019/01/26/concurrency_with...

I attribute it to two things. It's the least lisp-y of the lisps meaning it came with so much that you didn't have/want to make a (different) dsl on every project. The second is the jvm: partly from interoperability but even more so from being just another jvm language with packaging/deployment and some ops already paved.

Rich Hickey gives the best talks. It's that simple.

Was that a pun? ;)

Add onto the very real facility for being a highly reliable and productive tool for doing data acquisition, transformation and distribution it makes programming FUN!

Clojure suffers from a lack of ambition, at least as far as adoption is concerned, and I think that's why many leave the community. Rich Hickey is more than happy with Clojure's current reach which says everything when you look at Clojure's abysmal level of adoption. It could be argued that Matz and Rich are both similarly idealistic language designers but Matz recognises the danger of Ruby being left behind without improvements to the language and Ruby's community is already much bigger than Clojure's.

Although there's Luminus web frameworks are generally frowned upon in Clojure-land which concedes mindshare to other languages which recognise the importance of frameworks and killer apps. With so many languages competing for mindshare it is that much easier today for a language, however original and superior, to be sidelined by paying customers. For developers looking for a language which will also help pay the bills Clojure is unfortunately not an option.

Well, its got a chicken and egg problem here.

The chickens, experienced Clojure devs, don't need a framework, and are easily able to do whatever they want and use Clojure to pay their bills. Thus they're not motivated to build frameworks.

On the other hand, the eggs, devs unfamiliar to Clojure, find it difficult to get anything done in Clojure, and are looking for frameworks that abstract away some of the expertise needed to manage a full Clojure web app. They're hoping they can start using it to pay their bills before they become an expert in it, through the use of a framework.

I'm not sure how the future will unfold here.

The way we build web apps is changing away from back-end frameworks in general. Things like micro-services are a good example of that.

Traditionally, web was all about serving static documents, and server side rendering model is a perfect fit for that. However, nowadays we also have web apps that are highly interactive.

This requires moving away from the thin client model. Server-side rendering approach is inherently at odds with that because HTTP is a stateless protocol.

So, the modern way to write web apps is to have a thick client that manages things like views, business logic, and state in one place. With this approach you no longer need a complex framework on the back-end because vast majority of the complexity moves to the client. The server is mostly responsible for things like data governance. The other driver here is horizontal scalability. It's much easier to scale small stateless services than a giant monolith.

Clojure started getting a lot of use in the wild right around the time when SPA model started taking off, and I think that the ecosystem reflects that. We tend to have fairly light weight back-end architecture coupled with a thick client on the front-end.

Moving business logic to a js client looks like a crazy gift to any ill intentioned actor.

Not sure how that's the case since authentication and data governance is done by the server. This is just the difference between thin and thick client models. Any app on a phone is a thick client as well, and uses exactly the same model as SPA.

> Clojure suffers from a lack of ambition

it already has reached an unprecedented level of adoption for a Lisp used in commercial software. But I agree with you - it would be nice to see it grow faster and bigger.

From technical evaluation - Clojure has an amazing ecosystem. But having technologically superior toolset does not guarantee success, nimlang is a good example.

What Clojure needs today is better marketing. It is fashionable to blame Cognitect (basically for anything), but the truth is - they are not a big company. Even if they really wanted it - they don't have enough people to drive it to wider success.

There are a few large companies using Clojure in production today, companies like Apple, Walmart and Cisco. It would be nice if they recognized the importance of Clojure and put some effort to evangelize for it, support initiatives like ClojuristsTogether, sponsor Clojure events.

> For developers looking for a language which will also help pay the bills Clojure is unfortunately not an option

From my own experience - the reality is the opposite. There are too many people interested in Clojure, but not enough developers with real Clojure experience. And companies using Clojure desperately want experienced Clojurists and often willing to pay more. It's disproportionate - we need more companies using Clojure, willing to hire juniors. We need to improve the funnel, making it easy to get into Clojure and become an expert fast.

> unprecedented level of adoption for a Lisp used in commercial software

if one ignores the 80s / early 90s of the AI boom

I thought that Clojure was losing its buzz in recent times. Perhaps it is that even though the core language is elegant its ecosystem of leiningen, maven for dependencies, etc, and the whole brittle emacs-cider-nrepl chain is detracting from its core strengths.

And poor startup times, and unhelpful error messages (though I hear there has been some improvement in recent versions).

I can't speak about lein or maven, but it is in no way a requirement to use emacs as your editor. In fact, there is really interesting work going on in other editors like Atom with proto-repl[1]. For the web, you can spin up a Gorilla REPL[2] which basically gives you a Jupyter notebook but for Clojure.

[1] https://atom.io/packages/proto-repl [2] http://gorilla-repl.org/

I thought the emacs tooling depended upon leiningen.

No. It depends on a library called nREPL, but you can start that in multiple ways (the other candidates being tools.deps and Boot)

No it doesn't. It depends on nRepl.

Personally, I find maven repos and lein to be one of the better dependency/build ecosystem of any language. So I disagree with your premise. But, if you don't like them, there's a lot of alternatives.

For dependency management, there's now an official one, maintained and bundled with Clojure, called `tools.deps`. It supports multiple providers, currently maven repos, git repos and local folders.

For build tasks, appart from Lein, you can use Boot or Gradle. But there's a new trend now to rely on individual Clojure programs as the foundation for build tasks, which are then packaged and distributed through `tools.deps`. You can see their full list here: https://github.com/clojure/tools.deps.alpha/wiki/Tools

In the ClojureScript world, there are less build options, but `shadow-cljs` has emerged as a replacement for Lein, and gained a lot of popularity, because it integrates transparently with npm.

Certainly the pace of growth has slowed, but the language is still growing at a lessor pace. I would say Leiningen/Maven is a major strength of the clojure ecosystem. However, if you do not like Leiningen, now there is now an "out of the box" solution to manage dependencies and construct the classpath. https://clojure.org/guides/deps_and_cli

In addition, you do not need to use nrepl. Clojure has a socket repl and a socket prepl as of Clojure 1.10

That sounds better, perhaps now Clojure can outgrow the "batteries-not-included" feel.

Amen to Emacs-Cider-nREPL hell. The number of hours/days I've wasted over breakage between these three. I would also add the obsessions with boot and elaborate "quickstart" guides to Clojurescript which eschew the easiest path to getting started - Leiningen - in favour of the latest DIY fad such as shadow-cljs.

I disagree with the characterisation of shadow-cljs as a fad.

It's a 4 year old project at this point, and in my opinion the easiest way to get started with cljs for a newcomer. It has sane defaults, and incorporates a ton of effort to "just work", such as with regards to the effortless way that you can pair it with npm or yarn to pull in arbitrary JS libs.

You can use pretty much any popular editor, such as Atom, VS Code, IntelliJ, and even VIM, for Clojure development nowadays. I personally use IntelliJ and I've never touched Emacs for any serious projects.

I'm also not sure what the problem with Leiningen is, it's a hell of a lot better than most build tools I've used. If you want to see brittle, you should take a look at NPM clusterfuck. Yet, people happily use that mess all the time.

Poor start up time is typically not an issue for web applications, which is the primary domain Clojure is used in. Meanwhile, at dev time you're using the REPL, so you're not restarting your app all the time. I often have an instance of the app running for weeks on end as I work on it. For environments where start up time matters we now have ClojureScript on Node and GraalVM.

The error messages are indeed a huge improvement in Clojure 1.10.

Any tips for Lisp writing/editing in a non-emacs editor?

My recommendation would be to either try Calva [1] with VS Code, or Cursive [2] with IntelliJ.

[1] https://marketplace.visualstudio.com/items?itemName=cospaia....

[2] https://cursive-ide.com/

Thanks!

I recommend cursive to start. Use the trial version for a month and see how it feels. It’ll spoil you though with how little setup is needed aside from choosing between parinfer and paredit for structural editing (I prefer the former).

Leiningen on Windows seemed like an after thought.

Even on Linux I have run into package management issues with lein1/lein2. I should say a build tool should largely be stable, not changing it behavior. On many functions like how you use local jars the features and methods have changed drastically at various points of time in Clojure/lein. This kind of churn has even effects like invalidating much documentation already written, and many stack overflow answers too. Why should an application developer be distracted by issues like these?

Most importantly if Clojure development is essentially depending upon lein, then I feel it should be distributed with Clojure itself. And emacs, cider, nrepl and lein chain has so many parts which are getting version upgrades independently, it breaks frequently for the user, functionality that was working suddenly stop working after an upgrade and you have to debug why it stopped working. Also I wonder how complicated it might be to set up the whole chain on a Windows machine.

I can't speak to these problem as I haven't used Windows in years myself. I imagine majority of the community uses Linux and MacOS hence why Windows has been an afterthought. This isn't unique to Clojure by any means either, a lot of dev tools are difficult or impossible to use on Windows from what I've seen.

Clojure now has dependency management built in with deps.tools that ships with it. It's simpler and less opinionated than lein. So, perhaps that will address the problem with getting started on Windows.

The Windows experience with leiningen can be pretty good when kept up to date, which can be done easily using https://scoop.sh/ to install it and update it once in a while

[Notepad++ 7.5.6 for Clojure](https://github.com/linpengcheng/ClojureBoxNpp)

It didn‘t gain any popularity with me, since I had to debug it once. Most Clojure code is just a collection of overly clever recursive subroutines. In the end I rewrote the code in Java. It was shorter, easier to read and probably more performant than the same code in Clojure. Don‘t believe the hype. Clojure has some nice constructs for concurrent programming, but most of them are available also in Java since version 8, or are available as some external package. In the end programming in Clojure will just make your project more expensive, as good Clojure programmer are a rare species. The small advantages of smart Clojure constructs won‘t outweight the costs of maintaining Clojure code in the long run. The same is actually true for Scala.

Java didn't gain any popularity with me. I prefer Basic. Basic syntax is significantly less complicated than java and none of that overly clever object orientation making the code more readable and more performant. Don't believe the hype.

I'm not sure you would end up with a good or accurate impression of any language where the extent of your Interaction with it were "I had to debug it once".

I'm sorry, but as somebody who's been working with Clojure professionally for nearly a decade I have to say that this is just utter nonsense.

Summary: The JVM made Clojure popular. The JVM made Clojure hated.

Yeah, haters gonna hate - it's human's nature. Give a man a powerful tool that solves lots of problems with two* most popular platforms today: Java and Javascript and they still will be complaining.

* actually three - if you include CLR

I love clojure. Would it be possible to get it running on a rust runtime? That would be awesome

How much did it get?

Clojure is a functional programming language based on relational database theory.

The Clojure system can be treated as a RMDB using the Lisp language, RMDB Schema (Clojure Spec) is a static type, and SQL (LISP) is a dynamic type.

It is an example of the best combination of dynamic and static types.

It's innovation, simple, practical and reliable.

https://github.com/linpengcheng/PurefunctionPipelineDataflow...

could you elaborate on that ? how is it based on relational DB ?

               Clojure -> DBMS, Super Foxpro

                   STM -> Transaction，MVCC

Persistent Collections -> db, table, col

              hash-map -> indexed data

                 Watch -> trigger, log

                  Spec -> constraint

              Core API -> SQL, Built-in function

              function -> Stored Procedure

             Meta Data -> System Table

In the latest spec2, spec is more like RMDB. see: https://github.com/clojure/spec-alpha2/wiki/Schema-and-selec...

The main development goal of clojure is to write the database. The development idea is actually from the database, not the FP.

With reference to the database, as long as I use spec to strictly define (standardization) the core data model, I can ensure the correctness of the system.

I've turned the traditional work of ensuring code correctness from in-code type system validation to data self-validation. Turn the work into verifying the core data model instead of validating the input and output data types of the function.

Similar to industry, verify that all finished products meet the standards before entering the warehouse. Also similar to databases, verify their compliance before data enters the database.

That is "Data as a service, Warehouse as the core, operates around it".

A system requires only a few core data models, and development is built around the core data model.

Persistent data structures ensure that the modification of the immutable big data model are high performance and memory efficient.

In addition, using my pure function pipeline data stream (https://github.com/linpengcheng/PurefunctionPipelineDataflow) makes debugging, parallelism and extension very simple.

from: https://github.com/linpengcheng/PurefunctionPipelineDataflow...

The main development goal of clojure is to write the database. The development idea is actually from the database, not the FP.

This seems a tenuous theory. Clojure has been described as a language for data processing, but this isn't the same as basing the design on relational databases. Clojure isn't even relational; its data is generally kept in standard collection types, rather than relations.

I'd describe Clojure as a dynamically typed, functional Lisp that emphisizes open data models and explicit time.

I think that mapping the data type of Clojure to the RMDB data model can achieve RMDB theory, simple operation, convenient viewing, high performance, and reliable results. I think the open data model is not an excellent and safe engineering practice.

Map to static type RMDB schema, similar to RMDB with schema and data as the core, with sql operation, simple and reliable.

Unrestricted open data model + dynamic type is not a good combination for constructing a complex large system.

its data is generally kept in standard collection types, rather than relations.

Isn't that the same as relational databases once you dig a layer deeper?

No, it's hierarchical (because we nest collection types) vs relational.

In RMDB(For example postgresql), Database is hierarchical nest collection types. It is also consistent with Clojure.

```clojure

{:table01 {:row01 {:col-array [0 1 2]

                   :col-json  "{\"a\": \"Hello\"}"

                   :col-text  "abc"}

           :row02 {}}
 
  :table02 {:row01 {}

            :row02 {}}}

```

In addition, postgresql supports inheritance, which is also hierarchical nest collection types.

While Postgres supports hierarchical data in the form of JSON, that doesn't mean it's a form of relational data, it just means Postgres supports both.

Clojure doesn't have good tools in its core library for working with relational data. There's no core type that explicitly represents a relation, and Clojure lacks functions for many basic relational algebra operations. For example, how would you perform a natural join across your data structure?

In programming, data modeling is the most important.

Convert(or design) data to hash-map, join (or merge) by key.

it can write commonly used operations as functions, try to row (or col) operations as much as possible, and join all data only when necessary(reduce the row-join).

```clojure

(def a {:a-id-01 {:a-name "a1"}

        :a-id-02 {:a-name "a2"}})

(def b {:b-id-01 {:a-id :a-id-01 :b-name "b2"}

        :b-id-02 {:a-id :a-id-02 :b-name "b2"}})

(->> b

     :b-id-01

     :a-id

     a

     :a-name)

;=>

;"a1"

(let [x (b :b-id-01)]

   (->> x  

        :a-id

        a

        (merge x ,)))

;=>

;{:a-id :a-id-01,

; :b-name "b2",

; :a-name "a1"}

```

You're describing a language whose data model is hierarchical, not relational. In your code you first convert a relation into a hierarchy of collections. Why would you need to do that if Clojure used a relational data model?

In RMDB, database data model (db,table, row, col, value[array,json, int, text, etc.] ) is hierarchical.

Relation is a logical model mapping of data structures, it is just a logical thinking that exists in the brain.

SQL, Prolog, clojure.core, minikanren can be used for relational operations.

The relational model isn't arbitrarily hierarchical. We can place a map inside a map, or a map inside a vector, and keep going indefinitely until we run out of memory. Conversely, in a relational database we can't place a database into a table, or a table into a row.

It's true that we can represent relations using collections. In Clojure we'd write:

    (def a
      #{{:a/id 1, :a/name "a1"}
        {:a/id 2, :a/name "a2"}}

    (def b
      #{{:b/id 1, :b/name "b1", :a/id 2}}

But these structures don't allow for efficient lookup or joins, and we lack inbuilt functions to easily deal with data modelled in this way.

Relational databases are based on relational algebra. If Clojure is based on relational databases, then we'd expect to be able to do relational algebra easily in Clojure. But we can't: the core library isn't designed for it, and the built-in data structures aren't designed for it.

In general, I think:

1. arbitrary layering and deep nesting are not good engineering practices.

2. refer to the data-model & code of my latest two posts. I prefer to use hash-map as the table with the primary key hash index, with key as the primary key and val(colname-colval-hashmap) as the row content.

I also don't think relational algebra operations must be implemented in the form of RMDB and SQL. It can also be implemented very elegantly with clojure.core. using hash-map operation is simpler, clearer, smoother and high performance.

There are many ways to implement relational algebra. The thinking is not limited by the "information structure" displayed by the traditional RMDB interface. In clojure, the hash-map(NoSQL) is the underlying physical model, and the relational model is the upper logical model.clojure core function acts as a data manipulation language, I named this architecture SuperSQL or SuperRMDB.

In fact, the original data manipulation language of posgresql and foxpro is not SQL. Clojure core function is closer to foxpro's commands (DML).

3. set, vector, list is generally not a good default data container, only used when needed. you use the set as container, it's difficult to operate data (table, row, column, value).

4. In summary, I think: programming is the process of designing a data model that is simple and fluent in manipulation. To have open thinking, not to be restricted by traditional thinking, to be flexible, adapt to local conditions, and design as needed.

"arbitrary layering and deep nesting are not good engineering practices"

Perhaps, but that's irrelevant; I'm describing the difference in how hierarchical and relational models are designed.

"I prefer to use hash-map as the table with the primary key hash index, with key as the primary key and val(colname-colval-hashmap) as the row content."

And what if you need a second index? Your indexing should be separate from your data model, otherwise you can't write performant relational algebra operations that apply in the general case.

"It can also be implemented very elegantly with clojure.core. using hash-map operation is simpler, clearer, smoother and high performance."

No it can't. Suppose I have a relation with keys: a, b, c, d and e. I want to index on a, b and the pair (c, d). How would I do that in Clojure? What happens if I later decide I also want to index on e?

This is the sort of problem that's trivial to solve in a relational database, and extremely hard in Clojure, because Clojure doesn't have the functions or data structures to support data modelled in this way.

That's not to say that Clojure can't have these tools; just that they aren't built into clojure.core, because that's not what it's designed for.

"set, vector, list is generally not a good default data container, only used when needed"

Yes they are. Sets are the basis of relational algebra.

You're complecting the ideas of data representation with data indexing. Sets are a good representation of a relation, but a poor index.

We can get the best of both worlds by combining the two:

    (def a
      (let [r1 {:a/id 1, :a/name "a1", :b/id 1}
            r2 {:a/id 2, :a/name "a2", :b/id 1}]
        {:relation #{r1 r2}
         :index
         {:a/id   {1 #{r1}, 2 #{r2}}
          :a/name {"a1" #{r1}, "a2" #{r2}}
          :b/id   {1 #{r1 r2}}}}))

A data structure like this allows us to start writing efficient relational algebra. For example, with a natural join we can look for the smallest index two relations have in common.

So we can begin to construct the infrastructure we need to perform relational algebra in Clojure, but it's not there to begin with, and therefore Clojure isn't designed around the relational model.

If you want to build an RMDB with Clojure, you are right, but this kind of project is very rare, including Datomic, which is built on top of RMDB, I don't think it needs to deal with such low-level and general operations.

Implementing an RMDB is not equivalent to relational algebraic operations. I think it should be to use simple, direct and lightweight the relational logic model to solve real-world problems. Don't over-optimize, over-generalize and over-complicate, keep it simple and direct.

Just like we design a Database in RMDB to solve real-world projects, This is the normal way to use relational algebra and models. After the database design is complete, we don't need care how the index is implemented, and we don't need care the underlying storage of the data. I mean, clojure is used as RMDB , is not used as tool of construct RMDB .

Therefore, my method is to design the application-level data model. The problem you said does not exist. When I get the data from the database (or elsewhere), I simply transform the data to the target model, you can think of this model as a table or view, you don't need to transform again, so you don't need multiple indexes.

I'm not talking about building a database; I'm talking about using a relational model for data.

Like most general-purpose programming languages, Clojure has a hierarchical data model. We have a number of collection types, and we can put any collection into any other collection.

A relational model takes a fundamentally different approach. Relational data is represented not by nesting collections, but by a flat set of tuples. Efficiency is achieved through indexing, not by rearranging collections.

There's some interesting research around on using the relational model outside of a database, but that's not a design goal of Clojure.

My method can also generate a "derived index" based on the "primary key hash index",it's Higher performance, not by rearranging collections.

```clojure

(def table01 {:t1-pk1 {:pk :t1-pk1

                       :name    "t1-r1"

                       :manager :m1}

              :t1-pk2 {:pk      :t1-pk2

                       :name    "t1-r2"

                       :manager :m2}

              :t1-pk3 {:pk      :t1-pk3

                       :name    "t1-r3"

                       :manager :m3}

              :t1-pk4 {:pk      :t1-pk4

                       :name    "t1-r4"

                       :manager :m2}})

(def t1-manager-index {:m1 #{:t1-pk1}

                       :m2 #{:t1-pk2 

                             :t1-pk4}

                       :m3 #{:t1-pk3}})

(->> :m2

     t1-manager-index

     (select-keys table01 ,))

; =>

; {:t1-pk2 {:pk :t1-pk2, :name "t1-r2", :manager :m2},

; :t1-pk4 {:pk :t1-pk4, :name "t1-r4", :manager :m2}}

(->> [:m2 :m3]

     (select-keys t1-manager-index ,)

     vals

     (apply clojure.set/union ,)

     (select-keys table01 ,))

; =>

; {:t1-pk2 {:pk :t1-pk2, :name "t1-r2", :manager :m2},

; :t1-pk4 {:pk :t1-pk4, :name "t1-r4", :manager :m2},

; :t1-pk3 {:pk :t1-pk3, :name "t1-r3", :manager :m3}}

```

Your point of view is mainly to emphasize that Clojure is a multi-paradigm, general-purpose functional programming language.

The postgresql development team is also this view, so postgresql is not only RMDB (relational modeling), but also supports OO and json (NoSQL). But postgresql default data modeling is relational modeling

My point of view is mainly to emphasize the best practices of data modeling and programming.

Both views are correct and can exist in parallel.

"Your point of view is mainly to emphasize that Clojure is a multi-paradigm, general-purpose functional programming language."

No, that's not my point at all. I'm saying that Clojure's core library and data structures are built around a hierarchical data model and not a relational one.

If you want to model your data as a relation, then you need to build the tools and structures for it. Look at your code, then consider how it would look in a language designed around relational algebra:

    (def table01
      #rel [{name "t1-r1", manager :m1}
            {name "t1-r2", manager :m2}
            {name "t1-r3", manager :m3}
            {name "t1-r4", manager :m2}])

    (select table01 (= manager :m2))

    ; => #rel [{name "t1-r2", manager :m2}
    ;          {name "t1-r4", manager :m2}] 

    (select table01 (or (= manager :m2) (= manager :m3)))

    ; => #rel [{name "t1-r2", manager :m2}
    ;          {name "t1-r3", manager :m3}
    ;          {name "t1-r4", manager :m2]

An indexed selection against a relation would just be a single function or macro. We wouldn't need to mess around with select-keys and set union to achieve such a simple operation, as you needed to do in your code. It would be built into the core library or the language.

I want to emphasize that I'm not saying you shouldn't model data the way you are. There are plenty of advantages to it. But the more you go down the relational rabbit hole, the less suitable clojure.core is to handle it.

Clojure is about data modelling and processing, but it isn't based on a relational model, as you suggest:

"Clojure is a functional programming language based on relational database theory"

Clojure is a functional programming language, but it's not based on relational database theory.

If Clojure is the sea, programming is to sail on the sea, I use the relational model as a lighthouse and route, as a reference model for simple programming, because the relationship theory is simple and scientific..

I had implemented a DataFrame with hash-map, which implements relational operations. A relational operation is just a function. The advantage of hash-map is that key-chain can be used as a pointer, and processing data elements is simple and efficient. Therefore, DataFrame has advantages of RMDB and NoSQL.

A strict relational model will lose the flexibility of data element operations.

For me, Clojure was full of false promises and dead ends / false starts regarding concurrency[0]. Elixir delivers where Clojure failed.

[0] for example, nobody uses the STM implementation

Where did Clojure fail? People don't use Clojure's STM because it turned out not to be necessary most of the time, not because of any inherent issue with the implementation.

Come back when you are ready to use Clojure for real, not for a weekend reading through tutorial blogs. Clojure's concurrency story is extremely well implemented.

The thing that kills Clojure for a lot of teams I worked with is the JVM. It's too limiting and constrains the problems one can solve.

What problems did you run into where the JVM was the limiting factor?

Whenever I ask to explain JVM criticism, people either really have nothing to say, or most of the time it turns out to be hatred of Java. JVM is a very solid piece of tech and it is hated for all the wrong reasons.

I'm baffled: give people a powerful tool that can simplify and bring enormous joy to work with two, most popular platforms of all time, even making it possible to share code between completely different worlds - they'd still be complaining about it.

Clojure is popular? It's at the bottom:

https://insights.stackoverflow.com/survey/2019#technology

As much as I love Clojure, it's still a harrowing language for beginners to learn and use. The learning curve is just too high. If you have to learn category theory in other to handle basic programming flows, then there's just something wrong.

Elixir, for instance, has many functional primitives, but isn't dogmatic about it. It can "feel" like traditional OO-style programming without the cruft.

How the heck Clojure has any relation to category theory? Clojure explicitly does NOT require nor use any explicit CT. Maybe you confused Clojure with Haskell?

Perhaps my understanding is outdated! IIRC you needed CT because you couldn't do some basic conditional flow things, but I could be wrong. It's been a while.

I dont think you need Category Theory to understand / use Clojure at all.

I say this as someone who uses CLojure and has at best a very sleight understanding of category theory.

With respect, I think you may be confusing Clojure with a different language.

Please don't disparage that which you do not know. You must be confusing Clojure with Haskell, it requires no category theory.

> As much as I love Clojure, it's still a harrowing language for beginners to learn and use

I have worked with several people for whom Clojure was the very first PL they have learned and they were very successful. One thing is nice about Clojure: it stays consistent - the idioms, data structures, conventions don't suddenly change. In most other languages - you pick up a framework, you try to learn it and it feels like you are learning a whole new language.

Yes, Clojure can be very intimidating and confusing for the beginners but its learning curve is not steeper than of Java's, Ruby's or Python's.

In The Pure Function Pipeline Data Flow, it only need Unix-style pipes and functions. It does not need OO, FP, design patterns, category theory, etc.

[The Pure Function Pipeline Data Flow](https://github.com/linpengcheng/PurefunctionPipelineDataflow)

Reply on news.ycombinator.com