Clojure 15th Anniversary: A Retrospective
building.nubank.com.br
building.nubank.com.br
1. No framework. Clojurists tend to prefer libraries and I would too, but for a newbie starting out, when every other language seems to have converged on one standard way of doing things (Spring, Rails, Phoenix, Django etc), Clojure just gets in the way with decision fatigue.
2. Leaky abstraction. At every level it is apparent that Clojure is hosted on JVM or Javascript. Simply knowing Clojure is not enough, and you have to understand the underlying semantics to know what is going on. Exception stack traces are excellent example.
3. Unmaintained libs. While the language is stable and libs are much more likely to work still, unmaintained libs often mean some stuff does not work or needs to be figured out. Often people just drop to using Java libs instead, but then the wrapping business tends to get tedious.
4. Java is improving faster with more pattern matching, records and functional programming features. Other JVM hosted languages are becoming less appealing.
2) Thats true, but its not leaky abstraction. Clojure never claims to abstract the host away, it embraces it so you can do all low level Java/JS stuff and seemlessly use their libraries without FFIs.
3) You have all Java/JS libs. Most Clojure libs are simply "done", so they get no commits. I don't know of any unmaintend ones.
4) Java is improving? Clojure is mostly done with that already. It was designed long after Java and makes many improvements obsolete.
I really _really_ want to love Clojure but the first three points (unfortunately along with the Lisp syntax simply because it’s different enough) alienate too many people from the language and it seems like that’s something that Clojure has at best accepted as not the target audience and at worst is still unwilling to acknowledge.
Its quite frustrating actually. The people who are likely to be greatest advocates of the language (its a new-age Lisp!) are likely to be vocal about its failings, and are also hit with worst reactions like being told their issues are not issues.
If you are already part of the team, Clojure folks are very friendly. I want to avoid saying cult, but sometimes it has characteristics of one.
Failing to write a native runtime from scratch and unearth one of the founding bedrocks of the language doesn’t mean Clojure is hostile to input or trying to alienate anyone, it’s just a massive project that no one is currently interested in (for good reason IMO).
Java is the standard for JVM. However, if Java was the perfect tool for any job, no other language would have been implemented over JVM.
I would rather use Kotlin than Java, who, to me, seems bloated and ancient. Also Java, the greater programming environment, kind of forces all things to be objects, unneeded inheritance, unneeded abstraction and patterns on top of patterns. I kind of like to use a more data oriented approach than object oriented approach, or pattern oriented approach, or Uncle Bob oriented approach. I strongly think that the shortest path is the best path, the less abstraction the better, the less CPU cycles used for something, the better.
ABCL was separated from J in 2008, when Java was at version. Kotlin was released in 2011, when Java 1.7 was released. Groovy was released in 2007, like Clojure. Scala was released in 2004, when we were at Java 1.5. GNU Kawa was registered on Savannah in 2001, when we had J2SE 1.3.
Now just because Java has become a lot better in the interval doesn't mean those languages have no purpose, but if Java in 2001 had all the nice features (streaming API, try with resources, the Optional type, switch expressions,...) it has now then the JVM landscape may very well have looked very different.
JavaScript is the standard for the Web. However, if JavaScript was the perfect tool for any job, no other language would have been implemented over JavaScript.
And so on and on and on...
Platform languages have an advantage for day 1 access to every single feature the platform offers, while guest languages always have to deal with impedance mismatch and additional tooling.
Personally I really like that Clojure, Python, and browser targeting languages have less frameworky cultures.
Or in areas that do end up having to do the "don't call us, we'll call you" hollyood principle due to reactive nature, there's at least significant amount of choice and there's not one to rule them all (FastAPI, Flask, Django on the web side for example)
I am still early in my career. I'm still discovering stuff for first time. I've also found that I am much more likely to use streams, lambdas, Optional and predicates in code compared to my more experienced colleagues. My functions tend to be pure more often, code has more interfaces than inheritance and generally more unit testable. One of the surprising moments was learning Elixir and realising I'm already doing the pipe thing in Java with Optional.map().
Java will never shed its OO skin but it is starting to grow FP muscles in the right places.
Collection factory methods https://openjdk.org/jeps/269
Stream API improvements https://www.javatpoint.com/java-9-stream-api-improvement
Honorable mention since not strictly functional, but makes code clearer and integrates well with functional patterns: pattern matching for switch statements https://openjdk.org/jeps/427
And these are only language level changes, the platform has expanded insanely significantly, but that also benefits clojure greatly (which is a good thing, it’s no competition)
As someone who codes professionally with Spring frameworks every day, this is a huge selling point for Clojure to me!
It is such a huge, opaque, convoluted, complex unpredictable mass of code, that forces broad changes to working code whenever you upgrade to a new major version.
One of my favorite Spring debugging anecdotes was where a coworker was trying to find a class from a runtime stack trace in the libraries. Only to discover that the class didn't exist in the source at all, but was dynamically generated by Spring at runtime!
Your other points I largely agree with.
With all due respect, that’s like numero 0 thing to know about Spring’s AOP which is like the second most important part of Spring after dependency injection.
It breaks down the fundamental debugging model of looking up the code in stack traces and seeing what it does. When an exception is thrown in generated code, what do you do to debug it?
In fact, I'd go as far as to argue that approach taken by frameworks like Rails and Django is an antipattern because it couples your frontend with the backend. This leads to all kinds of problems.
For example, it's much easier to horizontally scale your backend when you have stateless services. It's also much easier to add alternative frontends, such as mobile apps, since you already have a service API that can be resued. These are just two concrete examples.
2) Knowing the underlying platform helps, but you certainly don't need any deep knowledge of it. I've worked with excellent Clojure developers who only had superficial knowledge of Java or Js before.
3) I'd argue that Clojure actually does better in this regard than a lot of languages. There are orgs like Clojurists Together that actively fund development and maintenance of Clojure libraries https://www.clojuriststogether.org/ and Nubank themselves fund open source developers as well.
Another aspect is that many Clojure libraries are simply doing data transformations. The API accepts a data structure, and returns a different data structure as a result. Such libraries generally don't need a lot of maintenance work. I personally maintain a popular HTMLtemplating library called Selmer and a markdown parser. After the initial feature set was finished, there really hasn't been a lot of work that I've had to do on either library. I do an occasional update to add features, but these have been few and far between lately because these libraries are essentially complete.
4) The difference is that Java keeps growing as a language and that adds cognitive complexity. The bigger the language is the more things you have to keep in your head working with it, the more patterns you have to know, the more styles you end up seeing. All of this makes the language harder to use effectively, and creates room for errors.
The value of Clojure is that it's a small and focused language. It's stayed largely the same over the years, and people tend to use a common set of patterns working with it. I'd much rather have a small and expressive language than a large and complex one.
Furthermore, Clojure is completely different from Java semantically. It defaults to immutability, and it encourages much better patterns out of the box than Java. Personally, I don't think you can just keep bolting things onto Java to achieve parity with that.
1. Improve startup times.
2. Ditch leiningen, its functionality should be included in base clojure.
3. Emacs toolchain for Clojure seems massively complex and have too many parts. Makes it difficult to get it reliably installed/setup.
4. Getting it set up on Windows seems obscure, which shouldn't be. Part of the problem is that lein is not distributed with clojure.
5. Error messages are cryptic.
6. Needs a lisp style debugger. Once an exception occurs we need to get a repl at that point of state.
7. Needs provision include Local jars easily without having to set up a maven repo for the same. This is very important for hobbyist work, and for exploratory programming.
And ditch Emacs (unless you are already using it/used to it) and try VSCode + Calva that runs basically out of the box.
Exceptions... yes.... they are clear when you know what they mean, but it takes time, because they are so long.
Startup... you only start a REPL once, so does not really matter (to me at least). For compiled code, you can try GraalVM, that has zero-ish startup time - because all static initialization is resolved at compile-time.
Just my $.02
Edit: Calva also includes a debugger - never used it, but here it is https://calva.io/debugger/
People say this a lot in Clojure, but don't you worry about global vars that no longer point to what you thought they pointed to? For this reason alone, I almost always load a new Python REPL.
I usually restart the REPL only when I add a dependency, so it will be fetched without me caring about it.
And the OC's onboarding experience reminds me of mine. You need some external tutorial / book to get setup.
Also, last time I used calva, the only thing I got was kondo complaining they the default test created in a brand new project were using unknown symbols, or something. So I assumed my setup was wrong, and gave up.
Clojure is "simple made easy _once you've done all the hard part as a bizuthage_"
I hear you about this, but during prototyping one cannot be excessively concerned with did I change the global state enough such that it's better to just start a new REPL (and have to worry about the associated costs)? And if you ask this question a lot, long running REPLs are not a good idea for development.
For validation/testing starting with a clean slate takes away a lot of my worries.
Deps you install Clojure and ... no hints how to get to a running project.
"touch deps.edn"? Still no hints how you get from here to functioning clojure dependencies. It isn't a YAML file. Isn't JSON. No template.
Read the reference page (https://clojure.org/reference/deps_and_cli)? We quickly learn that you can execute a function by invoking "clojure [clj-opt] -X[aliases] [a/fn] [kpath v] kv-map?". Still not much closer to installing a dependency, a lot of people probably give up here.
Oh, there is a guide! Read the guide (https://clojure.org/guides/deps_and_cli). Now we get to our first dependency.
Lets add a 2nd one!
{:deps
{com.github.seancorfield/next.jdbc {:mvn/version "1.3.834"}}
{clojure.java-time/clojure.java-time {:mvn/version "1.1.0"}}}
$ clj
Error building classpath. Error reading edn. Map literal must contain an even number of forms (~/deps.edn)
This is a pretty plausible on-boarding with deps. Beginners should be using lein, it is a lot more forgiving and doesn't leave people staring at an error message feeling stupid after 10 minutes being confused about what the docs are trying to prepare them for. It'd be good to include something like that in the default distribution so they don't have to figure that fact out by failing a bit with deps.Maven (like Leiningen) bundles all of that in a single tool, plus adds tons of more options. On top of that, you have a plugin for almost anything.
Or you could read the guide on the website: https://clojure.org/guides/deps_and_cli
There is https://github.com/practicalli/clojure-deps-edn which solves this but it’s not linked to from any official docs which seems a miss to me. As well as the config and full documentation, it also comes with a video walking you through a demo of all the features.
I think their should be a "lein-for-deps-template" that subtly guides people in the right direction by starting the project up with 2+ dependencies. Lein dodges this by using a list-of-list data structure where this mistake is harder to make (copy and pasting stuff in brackets is the right strategy in lein) and there is much more boilerplate in a project.clj to hint what our aspiring Clojure programmer needs to fill in.
My problem here is that the entire way through that story, at no point is there a simple prompt in the official reference documentation telling people what a deps file with multiple dependencies should look like. Even finding the examples of 1 dependency is quite deep in and seems to be an advanced topic (you'll note the sibling comment that considers dependencies an advanced topic for people who already know the language well enough to reflexively get maps right, and it says a lot about the actual complexity of deps that the official guide to installing dependencies thinks 2x dependencies is out of its scope).
There are plenty of reasons to start a new REPL: - adding a library
- changing the project config
- messing up the repl state
- starting with a clean slate
- switching branches
.Net code can be AOT compiled to native code. Doesn't JVM have a similar option?
2. It is. tools.deps
3. It's just one package to install? Why not use intellij or vs code if you want?
4. Yep. This is true.
5. I don't think I've ever thought that. Could you give an example?
6. Cider (emacs) and Calva (vs code) both have this.
7. You can do that just fine, once again tools.deps lets you do this with no hassle.
6. When I last checked emacs-cider only gave a stack trace when exception occurred, not a live repl at that program state. See how emacs responds when there is an exception in elisp, one gets dropped into repl where one can examine, even modify the value of variables at that point of exception, and even try restarting execution from that point.
Much of tutorial and other information seems to use leiningen atleast when I had last seen it. If tools.deps is the canonical way to do it, they why are people refering to lein which is a third party tool.
Using lein, is a batteries-not-included approach, I am not sure why it is seems so popular.
If lein is so essential, then why not bundle it into the clojure base jar itself.
The best part is that you can develop it interactively via the REPL the same way as regular Clojure. You just run bb --nrepl-server and connect an editor to it. Calva with VS Code works great for this incidentally.
Startup times are a non issue. Does the fact your server takes a few extra seconds to start on deploy actually matter? No, it doesn't. In development I hardly restart my REPL unless I update a dependency or pull from a branch. Even in the latter case I can usually use Cider to refresh it. In my 8 years of writing Clojure that other people give me money for I can confidently say it's never been a big issue.
I love that it runs on the JVM and doesn't hide it. There's infinite libraries for every weird enterprise B2B widget so I don't have to screw around (much) to get them to work. The JVM is a great tool and runs on anything.
Types. Everyone always hand wrings about types because they don't understand how anyone could possibly refactor the huge data structures they're used to from OOP without types. It's also, largely, a non issue. For one, data structures are not the abstraction in functional programming. Functions are. You don't need to write class OtherCompanyAccountsGetter, you just write a function that maybe takes a couple parameters and returns the accounts. You then test that function. Then you build more functions on top of it. If the underlying API changes you go change that one function and make sure your tests still pass. If something drastically changes about inputs you can easily find all usages and fix things up. Additionally, in my experience, FP leads to systems that need far less big time refactoring than OOP, since things are encouraged to be broken up into much smaller pieces (functions) and those pieces are meant to be black boxes. The worst Clojure codebase I've worked in was far easier to understand than the worst OOP codebase I've worked in.
I haven’t written a ton of clojure but every large untyped code base I’ve seen is a mess, and usually there are people around it claiming it’s not
While I use Clojure less than I used to, I view it as a truly beautiful language with a coherent and thoughtful design. The community has always been fantastic, and the thought leaders (rich, stu, fogus, chouser, david nolan, etc) have given so much to the world.
I'm very happy to see the language is still making forward progress and has carved out ecosystems (i.e., finance) where it's finding solid adoption. Not much else to say besides a huge THANK YOU to everyone that has helped bring it this far. It has brought a lot of people a lot of joy.
Personally I moved to my own language Calcit for my own projects using dynamic language tools. Calcit is still following lots of designs from Clojure.
Also, due to the extensive use of FP and not much side effects, you can catch most of these mistakes at test time - not the same as a good static type system, but it is an interesting tradeoff, better expressivity, but not happening at compile time.
As a runtime contract, spec can check things type systems can't, but IMO it's not pervasive enough in most code bases to really satisfy.
(Though perhaps using them internally as well may be okay, just disable it at prod)
> compiler telling you if the pieces still fit together?
You still write tests in scala do you?
Open a REPL, and start working. You can just execute the code in realtime as you're refactoring. The size of the codebase is largely irrelevant.
Ironically, in Clojure the compiler _does_ tell you what is working and what isn't when you work this way :)
And anyway, you write tests don't you?
Besides that as others have mentioned: Clojure spec, various kind of automated testing, including, if that's your thing, generative-testing/property-based testing generated based on your specs (it's not always trivial but can really help at times).
You have to write many more tests and add a lot of predicates as pre/post conditions (invariants) to functions to check at runtime what a more modern compiler could have told you at compile time.
It adds a lot of friction and slows down the development process.
Combined with Clojure having lazy evaluation and nil punning it becomes frustrating in larger, domain heavy code bases.
Simple common “CRUD” line of business applications with shallow code paths and a database schema to serve as a secondary type system would probably suffer less from this, explaining why some devs say they don’t experience this.
For comparison, Typescript became so popular precisely because it mitigates similar problems for JavaScript once code based grew to non-trivial sizes.
What you point out is exactly what I think is the main drawback as Clojure (any dynamic language, really). It gets very tedious to waste so much time refactoring code and in the increased number of unit tests you need to write in order to avoid this from happening.
Personally, I think the lack of types in Clojure (and other dynamic languages) has another major drawback in that it doesn't allow you to easily express (and take advantage of) useful abstractions. In typed languages I got used to first write a "skeleton" of the application / library / module as a set of interfaces (traits, protocols, whatever expresses contracts in your language) and then develop and tests the implementations mainly independently. I find this extremely productive, in actual terms of amount of functionality per unit of time.
In Clojure I find myself doing diagrams or writing down data structures, etc. because there is no practical way of doing that in the language itself. Yes, I know protocols, types, spec, malli, etc. but they are not nearly as clean and useful (IDE help, etc) as let's say Typescript types.
I find this lack of expressiveness very frustrating, because I love Clojure otherwise. Persistent data structures are a game changer and the power you can get from a very small, generic API is mind blowing. And the data oriented paradigm, where everything tends to be expressed as data structures which are trivial to operate on, is simple conceptually and very powerful. I even got to like the syntax a lot. Once you get used to it, it's extremely clean, elegant and even clear (!).
Being as it is, though, I'm not sure that I would chose Clojure for a large project, even if I was the only one working on it. My brain needs typed languages as soon as I have more than 200 likes of code, it seems.
you're not supposed to do that in a dynamic language. Clojure's mentality towards data is "take what you need and ignore the rest", it's always the receiver of some data who is responsible to handle what they receive.
A lot of problems people have with dynamic languages would go away if they embraced the dynamism. If you're constantly refactoring because you make assumptions about the state of the program globally you're not actually utilizing the dynamic aspect of the language.
In practical terms, treat data in clojure the way you treat data over the web. When you get a some json data over a network you don't necessarily make absolute assumptions over its structure, you take what you need and deal with it when you receive it.
I see the fact that people often end up creating large and tightly coupled monolithic codebases in static languages as a negative aspect of static typing. Such codebases are difficult to reason about even if you have guarantees that the types align. Ultimately, you need to understand the relationships in code, and how they relate to business logic. The more coupling an application has the harder it becomes to reason about it as a whole.
Ideally, I think applications should be structured as a bunch of Lego blocks that can be composed together. Each component should encapsulate some functionality, and then the flow of the business logic should bubble up to the top and expressed in how these components are chained together.
most of my projects these days in clojure/script, including
- an (admittedly tiny) BI tool clone (think metabase / PowerBI)
- a search engine for private charter flights
- a live trading bot for cryptocurrencies (it's only breaking even though)
Altogether 17k lines of code. What else is there to say?
Clojure doubled my salary in 2 years and made programming fun again
The emphasis to work directly with the data is quite unique to clojure (as I'm aware of, please proof me wrong on this) which allows me to just dive in and think about the problem I want to solve instead of fighting my language or framework. The interactivity of the environment paired with immutability results in an instant feedback loop which let me get into mental flow really fast, which is a joy.
Even if your assumption is true, what's wrong with that? A language which offers more value over an existing base of knowledge is valuable and isn't uncommon.
You def don't need to know Java as a whole, but it was actually a little challenging and confusing to catch up on the relevant JVM bits.
Educational material either doesn't use it at all or assumes you know it
- You didn't have to search for it
- Someone's (Rich Hickey's) opinion
- He has worked with Clojure
- He has worked with another Lisp dialect
Clojure, the product (largely) of one person, is more opinionated than CL, the product of a committee. As a result, I find its design to be more cohesive, being oriented around functional programming and a set of core abstractions which play very well together. It has a bit more syntax than CL does out of the box, mostly for collection types.
But the biggest practical difference for many people, once they get their heads around Clojure's functional nature, is Java interop and the JVM. It is idiomatic to use Java classes directly from Clojure, and the library ecosystem is just vast, dwarfing any other lisp's. Deployment as an uberjar is typically a smooth story.
These days I'll reach for Common Lisp for personal projects, simply because of its startup speed and because I like some of the quirks of the language. But if I am going to need a library I know exists in Clojure/Java and is solid, which does happen, Clojure's a no-brainer for me.
Finally, there are a lot more jobs for Clojure than for CL, as far as I can tell.
But wilder issue is can it run on notebook (I can do lisp after some struggle but not sure about this) for ease of sharing of idea.
The iPad version used a bit but not sure about too many things and I switch back to common lisp.