The Future of Clojure
thoughtworks.com
thoughtworks.com
You will hire smarter-than-average engineers, but with a caveat that no one talks about: they are all self-selected for being people who enjoy tinkering on computer science problems to a fault, and not necessarily your business. So the breakdown is that you get 10 super-smart people, but 9 of them are focused on myopic bits of code plumbing or reinventing APIs or something, and in practice you only have 1 smart person focused on your business.
As the other 9 continue to reinvent wheels, the nature of clojure being small relative to other open-source communities means that eventually your shiny new clojure tools get replaced by better community offerings, except you don't have the bandwidth to switch over and start using them.
Overall, velocity would be improved if you had hired 1 super smart engineer and 9 mediocre ones.
It's important to note here that while technically clojure can interoperate with java or javascript, in practice everyone expects you to use poorly-maintained clojure-ish libraries to paper over the java/javascript bits, which get out of date quickly. This effect is 10 times worse for clojurescript in the browser, although it's pretty bad for clojure targeting the JVM as well.
Care to elaborate? For example, re-frame, reagent have been stable, reliable libraries for years. In contrast to many JS libraries. JS interop has been virtually unchanged for years.
What am I not seeing here?
When you use webpack, you can use that ecosystem of space optimizers too.
I cannot speak for anyone else, but as someone who writes a lot of Clojure (not much ClojureScript), I actually tend-towards the most-maintained libraries, and that usually just means Java. Personally, I think Clojure is a "better Java than Java", and I've never had a huge problem calling into the Java versions of libraries.
For example, I've had fewer headaches using the regular Java Kafka bindings or JeroMQ than by using the "Clojure-ified" versions of these libraries. I absolutely hate Java as a language, but personally as an engineer it's hard to dispute that Java libraries tend to get a lot of money spent on them, and it feels silly to completely ignore that in some pursuit of "purity".
This actually seems to be more-or-less the ethos of the Clojure engineers where I work, but maybe my situation an outlier.
As someone who is trying to decide how to interop Clojure + ZeroMQ, do you have any pointers for working with JeroMQ within Clojure?
Yesterday, I was browsing/evaluating the Clojure libraries for ZeroMQ, and all them haven't seen git pushes in the last three years and have between 10 and 120 stars on GitHub -- but JeroMQ has recent activity and >1.8k stars, so I'd like to stick with that for sake of support/documentation/etc.
--
(Caveat: ... not that GitHub stars are the best metric for quality... but they're not a terrible one...)
I definitely think there needs to be a better messaging with the library support in Clojure; as it stands, a Clojure newbie might (very reasonably) look up "clojure zeromq" on Google, get a crappy, unmaintained library, and dismiss the language as having "bad library support", when in reality most of the Clojure veterans that I work with do the same thing that I do: when the Clojure libraries are bad, just use the Java ones.
There are exceptions to this in rare cases; I haven't done a ton of ClojureScript, but the bit I have, I genuinely really liked the Re-frame framework. I'm not a frontend guy, so I'm speaking largely out of my ass, but I found it to be a lot more pleasant then vanilla React.
[0]https://github.com/zeromq/jeromq/tree/master/src/test/java/g...
The (excellent) Java interop is a major strength. I’d argue it’s also a double edged sword: Java libraries tend to be very mature and capable, and Clojure programmers, especially the senior ones, tend to be very comfortable in Java, so often the native libraries just don’t get written.
I went to look for a CSS selector library the other day, the accepted solution seems to be to just call into Jsoup. I did find a native xpath library but it was such a thin wrapper around Java stuff that I ended up needing to learn those APIs to get something done the wrapper author hadn’t considered.
The thing about relying on Java libraries is the users, who came because they like lisp and Clojure, spend an inordinate amount of time trying to wrap their heads around Java apis (if they don’t know them already). Which is not fun.
Clojure itself is decent. But as you mention-- "n practice everyone expects you to use poorly-maintained clojure-ish libraries to paper over the java/javascript bits, which get out of date quickly.'
That is indeed the problem I saw.
I've done React+Redux and it was a much worse experience than Reagent, which deliberately builds on Clojure's native immutability-friendly state management solution (atoms) and is based entirely around functions.
Not that devs coming from the JVM can't contribute! But there is some eyerolling from the devs coming from javascript when the primarily backend devs throw together a too-simple UI and wonder why everybody doesn't use reagent exclusively.
"Dealing with immutable data in JavaScript is more difficult than in languages designed for it, like Clojure. "
From React's docs.
The most used Clojure libs are well maintained and solid IME. Of course there are a lot of inactive ones too, it's natural especially as Clojure is not a young language anymore.
Hey, that's me! After 20 years I learnt to let go and start caring about the business a little bit more than about the code plumbing.
Clojure remedies all of my problems with previously used languages (Basic, C, Java, ObjC, Javascript, Python), in that it allows me to remove boilerplate and create layers of abstraction that are appropriate for problem solving. It's also a fairly opinionated language, preferring functional programming methods that work with how I like to solve problems. This is refreshing, as it feels like I can bypass all of the style wars that I encountered with Python., and the verbosity of C/Java/Obj-C. No longer being told to rewrite functional code as an object, by an engineering manager that only ever writes imperative scripts.
Lastly I now love writing front end code in Clojurescript...something I never would have expected. Reagent and Re-frame are a dream for large complex SPAs, which while I dislike in principle, seem to be the trend in cross-platform GUIs.
And it makes programming fun again!
When I first started in Java in 1997, there were few companies doing Java and few devs who knew it and it was much the same. Sun spent like a billion dollars marketing Java to create that ecosystem.
I guess now the question is if Nubank could/will also promote Clojure and to what extent (or if they will support Cognitect enough to do that). I would wonder if they went through that thought process and what RoIs for that are from their perspective.
It might even take much lesser investment than equivalent of $1bil from 90s... The Java/JVM market is huge so it might need just a little nudge :)
Within the JVM ecosystems it seems that basically Kotlin (as nice as it is) has been stealing some market share that might have belonged to Clojure. And all that just by selling some cheap cut syntax sugar on the corner.
There definitely is a huge desire within Java/JVM community to innovate and if approached correctly Clojure could actually shine - there are just few misconceptions and fears that could be put to sleep by some smart marketing strategy. Nubank is great first step in that direction, because it gives an example of a mature large yet innovative company relying on Clojure in production.
I suppose you could dismiss a lot of what Kotlin adds as mere syntactic sugar. But it also makes some much more fundamental improvements. It provides a much cleaner, unified, object-oriented type system. It provides about as sane an approach to null safety as is possible on the JVM. (Clojure's is arguably better, but I'll concede that nil punning is not for everyone.) And it provides a clean set of core libraries that doesn't have nearly as many friction points and inconsistencies as the core JDK does.
I don't see any technical reason why Kotlin couldn't have done the same.
Kotlin does do the same. I totally disagree with GP's assertion that Kotlin has no null safety. I use both Kotlin and Clojure and I find Kotlin much better than either Clojure or Java with respect to NPEs.
KMM can only help so much, specially when all the big Java projects are all integrated into stable releases.
> It's interesting to see so many people complain that it's hard to hire Clojure engineers, when I've had the exact opposite problem
I do think that Clojure shops (like all software shops) need to dial back the torture interviews / homework / tests, but that's really a bigger thing that just Clojure. I think more could be done to market Clojure to the non-believers, but I don't have specific ideas on how that could be done.
You can take my parentheses from my cold, unemployed hands.
That process was created by FANG because they can abuse their candidates and still get a lot of applicants (although I think this is less true than it used to be)
But to use that same hazing ritual at a smaller company with the hiring funnel further reduced by a niche language? Really stupid.
Say you're writing a pure function...
You start with just data and a sense of how the output might look. Let's say I have APIs providing me with a user record and a list of transactions, and I want to get that person's balance...
Maybe you start with some canned data
(let [person {:name "Matt"
:id 12345
:current_balance 100.10}
txns [{:person_id 12345 :amt 10}
{:person_id 44444 :amt 0.5}
{:person_id 55555 :amt 10}
{:person_id 12345 :amt 11}
{:person_id 12345 :amt -5}
{:person_id 66666 :amt 3}]]
(->> txns
count)
)
=> 6
(let [person {:name "Matt"
:id 12345
:current_balance 100.10}
txns [{:person_id 12345 :amt 10}
{:person_id 44444 :amt 0.5}
{:person_id 55555 :amt 10}
{:person_id 12345 :amt 11}
{:person_id 12345 :amt -5}
{:person_id 66666 :amt 3}]]
(->> txns
(filter #(= (:id person)
(:person_id %)))
count)
)
=> 3
(let [person {:name "Matt"
:id 12345
:current_balance 100.10}
txns [{:person_id 12345 :amt 10}
{:person_id 44444 :amt 0.5}
{:person_id 55555 :amt 10}
{:person_id 12345 :amt 11}
{:person_id 12345 :amt -5}
{:person_id 66666 :amt 3}]]
(->> txns
(filter #(= (:id person)
(:person_id %)))
(map :amt))
)
=> (10 11 -5)
(let [person {:name "Matt"
:id 12345
:current_balance 100.10}
txns [{:person_id 12345 :amt 10}
{:person_id 44444 :amt 0.5}
{:person_id 55555 :amt 10}
{:person_id 12345 :amt 11}
{:person_id 12345 :amt -5}
{:person_id 66666 :amt 3}]]
(->> txns
(filter #(= (:id person)
(:person_id %)))
(map :amt)
(apply +))
)
=> 16
(let [person {:name "Matt"
:id 12345
:current_balance 100.10}
txns [{:person_id 12345 :amt 10}
{:person_id 44444 :amt 0.5}
{:person_id 55555 :amt 10}
{:person_id 12345 :amt 11}
{:person_id 12345 :amt -5}
{:person_id 66666 :amt 3}]
bal-change (->> txns
(filter #(= (:id person)
(:person_id %)))
(map :amt)
(apply +))]
(+ bal-change (:current_balance person))
)
=> 116.1
(defn update-person-balance [{:keys [id current_balance]} txns]
(let [bal-change (->> txns
(filter #(= id
(:person_id %)))
(map :amt)
(apply +))]
(+ bal-change current_balance)))
"But I can do that in any ol' REPL!" you might say. Try Clojure -- really try it, with an nREPL connected to your running application and to your editor, capturing inputs, playing them back, writing evaluation output as comments -- and then tell me you want to go back to your old REPL. A notebook environment is similar, though clumsier IMO.
Come for the REPL-driven-development, stay for the immutability, the Java library ecosystem, the lisp syntax, the concurrency support, etc. I know going on and on about how great Clojure development is is basically a meme at this point, but that's because it's really great!
I mess with Clojure with some regularity and each time ultimately back away in a combination of frustration and nerd sniping self awareness as the ratio of editor setup blog reading meta work to actual work hits infinity.
Either I am terrible at finding good setup guides or it’s pearls before swine and I just don’t get the aha moment. I’m starting to think it’s the latter.
You're not the 'swine' here, you shouldn't feel that way. I've learned that sometimes paradigms require the programmer to be in the correct 'frame of mind' before it even begins to make sense, its not you.
In my limited experience, keeping an open mind to tools will get you much further and have more enjoyment in your career and life. Trying to force that square peg into the round hole will just make you hate new things. If the time comes and your mindset is right, the tech will still be there.
The thing that really helped was watching videos of talks / tutorials of people on youtube coding something up, for example [0]. I'm not calling that one out specifically as a great tutorial, that's just an example of what I mean =)... I personally would start with an example of a form and then build it up in stages, basically tinkering with the form as I work out how I want it to function.
Another thing is that you don't have to use a particular editor etc. I'm not sure for example if you think you have to use emacs. Don't get me wrong it's a great environment, but if you're a beginner to it and clojure, that's not a good combination. Pick something you like which hopefully has a good repl plugin. There are a fair few options these days. Intellij, vscode, atom, vim among others.
Worst case if you'd like me to sit down and have a short call to go over stuff I'd be happy to set aside a little time =)... I've done tutorial stuff before and it's always nice to help someone else get comfortable in the ecosystem.
Hope that helps.
- [0]: [Clojure, REPL & TDD: Feedback at Ludicrous Speed - Avishai Ish-Shalom](https://www.youtube.com/watch?v=Ngt29DyNDRM)
Your specific setup will depend on your primary editor. For emacs, the go-to is https://cider.mx/. E.g., with my cursor beside an s-expression, I can hit a couple keys to evaluate and view the output. Don't give up!
You could also try Calva if you use Visual Studio Code. It works great for me.
let updatePersonBalance = ({id, currentBalance}, txns) => {
let balChange = txns
.filter(i => id === i.personID)
.map(i => i.amt)
.reduce((i,j) => i + j);
return (balChange + currentBalance);
}
Picking Clojure over JS today is a very hard sell. Both are dynamic and use the same information model, objects/arrays vs maps/vectors but with the difference that JS has a big enabling ecosystem of libraries. And there's Typescript when you need it.
> Picking Clojure over JS today is a very hard sell.
I'm not sure I would choose ClojureScript over JS on the frontend. I haven't worked with ClojureScript much nor thought much about the trade-offs.
> Defaulting to immutability it's not as useful in single-threaded environments like nodejs or the browser.
It might not be as useful, but I think it's still useful. With immutability, you know that inside a scope you can do whatever you want with any data (ignoring atoms for a moment) and you're not going to step on somebody else's state.
> with the difference that JS has a big enabling ecosystem of libraries
ClojureScript has interoperability with all the JS libraries, no?
And if you need strong typing, then I'd agree Clojure is not a good fit.
Looking at state mgmt solutions for JS today, many will require or recommend immutable data - for very good reasons.
That, combined with the fact that everything is generally made up of simple data structures (lists, vectors, maps, and sets) just makes working with Clojure fun (for me at least).
Here’s a great example of what I’m talking about:
That includes every major Lisp dialect.
What's not Lisp heritage is to have a vector or hash table to which you can add an item to get a new container, such that the original remains the same.
It's a generalization of consing an item onto a list, though.
REPL based development gives you instant feedback. lets you write a function in your editor and then run it, change it, run it again. If your idea of REPL is python or ruby, then you are not getting the entire picture. https://stackoverflow.com/questions/5671214/is-lisp-the-only...
Immutable by default. This helps with debugging and solves thread safety. Feel free to pass around references without worry.
Java interop - you can use all the jars
"isomorphic" - write clojurescript for the browser, Clojure for the server
Some other example, Python and JS are both similar languages in many respects. But even something as simple as not having list comprehensions makes JS more fidgety when you're writing simple code.
Python not having great utils for stuff like "give me this dictionary, but without these three keys, but keep the original dictionary intact" (Clojure has dissoc for this) also makes it a bit fidgety compared to Clojure.
"You can write all of this" maybe, but at one point you're really going upstream from the base language.
new_dictionary = {__k: __v for __k, __v in original_dictionary.items() if __k not in {‘key1’, ‘key2’, ‘key3’}}
I want `select` and `dissoc`. I also would like for those to be performant (instead of requiring a bunch of copies). I don't want to have to choose between expressiveness and performance.
I don't write Clojure in the day-to-day but I think it hits those beats very well.
I work in three main language, including several years of Clojure work. When I applied to a new Clojure shop in Germany, the first step was a lengthy take-home assignment that was time-consuming and vague. After spending a couple days on it, I never got a reply.
A second Clojure startup in Italy did the same to me.
In both cases, there was zero guidance and it felt more like a guessing game about what they were looking for than an actual coding evaluation.
I have never experienced this in any of the other langauges I've used professionally. I've had to do simple, short technical screens, but only Clojure companies seem to expect that you give up big amounts of your time for no pay for the honor of applying to them.
That really needs to change.
Here's a different anecdote:
I've had two Clojure-only jobs here in Denmark and neither of them had any homework assignments. I got hired at the first and only interview in both cases. One was my first job after finishing uni and the other one was the job after that.
Now I work at Copenhagen University, also writing Clojure and ClojureScript (so the third Clojure job, although it was never advertised as such) along with a little bit of Java for projects inherited from my predecessor. I didn't have any programming assignment while interviewing here either.
It's not that we don't get interview assignments in Denmark, but I've only had to do them in Python, PHP, Java, and Javascript - i.e. mainstream languages.
Things I'd ask:
- What are the new resources (for Clojure? or just Datomic?), how are they being used? Does Nubank have opinions / direction on these resources.
- Has an open sourced Datomic been discussed? It wouldn't seem to be strategic for Nubank and could be a big boost to the Clojure ecosystem. I'd read every line..
- Who owns the Clojure trademarks and IP going forward. Any talk of a Clojure Foundation?
- Alex Miller does an awesome job but boy he has a lot to cover. It suits Rich to have a small team of trustees for core (though boy its got small) but the community stuff could surely be advanced quicker. Old tickets in contrib and key libs etc..
- There was talk a few years ago of facilitating community involvement with a PEP type process, https://www.python.org/dev/peps/pep-0001/? Any further thoughts?
- Clojurescript has left Core. Is Nubank likely to want to get involved here?
I have no right to these answers, some may be commercially sensitive and of course Clojure doesn't belong to me. However there is no harm in asking when answers to the questions are important to me in my future uses of Clojure. Indeed I think it would be superior - in terms of Clojure's continued acceptance into industry - to know more of it's future direction than has traditionally been the case. Hopefully Nubank provides the confidence needed for this to happen.
> What are the new resources (for Clojure? or just Datomic?), how are they being used? Does Nubank have opinions / direction on these resources.
Since Cognitect joined Nubank, several people have joined the Datomic team, both new hires and internal moves from Nubank and the rate of development has increased. We are also looking at expanding the Clojure team in the near future.
> Has an open sourced Datomic been discussed?
No changes planned.
> Who owns the Clojure trademarks and IP going forward. Any talk of a Clojure Foundation?
Clojure was and is an independent project and Rich Hickey is a joint owner (along with contributors) to the Clojure copyrights. From Rich's announcement on the Clojure mailing list: "Clojure remains independent, and development and stewardship will continue as it has, with more resources and a much larger sponsoring organization in Nubank."
> Alex Miller does an awesome job but boy he has a lot to cover.
Thanks and indeed! As above, we are likely to expand the Clojure team in the future.
> There was talk a few years ago of facilitating community involvement with a PEP type process
We created https://ask.clojure.org as a way for Clojure users to discuss problems important for them and to allow voting on those problems for their importance. We use that information to inform decisions about what to work on so I encourage all Clojure users to express their needs there!
> Clojurescript has left Core. Is Nubank likely to want to get involved here?
Cognitect was, and Nubank now is, the primary sponsor for ClojureScript (funding both David Nolen and Mike Fikes).
The converse of this is that good, wise developers are going to (ultimately) _demand_ Clojure.
What I mean by this:
I was a Java programmer for years and increasingly started writing code in a more functional, immutable, dynamic style with the occasional need for meta-programming -- for the sheer need of being more productive and writing more robust code.
Yes, you can write functional, immutable, dynamic, code in Java and do meta-programming in Java!
It's just somewhat cumbersome and prickly.
Enter: Clojure.
I've seen many an 'unwise' programmer make just as much a mess of Java systems as people are suggesting they've seen in Clojure projects. It may be simply that Java slows you/your team down enough so that the messes of unwise programmers are just made more slowly.
But one wonders if this is really a win: you're probably also delivering your business value more slowly, as well.
:thinking-face:
I sometimes think I could have massively accelerated my ability to produce decent code in any language if I had been forced to work through "The Little Schemer" and "How to Design Programs" before I saw anything else.
But that is probably just hindsight or personal bias. I found functional programming in Lisp and Clojure to simply click for me in a way that enterprise Java class hierarchies never did . . .
I made myself a Lisp dialect with a nice little object system.
More precisely, I made it without that object system, but I eventually couldn't stand the situation.
I use objects even in small, throwaway programs used once. Sometimes it turns out that they aren't just used once, and the use of objects makes them easier to read later.
Because this in HN: Yes, some people are writing operating systems, video games, database engines. Most people are not.
They also don't want to throw away all their other valuable techniques.
Programming with mutation is not just efficient, it is also highly expressive and easy to verify, for the right kinds of problems.
Just because it's useful for writing operating systems, video games and database engines, don't be mistaken in assuming that it's only useful for operating systems, video games and database engines.
Yes, it is also absolutely necessary for expressive A* pathfinding.
This is unnecessary for someone connecting two web APIs. Correctness in most domains is totally worth not having to worry about pass-by-reference defaults.
Most Java programmers get enlightened and feel relieved as you did when they switched to language X from Java. Is a recurring theme, not specific to Clojure.
HN seems to blindly accept the idea that developers using niche languages are good but I certainly haven't seen any proof of that and I haven't seen anything to even remotely suggest that good developers gravitate towards Clojure.
Great developers have to be working on hard problems and almost all of the hard problems in our industry, outside of language/chip design and correctness proofs, are being worked on in relatively boring languages (C, C++, Java etc.). Most of the niche language users I've met are working on basic CRUD web applications.
However, I don't think it follows that great developers work on hard problems. Why can't you be a great developer working on CRUD apps and occasionally finding ways to do them better?
Maybe there’s something peculiar about academic work that requires an imperative mind. This is why we see the highest paid practitioners programming exclusively in functional languages ... oh wait, the opposite is true.
My theory is that FP hits a nice sweet spot for someone who would like to casually improve their craft. Something akin to breaking out Popular Mechanics after work.
* - excluding, of course, papers about programming languages themselves
Well I don't know how accurate this is, but in 2019 Stackoverflow survey, Clojure practitioners averaged the highest pay of any languages. (in 2020 they removed Clojure from the options so the data is missing).
> I challenge anyone to find a cutting edge CS paper* that is written in their favorite functional language
I rarely see papers written in any particular programming language to be honest, they tend to be pseudo-code and math like. And when you write a paper, you don't really need to be more productive or more reliable, since you write so little code. Plus any paper focused of low level or raw performance will obviously need something with semantics closer to the hardware.
> * - excluding, of course, papers about programming languages themselves
What about papers studying functional algorithms, type theory, automatic differentiation, parallel computing, and all that? Do you exclude those as well? Cause they're often written in a functional language.
Well I didn't believe it, but wow: https://insights.stackoverflow.com/survey/2019#top-paying-te...
I'm not sure if I should conclude that I'm incredibly ignorant of the engineering market, or that Stack Overflow's survey is not representative.
It makes me suspicious that Clojure is shown as the most highly paid language at $90k. Basically any non-new grad developer in a high cost of living area earns more than that, regardless of the language. I'm also suspicious that only 1.4% of survey respondents use Clojure, which may show a bit of a base rate fallacy here.
What would be the explanatory thesis for Clojure programmers being the most highly paid? None of the most highly paid roles I'm personally familiar with in big tech or finance use Clojure, and people who work in those roles earn well into the six (and sometimes seven) figures.
SO is used by developers all over the world, not just in the US.
I couldn't find a breakdown of the stats by state or city, but here's my guess: Clojure is only used at companies that can take the risk. That is, SF, NYC, and a few other metros. There are businesses that probably run Java or something on their cash cow and want to throw a few bones to the developers for recruitment/resume-driven-development purposes. Whereas JavaScript/C++/Java developers exist at every company and in every tiny city with a much broader pay scale.
Anecdotal, my company does some Clojure. But it's limited to a handful of internal tools and we pretty much ended the experiment years ago. Nothing new is made in Clojure. To my knowledge, we've never hired specifically for Clojure. It's always some dev that knows another language.
Right now my guess is that, if you also look at one of the other questions, it turns out Clojure has one of the highest ratio of senior to junior, like Clojure devs average a higher amount of experience as well. I haven't checked the others like F#, but my guess is that most dev using functional programming languages actively and professionally tend to be more senior and thus command a higher salary mostly. That means there isn't as many junior or mid-levels to skew the average salary down, where as my guess is other languages might have an even bigger number of new devs using them like JavaScript for example, which would skew the salary average towards their pay levels.
Seriously though, there are loads of papers out there that use ML variant programming languages (and not _about the language_) that I really wonder what papers you have been reading.
Let's not even get to all the older expert-system papers with stuff written in Prolog.
And yeah, loads of seminal CS papers are effectively just math.
Functional code and immutable data are fundamental ideas for managing complexity in big systems, irrespective of language. Even modern C++ tries to be functional until it has a good reason not to be.
(I also went to MIT, and I work on low-level systems at Google.)
If anything, as someone who has a Ph.D. and a career spanning both academia and industry, I can assure you that functional programming languages like Clojure is much more productive than the mainstream languages, even for implementing cutting edge CS research.
My experience of implementing algorithms from papers is that, in Clojure, the implementation is much more straightforward, always almost a direct translation from pseudo code in the paper to actual working code, even the pseudo code is inevitably written in an imperative style. However, the original author's code implemented in c/c++/java is always much more convoluted and bears no resemblance to their pseudo code in the paper. That should tell you one thing: that is functional languages often capture the bare essence of computer science, whereas mainstream language often have too much incidental complexity imposed by the languages themselves.
There is an opportunity cost and there are plenty of motivated developers that instead of learning yet another niche language decide to work on algorithms, applications and generally more difficult problems. Few languages offer enough advantages to overcome the fact that the latter is almost always a far better use of your time. As someone that knows Clojure relatively well, I'd argue that it's certainly not one of those few languages. Also, the trade off continuously gets worse as the popular languages adopt the good ideas from the niche languages and as problems continue to get more difficult and rely on teams of developers with library support in a wide variety of domains.
Similarly I feel my Python code is getting simpler and simpler over time.
I'd probably even gravitate towards Go at some point nur there I do muss a few things.
I usually find the pay off is just better when I rather learn more about deep learning, statistics, domain knowledge, security, visualization or stuff like AWS, Kubernetes, Unity etc.
The language ends up to be a very small piece and usually more important to have the ecosystem and community available. Like Julia also does not solve the 2 language problem for me but leads to a 3 language problem.
Isn't that just the definition of what a great developer is? Mind you working on a boring CRUD app may very well surface hard problems, but it's difficult to see how one could be said to excel at their craft without doing hard and difficult work to prove it.
It's very easy to tell yourself that you're a great developer because you do some arcane thing that nobody else understands in an arcane language, but it's very easy to mistake esoteric behaviour for skill. Hard, real-world problems are a reality check. You either can tackle them or you don't, and if you can't do it in a fancy language it's no good to anyone.
Gread developers do great work, subject to the needs of the business, regardless of the difficulty of the problem.
What about stackoverflow survey? Clojure is the most paid language (evidence that businesses value Clojure programmers) and Clojure has the most mature user base (evidence that users stay or flock to Clojure after many years in the industry).
Sure, not the best kind of evidence and depending on your biases you can read it in many different ways, but it’s a datapoint.
I feel like I am continuously looking for a better alternative, but it's hard to overcome the practicality of using something with such immense buy-in. For example, nearly all major SaaS offer a Java SDK, or have Java-specific docs. They want the big companies to be their customers, and they know they use Java. I am all too happy to benefit from that.
It should also be noted that Java the language is rapidly being enhanced with features inspired by functional programming. For example, sum types (in the form of sealed classes) have been added to the language, and ML-style pattern matching with compile-time exhaustiveness checking is in the pipeline. More on the platform side, Java now has a REPL (JShell).
Of course, Java codebases you encounter in the wild are very likely to make you want to cry. The world of Java development has a lot of bad taste and questionable practices enshrined as best practices. That said, the Java community is slowly shifting toward a better style.
If Oracle, IBM, Microsoft, Azul, Amazon bring a KVM written in Kotlin, then I might consider it.
In the meantime it is just the language Android team is pushing to get rid of Android Java, while the offices next door are pushing ChromeOS/PWAs and Fuchsia.
"final var" anyone? Non-monadic, non-applicative "Optional"? No? Ok.
The guest languages require additional tooling, duplicated libraries (because just using the platform libs isn't idiomatic, whatever), and then as the platform moves on the seamless FFI stops being so seamless, specially when the guest language decides being a guest language in just one platform isn't enough for its life achievements.
Eventually the platform language acquires enough features, which remove the spotlight from the guest languages, and projects start to migrate back, ah then the "Why X in Y" blog posts, with reference to "Why Y in X" start to appear.
As for Arrow, if you want Haskell, it already exists.
For what it's worth, in case I come off as a die-hard Kotlin fan, I'm really not. I strongly disagree with many choices the designers make with Kotlin - there's too many to list, but the common theme is basically, as good as Kotlin is at discouraging the worst of Java, it still caters far too much to nonsensical Java practices IMO.
Eventually Kotlin needs to decide which master it wants to follow.
Java only needs to bother with keep being Java, everything else in terms of OS and AOT/JIT/GC support is just a matter of picking the respective implementation with zero code changes.
This is a great feature of Clojure, being able to access all of the benefits of the Java ecosystem and the zillions of hours invested into it.
There may not always be a ready-made Clojure wrapper over libraries, but the java interop tools are excellent and writing your own interface into a library is usually quite straightforward.
It sounds like Clojure is perfect for you then. You have all the upsides of Java you mention here with Clojure because it's hosted on the JVM (among other platforms). You can use Java libraries and SDKs seamlessly from Clojure, but the language itself was designed specifically for a functional style from the beginning.
I've never seen a "best practice" that wasn't horrible dogma parroted by developers that can't do their own thinking.
As I get older I care less and less about FP, OOP, imperative, and language fads. Or Martin Fowler. Especially Martin Fowler. What I really crave and value over anything else: consistency. I don't want to look at the code base like an archaeologist, digging through layers of fads. "And this code is when James and Adam were going through an ORM phase with such-and-such library" Please no. I don't have enough remaining time in my life for this shit.
And developers would look at the niche language devs and go, why write in C? I have my COBOL and most production code is written with it. Java? Why? I have Powerbuilder and Delphi. Most of the developers with any new language are creating CRUD apps, because well, that's what most computer programs do.
At a later job my views got reinforced by a team who were so excited about Scala (many years ago when it was particularly bad, not sure if it is better now) and all the language tricks they could do in it. So we spent tons of time on "stupid programmer tricks" (Letterman-style) and debugging obscure language constructs, less time on the actual working product.
So I don't want clever programmers on my teams nor clever languages. Give me mature ecosystem, great tooling and simple code. Java and C remain the gold standard for getting things done.
From my personal experience, before I become a Clojure user, I wouldn't dare to write "hard" software such as compiler, database, and such, all by myself. Now I have done all that, I would credit part of that to the productivity of the language. That's why with such a small community, Clojure ecosystem has produced so many first of its kind innovative systems, e.g. Storm, Datomic, etc.
Lol. I guess your definition of wise developers is very different from mine.
Maybe the really wise developer is aware that not everyone thinks the same, even themselves now vs six months from now, and is very wary of dynamic behavior and cleverness...
I don't want Java or Clojure in a team environment, now that I've got Kotlin.
Or maybe I'm just not wise enough. ;)
It takes around 5 minutes to figure out whom you are talking to, when you are talking about Clojure.
This worked perfectly for me on both sides: as a contractor on hire and as an employer for some project.
I like to write dumb, obvious code. Most Clojure code is about taking your data, representing it a sequence of maps, and transforming them into different sequences of maps. The maps are open (easy to change over time), immutable (impossible to encounter data races, weird equality semantics, or concurrency issues), dynamic (no pre-definition or ceremony required), concise (thanks to a literal representation that does not even require commas between elements), and have a generic access api (no custom functions/accessors/etc).
Because I use the exact same transformation functions on EVERY PROBLEM, there is an enormous amount of reuse of generic operations both within and across Clojure code (even Clojure code that manipulates Clojure code, which is after all, just data).
Because the center of our code is open data, coupled with generic functions open to later extension (multimethods, protocols), Clojure is notably good at handling information systems that evolve over time in requirements (a feature of essentially all of them).
The core constructs are not hard, certainly they are easier to learn and use than complicated things like mutable classes and locking. The unlearning from other languages is often bigger than the learning. Nubank for example is certainly not hiring 100s of Clojure developers - in most cases they are hiring good people and teaching them Clojure. There are other successful companies doing the same.
Large Clojure programs can be hard to reason about because large programs are hard to reason about. One benefit of Clojure programs is that they are often 100x smaller than the equivalent program in a popular OO language. They are also trivial to interact with live in your REPL so that you can inspect the data flowing through them. I will happily take live data and interactive function execution over 1000 classes with custom methods. Both require time to learn but I'm much happier changing the smaller, simpler one.
The idea that "Clojure requires wise developers" is completely backwards. Enormous modern class/annotation based OO programs are the ones that require the smartest developers because that's what it takes to understand them. Clojure is accessible to all.
Clojure is also not making this easier(interop) by not keeping up with Java advancements. Ex: functional interfaces
[0] https://github.com/redplanetlabs/specter
Data driven languages need simple & powerful transformation libraries. `get-in` is repetitive and tiresome to use.
Specter and it's ilk make transformations clear and simple.
No amount of planning or foresight negates the need for data transformation libraries.
It's not a necessity in the same way XPath wasn't a necessity to interrogate XML.
I think a couple pain points need to be remedied in the community so that Clojure is more inviting to the general programmer, who like you mentioned would ultimately have an easier time with Clojure once bootstrapped into the language.
The primary one for me was being met with Emacs out of the gate as the preferred and touted editor for Clojure. I dove into that for a good year, became proficient with it but sorely regretted it. It was a decision point and cognitive hurdle I wish wasn't even presented to me in the beginning. IntelliJ/Cursive or something like it is a better way for most newcomers.
Earlier books on Clojure were way to "intellectual" in my opinion and pretty offputting to the general programmer. Luckily that area has improved in recent years with books such as "Getting Clojure" and "Programming Clojure"
Also the newcomer could be confused about what a repl is and how it functions with the IDE or editor. It would be better if editors and plugins focused on raw Socket or prepl repls over middleware repls like nrepl.
I would hope actually that some resources in the community would make an awesome LSP Server for Clojure so that any editor that supports LSP would have a great Cursive like experience. "clojure-lsp" is a great start but needs more resources.
The points made above by puredanger are so very true in my experience and I am so lucky and joyful to be working on Clojure systems.
A common situation is that a mediocre Java programmers reads the first page of a clojure tutorial, finds out about list comprehensions and map and filter (probably not even fold) and decides he (yes almost always he) is a genius. A “good, wise” “engineer”.
Reality usually disagrees.
The reality of actually having to write a complete working system in a language in which your knowledge is limited to the first page of a tutorial is even harsher.
When your “wisdom” and “goodness” are up against a hard deadline, or even an interview, you get the confused messages in this thread that blame companies for not hiring you, people for it understanding you, everyone else for being stupid, and the world for your unemployment.
May be learn what you started with first, but well, and thence to expand your knowledge in various directions.
But of this fancy stuff is available in almost every language. If you think clojure is special you’re in a bad place. Hickey wrote clojure in java.
For the record, I don’t write java or clojure anymore, and if you think clojure gives you an advantage without knowing at least five other languages to a professional level, then you’re not getting hired anywhere near me...
Great programmers can be great in any language, including C, COBOL, or BANCStar. Their brainpower alone can compensate for the language's lack of abstractive power.
Lisp is a mind prosthesis for the programmer; it extends the programmer's reach. Unless you are working in a complicated, abstruse line of work, you stand most to benefit from this if you are bad at programming.
Hence, the most clamorous Lisp zealots tend to be terrible programmers in their own right. The "wise" programmers jeep plugging along in Java, because it's well-supported by a bigcorp and has a HUGE candidate pool when you need to hire developers who work in it.
Whereas for me: TypeScript is concise enough, functional enough, and has the added benefit of static types and multi-paradigm features for when you need them. The primitives aren't as good as Clojure's, and immutability requires a library, but I'm not so traumatized by Enterprise Java that I feel the need to seek out the exact opposite of it.
Not to invalidate that perspective; I'm just wondering if the Java background explains the discrepancy between different people's feelings about Clojure.
I recently decided to give Pharo a try. I'm still pretty early on in it, but, so far, I've really been having fun. If you look at it the right way, Smalltalk can almost be seen as more Clojure than Clojure. It's even more dynamic, and the live programming environment gives even tighter, more integrated feedback loops than REPL-driven development. Smalltalkers aren't exaggerating when they say you can literally program from inside the debugger.
A lot of familiar programming idioms from Clojure (and other functional languages) are present in Smalltalk, and have been for years. And, while objects in Smalltalk are nearly as dynamic as maps in Clojure, the fact that they're discrete, named entities makes it a lot easier to inspect and grok someone else's code.
It's not all perfect, of course. Some of the usual criticisms of Smalltalk are legitimate (though Pharo is making progress on addressing many of them), and I would like to see more of a culture of discipline around mutability. But, all in all, it's largely been a joy to learn.
My impression/hope is that this is another spot where Smalltalk is like lisp. As a clean language with a simple syntax and clean semantics, it seems like where it shines is malleability. It's a substrate on top of which you build a domain-specific language that's tailor-made to whatever problem domain you happen to be working in.
There's https://github.com/pharo-open-documentation/awesome-pharo
I would recommend even taking the time to listen to the parts that seem basic. The Smalltalk way of thinking about objects is different from how OOP is usually done in some subtle but important ways, so I'm finding it useful to take a "forget everything you know" approach.
Somewhat amusingly, the process of learning Smalltalk in the evenings and then going back to Java during the day is going a long way toward helping me understand why Yegor Bugayenko seems so chronically exasperated.
> I would recommend even taking the time to listen to the parts that seem basic.
Thanks for that recommendation! This is my default style, although it's enabled by not knowing much anyway. Lol.
Regarding your last point, this is my exact feeling knowing F# and Racket before being forced to use Python in some capacities.
If you are a budget constrained individual living in a third world country with nothing to invest but your intellect and time, Clojure is a very good language. It lets you to - 1. Create relatively large systems without pulling your hair out. 2. Juggle between different projects with relative ease. Great for managing a projects in maintenance mode. Great for contractors with long term support contracts. 3. Realize your vision of a complete product with relatively low time and money.
Overall, it's a way of doing effective programming. It's probably not a good way of making money if the focus is to siphon investor money with a large team and millions of lines to show for.
Ideally, it would be great if clojure replaced Python and Scala as the defacto language used to interact with Apache Spark. That's probably not likely, though. It's hard enough to get a lot of folks past spark's weirdness without throwing a lisp into the mix. But it does seem like a functional JVM language that's not Scala would be the optimal way to use spark.
As much as I love Clojure I see the problem is you need really good (not just intelligent but also wise) developers to reap benefits. Otherwise you can run into huge problems.
"With great power..."
Clojure has so bad rep here that even I think I made a mistake that I let on that I use it for my projects.
This isn't one and don, people move on even from the best jobs. You know need to keep replacing them, explain to HR why this "req" (requisition - aka job opening) needs them to look at resumes differently. Why they can't pull from their Java pool of resumes, explain to your director why you aren't like all the other teams. It's a lot of friction.
Now, imagine giving the people environment that imposes no structure on your project and gives hyper powerful tools like macros and you are in a big problem.
At least with Java you get Spring and this is how you do endpoint, this is how you connect to the database, and so on. The structure is suboptimal and redundant but at least they are getting some guidance which are most likely already familiar with.
In Clojure, as a lead of the project, you would have to do the same but now it would be up to you to define all of that guidance. The question is, are you better at this than entire organization that supports Spring?
The only situation I see where Clojure would realistically be beneficial is when you only get like world class developers who know what they are doing and can work together.
I can't work on a Clojure project with a person that is unable to go through something like Let over Lambda with full understanding of the code examples and trust they will be able to make good decisions when building abstractions with macros.
And why, as a developer, you want to be working for a software company where your work is generating revenue, not somewhere where you are a cost to be cut.
Isn't this a cultural, organizational problem? You said your original developers left or moved on to management and your new hires are rather inexperienced. Wouldn't it be a wiser long term strategy to facilitate learning and knowledge retention through mentoring, documentation, teaching and attractive technical career tracks?
For example there are several popular open source libs/solutions to manage application state and structure declaratively. And it is well known in the Clojure community that macros are a double-edged sword and should be used to solve specific problems.
I agree that Clojure is kind of a second (plus) language. But I also think there is a reasonable path from writing simple Clojure to wielding the more powerful features.
Found your problem.
The first time you see a technology you immediately see things that solve some of problems you have, especially because authors tend to market pros more aggressively than cons.
Finding issues with technology takes some experience handling it.
For a relatively new thing that is in growth stage there is disproportionate amount of people who just joined and so have skewed perception.
I also thing people who would write about a product would usually write quite quickly after adoption but then won't follow up later after they got some experience.
There's some doublespeak going on here.
When "leadership" says "maintainable" what they mean is that the project is recoverable after they frog march the project lead out the door, or lay off the team, or the team quits because the job blows. Their view of maintainability is dominated by Bus Factor and similar existential concerns, rather than agility experienced by the project's authors. They actually want the opposite of maintainable: Rigid and heavily specified.
Even there, the projects suffered so much to hit release dates and stay understandable as the system grew that velocity suffered relative to peers doing experiments in traditional OO languages. In fact, the maintenance burden (for code composed by smart senior people!) is so high that it's actually convinced me on the virtue of regular OO/procedural languages over lisp.
This is an example of the problem of Clojure being promoted as Lisp. There are now people whose idea of what is "Lisp" is represented by Clojure.
You might like coding in a normal, procedural Lisp with OOP, but due to a bias induced by Clojure, you don't suspect that's even a thing.
With Clojure the mess is still 100k to millions LOC because the guys did not know how to make worthwhile abstractions, but now it can't be parsed by IDE and you are screwed trying to figure out what happens at runtime.
ALso, if you think if there is less code then it is easier to understand, go and read any advanced Common Lisp book (like Let over Lambda) and try to really understand what some of the more advanced macros do, exactly. It is definitely not the same as reading pages upon pages of redundant code.
Worse, the abstractions made it so that even a simple feature would require about six files. Not counting the tests.
Not to say that clojure has been a breeze. Worst I see there is developers refusing to use libraries it external programs and instead taking a ridiculous "first principals" approach. Even if they get it working, it is typically not search friendly, as everyone else is using some other tool. Basically, the same trap I see in build systems,
Spring annotations, maybe?
I have an enterprise client right now sitting on 1.5M java 1M SQL 1.5M XML for a single government system that mostly does not function at all, it just gets passed from contractor to contractor, each one sucking down a hundred million dollars before tossing the hot potato. Thanks for the money, Java!
Can't hire for it, can't refactor the code at scale, etc.
Be careful with what you read, newer languages are usually trying to sell themselves where established languages have happy people not trying to sell it, or, resume driven developers trying to sell it as bad.
This is not my experience whatsoever.
Frankly, VSCode is reasonably good at giving you something for Python, and the relative lack of static analysis is offset by other qualities of the language, so this feels like a pretty weak argument.
I am writing a little bit of C# atm and astonished how you can just dot tab through your work (without ever having read a tutorial or book on C#) and all those other contextual information you get with "full" visual studio. Somehow feels like just letting you guide through the work :).
On the other hand, the students I teach really struggle when they don't have all their IDE tricks. They learn almost exclusively C# at their institution in the first year(s) while my experience is mostly playing with Unity ;).
What if Clojure projects are more risky by selection bias?
I think your observation that you need really good developers is the key insight to problems that come about from using Clojure. Stu Halloway even mentions it in the interview:
"And what we would quite often see is that people would write a Java app in Clojure, or people would write a Ruby on Rails app in Clojure. And not surprisingly, it would have weaknesses that you associate with idiomatic Java apps or idiomatic Ruby on Rails apps"
Clojure, as a language, makes it very easy to do stuff and sometimes that stuff means making code look like code that only makes sense in other languages. This is a terrible mistake but one that is easy to understand. Very few developers have any real background in functional programming and it is too easy to start hacking things together.
As a longtime Clojure user (dating back to attending the very first Clojure conference in Durham, North Carolina), I think there has been a little bit of "ego-driven" developer syndrome with projects in the language. There is definitely a tendency for new Clojure programmers to get too fancy.
This is not a problem with Clojure the language though. I made the mistake myself in early projects. The more I have used Clojure, the more I have realized that when I am doing it right, I am spending significantly more time understanding the problem I am solving than throwing code together . . .
Ok, but how much of that is just them not knowing Clojure? I've found people that get handed down a code base in a different language tend to hack their way around it, instead of like, pausing for a minute, take some time to learn the language, and then come back to it.
Maybe in your case it was a terrible code base of low quality, but I'm just curious.
The worst codebases (and that is true of other languages just as for Clojure) that I have seen are ones done by people who are intelligent enough to learn every feature of their language and any little trick available on the internet but haven't yet had the chance to acquire the wisdom on when to use them or what to use them for.
...but just as pyspark is not “pythonic”, clojure spark is super unidomatic to use. ...which is why its languished unused.
I mean, an idiomatic library isn’t impossible; see how databricks has wrapped the pandas api up with its koalas project; you just have invest the time and effort and actually make it.... for what, fundamentally, is a total absence of demand.
or do you mean like first party support for clojure? Thats never going to happen...
For those that don’t know, Rich Hickey (and by extension, Clojure) is really big on data-first dynamic typing. So idiomatically you’re supposed to do most of your work in Clojure using their first-rate immutable collections library, primarily maps and their sequence abstraction. Clojure provides some mechanisms for specialized domain objects, but it’s fair to say that this is typically considered non-idiomatic.
This works great if one of the following conditions are true:
1) All data flows through your system on separate tracks. Foo’s come in from foo endpoints and go to the foo database, and very rarely do data sets cross paths. Differences in business logic can be separated by API endpoint, kafka topic, or some other difference that lets you separate the call paths thoroughly.
2) Every type of data in your system looks different, so that you can easily determine whether or not a given piece of data is a foo or a bar once and send it down the right call path in one place.
3) In any case where similar types of data must be treated differently, it’s possible to organize your code in such a way that you only have to build up the cond-tree once, and you can use different call paths to treat the data differently.
All of these were often false for us. We ended up in situations where we had lots of data that looked very similar entering from common endpoints that required different business logic at multiple points in the pipeline. If you’re writing idiomatic Clojure this is the toughest case possible, as you end up littering your code with extremely similar cond-trees, which makes extension and verification unnecessarily hard.
In most other languages, even dynamic ones, you’d solve this via sub-classing to both group common behavior and allow specialization. Clojure provided some tools to make this work, but they were both maintenance nightmares in their own separate ways. My favorite is how the results of defrecord (which makes class-like maps in Clojure) could be treated like a map, except certain map operations could accidentally turn it back from a record to a regular map. That’s quite the foot gun, trust me.
For our team, we decided that pure Java was the way to go. Java by that point had started adding a lot of boilerplate eliminating niceness, including lambdas and streams, and the ability to make honest-to-god subclasses really helped out in our specific domain.
Isn't this where you would use multi-methods? If you can determine what the "type" of data is via `cond` (sounds like home-grown pattern matching?), couldn't you replace these situations with multi-methods that switch on the same conditions?
If you can get away with one multi-method, that's fine. The moment you start needing multiple multi-methods in different points in your pipeline then things begin to suck.
> sounds like home-grown pattern matching
Yup! And if you find yourself constantly writing pattern matches in Clojure, then you probably actually want classes, since that's a much better way to bind both data and behavior together.
That sounds more like an indictment of functional programming rather than Clojure specifically. I’m not a seasoned functional programmer but I often read glowing praise from others about the use of pattern matching in their code.
I’m also not a professional Clojure programmer, but I thought that multi-methods were supposed to be the solution to the expression problem [0] that both class-based inheritance and pattern matching suffer from.
It was my understanding that multi-methods are supposed to allow open extension of behaviour without having to do what you just described; track down every instance, and cover every case.
In the case of multi-methods, couldn’t you just co-locate the method implementations next to the data they are supposed to operate on? You should be able to add/modify behaviours without worrying about what the other data types are doing.
Honest questions here. Just trying to learn more about the real world pros/cons of these approaches.
Also, I agree, it's interesting to hear his experience report.
"In object-oriented languages, it's easy to add new types but difficult to add new operations. Whereas in functional languages, it's easy to add new operations but difficult to add new types".
Put more concretely: if your Java code base is full of one-off classes with little to no sub-classing or multiple implementations of internal interfaces, then Clojure might work for you. But if you have a lot of sub-classes and repeatedly implemented domain specific interfaces I would hesitate mightily before considering Clojure.
My team's domain was such that the operations were very fixed, but the types expanded constantly. There is only a fixed set of things that the back office can possibly do with a bond or a future, but new types of instruments come into existence (or sometimes our awareness) with pretty surprising regularity.
But the article you linked had a slight of hand trick with how they described multi-methods. They only give an example one multi-method. What if you need to emulate a Java interface with M methods on it? Well now you'll need M multi-methods with an implementation for each virtual "class". You quickly see how that falls apart if you need to add a type or an operation, since either way requires you to manually check and make sure that all operations are implemented for all types. Again, it works, but it's error prone.
Defrecord & defprotocol work much better. We ended up trying them and they were ... okay. They work, with some footguns, most of which I can't remember anymore, but if you end up going too far down that road (like we did) you end up asking yourself why you're writing Java in Clojure instead of just writing Java in Java.
I'd say that it's a problem (not indictment!) of Lisp style[0] functional programming, where there's a pretty hard wall between how you organize your data and your functions[1]. If we'd been using Haskell, we could have at least declared a type class for our shared behavior and used that to handle the different parts of behavior in the pipeline.
I say problem with some reservations, because your mileage will vary depending on the domain in question. My current team could use Clojure quite effectively since the domain is much more amenable to how Clojure approaches these problems. My position is less "this doesn't work" and much more "this has some tradebacks you should be aware of".
> I’m not a seasoned functional programmer but I often read glowing praise from others about the use of pattern matching in their code.
Pattern matching is superior to if/else/else if trees, full stop. If you have to have a bunch of if/else trees, you'd rather use pattern matching.
The problem comes when you start repeating the same if/else conditions (not behavior) in multiple places. In a language like Java, you would naturally start wondering if you should make a class or interface to factor this out somehow. In a language like Clojure you don't have nearly as many tools laying around to collect common behavior into shared locations. You can do it, but you'll be happier the less you have to use multi-methods and defrecords.
> I thought that multi-methods were supposed to be the solution to the expression problem
Multi methods are ... ok. There are a few footguns laying around with them, since using them suddenly makes imports side-effectful, but I can and have used them successfully.
The issue is that multi-methods inherently give you one function, and there's no real way to tie several of them together at all. That's great if your domain is organized in such a way that you can use a single multi-method in one spot to break up the dispatch tree. It's less great if you need to use multi-methods multiple times during processing to specialize based on similar traits of the data. You can do it, but it's error prone and you'll end up with a nagging feeling that you're just making a really bad object system using multi-methods.
> It was my understanding that multi-methods are supposed to allow open extension of behaviour without having to do what you just described; track down every instance, and cover every case.
Yes, and it's this open-ended nature that kind of screws you in some cases. If I have four multi-methods that all need to cover the same cases (with different behavior), it's real easy for me to forget one when modifying my code. That same flexibility in adding instances means that there's nothing preventing me from forgetting one too.
> couldn’t you just co-locate the method implementations next to the data they are supposed to operate on
Well, the data came from Bloomberg, so there was nowhere we could put them that would be "next to" the data. We could put them all in one namespace, but that starts getting unwieldy fast. We could separate them out into different namespaces for clarity, but now it's even easier to miss that you've forgotten things. It's a tough trade-off.
Oh, and multi-methods really messed with our dev tools at the time. Hopefully they've fixed it since then, but a lot of the time Cursive Clojure wouldn't refresh them properly, resulting in dozens of REPL restarts for those of us that preferred Cursive. That got old fast.
0 - As long as you pretend CLOS isn't a thing. This isn't a big deal, since lots of people like to pretend that CLOS isn't a thing.
1 - Yes yes, I know "functions are data too", that indeed was a neat trick ... half a century ago. In Lisps the hard wall is in organization; there typically is very little binding the data to functions like you get with objects[0] or even Haskell style typeclasses. Getting the right data into the right function is entirely driven by how functions are called, rather than by the data itself.
My instinct is that you could solve general "where does this come from" and "where does this go" questions with metadata. I believe that was one of the driving reasons for metadata in the first place.
> 2) Every type of data in your system looks different, so that you can easily determine whether or not a given piece of data is a foo or a bar once and send it down the right call path in one place.
Clojure spec, malli, schema are libraries that deal with this kind of problem. They are very expressive.
> 3) In any case where similar types of data must be treated differently, it’s possible to organize your code in such a way that you only have to build up the cond-tree once, and you can use different call paths to treat the data differently.
A combination of spec (or others) and multimethods (arbitrary dynamic dispatch) would come to mind here as a solution.
I assume these things are known and were discussed. Are there specific trade-offs that didn't work out?
> I assume these things are known and were discussed. Are there specific trade-offs that didn't work out?
As is always the case when a team switches programming language, the problem I described was the straw that broke the camels back for us. But fundamentally the issue in this case was "it sure as heck looks like we're trying to approximate class based dispatch inside of Clojure, this sure as heck is silly. Let's just use a language that actually wants to be used this way".
The general problem you describe was also mentioned in the podcast. Trying to write like language X in language Y. This can also be observed within the Go community; people writing Java or JS style programs in Go, especially early on.
It's hard to discuss your particular case, but what I wonder is if there wasn't a different way to model things that would fit in the Clojure data-droven model. And if not, would there be new patterns or even some extension constructs to it that could allow it to fit?
Weird - what is so problematic about installing a JVM? That is literally an apt-get or install away. If you are learning Clojure then installing the JVM should be the least of your problems.
Note that there are a bunch of really nice Clojure starter kits available. And now many options for editors. For learning you don't need anything fancy, even a REPL will do to get you excited.
If the JVM turns you off then yeah maybe not try out languages built on top of it.
The problem for me is the JVM itself. I don't like all the excess complexity that comes from its unsurpassably enterprise-grade level of configurability. I don't like being trapped behind a distressingly awkward FFI. I don't like being required to adjust control levers for aspects of memory usage I'd rather be automatic, while simultaneously having little ability to control the aspects of memory usage that matter to me. I don't like having Java's half-baked type model hiding just under the surface no matter what language I choose, just waiting for a chance to jump scare me. et cetera.
Don't get me wrong, other platforms have their problems, too. Oftentimes they're even more of a headache. But at least they're different headaches. Yes, I like to program for fun, but that doesn't mean I don't like a change of pace on the weekends.
Is there a language that, for a sufficiently large application, you don't need wise developers? What is it? How?
I think that Java might have benefited the most from the industry hellbent on making microservices, in that Java applications are less frequently built as gigantic unmaintainable messes, but instead much smaller more easily digestible messes.
I am clearly biased against Java, but having worked with it at several jobs I just don't think it's a good idea.
Java is weak in small scope and gets stronger with larger applications as it scales quite well in maintainability.
In that sense adoption of microservice has hurt Java since you still need to do a lot of Java ceremony, tomcat + spring boot still starts up slow, you can't leverage superior IDEs to refactor interfaces across services etc.
For thin microservices I would probably tend to use something like node.js, for traditional monoliths (IMHO still have their place) with a lot of complex logic I would use Java (or Kotlin, C#).
I've written C, C++, C#, Java, JS ES5+, ActionScript, HTML5 and CSS since IE6, many various compile-toJS languages, many different frameworks and more. Elm is the only language where I can jump into a foreign code base or my own months later and feel confident that I can change something, push to production, and not break things.
edit: I've also written Perl and Ruby in production too.
edit2: How does it help? When you make a change that isn't valid, like adding a new argument to a function, the compiler can tell you everywhere you need to make a fix and sometimes even suggest how to fix things. Add on top of that the editor tooling in Intellij or with the elm-langauge-server in VSCode and other editors and it's just super smooth. Build times are also crazy fast. There's no waiting for the compiler. The whole notion of "If it compiles, it works" I find to be true 99% of the time.
You literally posted that using a statically typed language prevented breaking changes?
I can say from experience for every language above, save C++ (with which I have never worked), that they will fail to compile if you change a function signature in a backwards-incompatible way and fail to update all call sites.
The purpose of static typing as a method of program verification is to enforce type constraints at compile-time for all possible executions of that program. A function signature is, you guessed it, a type constraint.
"The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt." [1]
[1] https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Fr...
Basically I think that Go makes it easy to open and read almost any file, regardless of what you know about the rest of the project. I think that's great, but I also don't really think that it prevents you from building bad software.
Fixed it for you
Certainly not when they are spending hundreds of hours writing the next great LRU Cache or Bubble Sort so they can pass interviews every other year.
What does it mean to see a rather promising Lisp-like language not getting traction, for the the future of Lisp(s)?
I think this sentiment ("Clojure is dying") has kind of become a Hacker News meme at this point.
Clojure basically came out of nowhere and pretty much displaced all other Lisps in about a decade. It is definitely not dying, it's just an opinionated, fairly niche language - it's not Go or TypeScript. It solves certain problems well and has some uses where it really shines. It's not a silver bullet, though, and it definitely requires serious learning/unlearning to get really good at it if all you know is the mainstream imperative/OOP style.
Clojure works well for those who buy its rationale: https://clojure.org/about/rationale
The idea that you need "wise programmers" is a really big problem. People leave, institutional knowledge is lost, and then the code archaeology is just more difficult than it would be where types exist as guard rails.
I think Clojure looks favorable in comparison to Java still (though perhaps less so). But with Kotlin and Scala as mature alternatives -- and Clojure without a real niche -- it's not what I would reach for.
I say this as someone squarely in the latter category who has exactly the opposite problem from you: I love clojure the language and its features, but all the major libraries seem to be written by refugees from Java working at big companies who have a will to massively overengineer everything.
I'm thinking of libraries like mount and such which everyone uses for everything, and which I experience as, like, "did you want to inject your dependencies in your injected dependencies? Well first you have to do a quadruple axel inversion of control at the fifth abstraction layer out and use the fooflarb design pattern to delegate your inheritance and you thought you could even find the place in the code where it talks to the database? Good luck, sucker." Like, I always thought the point of a functional language was to not have to do any of that stuff...
Like, I vividly remember the first time I used a clojure web framework that brought in all that stuff, and I literally, I'm serious here, could not find where in the code it was actually talking to the database. Just indirection on top of indirection on top of indirection. And when the application I was building on top of that, which was supposed to be quick and dirty, started mysteriously losing data, the only thing I could do was just tear it down and rewrite with Python and flask.
If you've built large scale system with statically type checked languages that solve all these problems, I assume you never had any bugs right? Never needed to write any unit tests?
Clojure's design allows it to serve the needs of both hobbyists and enterprise, but the latter requires strong, disciplined developers who understand to avoid loose, hobbyist code that the language allows, and who will agree on which patterns to use in the codebase (something the hobbyist gets for free mostly). It's a punishing language for a project with high developer turnover.
Clojure is a bit noisier than lisp in terms of syntax, but it's also more expressive. I find the balance and design choices pretty reasonable.
With mount on the other hand, you tell it how to start and stop the component (and stores a reference to the component in the same var where you specified this), keeps a reference to the component in exactly one place, and then it automatically makes some reasonable assumptions about the order in which they need to be initialized, while allowing you to opt out as needed. Re: your example of finding a reference to the db, shouldn't it just have been in the var set up by a call to `defstate`? (Also, which web framework was it? Luminus?)
So I agree that it is harder for experienced people than for young kids fresh from school.
In my startup, our onboarding time is two weeks. I would say nine out of ten fresh graduates we hired picked up Clojure and were able to do something useful after two weeks. But we did have one or two persons who were not able to and they ended up in one of FAAG (yes, we did do leetcode style interviews, but only asked medium level questions).
What can be more challenging is to find more senior people who are decent Clojurians (it is hard to take a junior developer and make them as, or more, senior than you, for example :-) ). Hiring experienced Clojure devs is also doable, but takes time, flexibility (e.g., open to remote... easier in 2020 than in 2019), and real effort.
People who already knew Clojure had a tendency to overcomplicate things with solutions that used every bit of their Clojure knowledge.
Apart from that, I think one thing that's missing for larger scale Clojure is best practices and common tools, IDEs, frameworks and libraries. When I compare it to Java, Java has so much ingrained best practice and information out there, even junior devs will quickly pick up a book about its design patterns, and all that. And basically everyone uses the same tools and frameworks, its always Spring or Guava, IntelliJ or Eclipse, Jetty or Netty, Log4J or Logback, Jackson, Hibernate, etc.
I might agree with you slightly on some of the ackward bits. If transducers had been there from the get go instead of lazy sequences. If named arguments were first class and the compiler could check that the mandatory arguments were passed in. If exceptions were functional, and pattern matching was added to compose them. And if a few more core functions/macros were added for convenience. I think it would facilitate a bit adoption on larger projects.
I don't use Clojure professionally, and have often wondered how well it holds up in companies where avg tenure of an engineer is 2-3 years.
I guess that assumes that a tenure of 2-3 years is not a problem in itself.
When I was working my way through Clojure for the Brave and True, I really liked all the clever techniques around data-level programming and informal interfaces. Not having to stop to define and name formal types lets you slap things together quickly. But I'm now seeing the where that approach leads to: A codebase where nothing is formally defined, and nothing has a name.
I see where there were some efforts to do it by the book, and formalize this stuff by using functions to define an interface. The problem, though, is that we live in an orderly cosmos, and some laws are universal. One of those laws is that people will always take the shortest practical path to get from point A to point B. No matter how politely you ask them not to. So, if a map can be manually hacked to shreds in an ad-hoc manner, it will be manually hacked to shreds in an ad-hoc manner.
- Scala is much closer to Java making it an easier sell to other Java programmers.
- I think static typing makes a lot more sense for larger projects
- I found a rich static type system easier to use and more generally applicable than macros.Because of the difficulties of sharing knowledge, Clojure will have an extremely difficult time growing.
I just started to work on a quite large Clojure codebase and I had no trouble making meaningful contributions within a couple weeks.
Having mostly pure functions makes it especially easy to understand what is going on. I had similar concerns, and this myth is hard to kill - for any dynamically typed language - but the REPL is your secret weapon.