Clojure isn’t for me (2020)
swlkr.com
swlkr.com
IMHO. Maybe I overexaggerate. Anyway, the author, from my perspective has decided that clojure IS in fact for him, but clojure on the JVM is not, so he chooses to use a different one.
I've been told this a couple times. I doubt I'd thrive in that world though, unless I was selling something I actually gave a damn about.
IDK, is guile a scheme? guile is it's own thing, but it's definitely a scheme. What about racket? Racket has a whole bunch of stuff that mit-scheme doesn't. But it's still kind of in the scheme family. That's the parallel I was drawing. That and the statement that Hickey made in his History of Clojure that Clojurescript _was_ clojure, not some kind of spinoff.
I'm certainly not detracting from Janet and Fennel, but they _are_ quite close to clojure proper. There is a clear lineage.
I would generalize your observation as a logical consequence of Greenspun's tenth rule and conclude that we will eventually get a hosted lisp for any sufficiently relevant programing language.
However, the broader lisp community writes code very differently from typical clojure. Different idioms, slight syntax variations, completely different tooling and ecosystems.
While I can agree with you that for 'us' it certainly feels emotionally true. But if you go into other lisps scheme, racket etc you would notice huge differences. Differences which may feel even stronger than java vs c++ diff.
As someone who has written Ruby for years and more recently switched over to the JVM, I don't see the big deal. Both ecosystems have their advantages and disadvantages. The fact that so much of Ruby's ecosystem is built on C has its downsides too, e.g. compilation errors on "bundle install" because your version of nokogiri doesn't work with whatever C libraries are installed on your system anymore, or a frequent need to install new ruby versions which can be rather slow when you have to compile them from scratch.
Clojure (the JVM runtime variant) is not compiled to JVM bytecode. It’s compiled to Java implementations. This detail isn’t very hidden, and few affordances are made (or were when I worked in Clojure) to accommodate the indirection when something goes wrong. When you get an error, you’re now walking down this stack:
0. Debugging the build (which has all of these steps before you might even run code)
1. Debugging your own code
2. Debugging Clojure source written in Clojure
3. Debugging the Java implementation of Clojure
4. Debugging Java’s runtime behavior
Most often, Clojure the language doesn’t do anything to make step 1 or 3 any easier. It’s just a call stack from whatever .clj source into a Java stack with no context whatsoever.
This is incredibly hard to debug, especially because a lot of other important details are hidden. For instance:
(let [foo {…}] …)
ClojureScript is fundamentally the same, except:
- It’s got a whole additional runtime to loop through
- The cljs maintainer is (or was when I was writing cljs) actively hostile to offers to contribute changes which would make this less painful. Literally close as wontfix, rejecting offers to contribute
Nearly 100% of these experiences are leaky language host marginalia (ie type errors caught in an unhelpful place at runtime). And I wanna be clear: I love the concept of Clojure and don’t regret any of the time I spent working in it (though I do regret some time with cljs). But the probability of encountering incredibly confusing host language stuff very quickly approaches 1 if you’re developing actual things for real users. And then it gets really “hope you have a good mental map in memory” really fast.
Or you're just annoyed that stacks includes Java frames? In CIDER you can make it only display frames from your own code or only Clojure frames. Then you don't need to see Java frames.. and it'll look equivalent to being compiled to bytecode...
When I did work in Clojure/Script, I would regularly find bugs I couldn’t understand without stepping through core libraries and the runtime to understand them. Often they were my own bugs, sometimes bugs or just non ideal behavior. Hiding the stack frames would have been a hinderance for these cases.
In actuality, Java is pretty great and a lot of the pain I experienced using it was not because there was something inherently worse about Java’s design decisions but rather because I had made the (very incorrect) assumption that one could simply port over a lot of the unstated assumptions we make about the semantics of other languages when coding in Java, which isn’t true.
Isn't that the crux of the general criticism of Java's generics though? That it's ugly and deeply limited precisely because it was designed around a hard requirement of backward-compatibility?
You know anything about .NET?
I feel that the two are being confused here (even though this post is literally about another language running on the JVM).
People just like to bond by hating the same unpopular things. It makes them feel 'in'.
I guess in a way they are right, given the various JVM implementations we can chose from, with capabilities the Go runtime will never be capable of.
For example: I mean, you can monkeypatch anything (including _Object_) at runtime. You can use this to make 1 + 1 = 3, if you desired, or to add a method to all instances of any class currently instantiated. Want your database to respond to .fuckoff(), no problem..
This kind of stuff is also how rspec (used to?) actually run tests.
It's cool, but also pretty insane.
irb(main):001:0> class Object
irb(main):002:1> def fuckoff
irb(main):003:2> "lol"
irb(main):004:2> end
irb(main):005:1> end
=> :fuckoff
irb(main):006:0> 5.fuckoff
=> "lol"
irb(main):007:0> "Hello World".fuckoff
=> "lol"
irb(main):008:0>The bad thing is that this is often abused horribly, I blame the Rails ecosystem with its "just add this gem to the Gemfile and it will magically change your application" attitude.
https://kotlinlang.org/docs/extensions.html
https://docs.scala-lang.org/scala3/book/ca-extension-methods...
In Ruby, however, there is no concept of an import. You can require other files but this is transitive. So if something anywhere in your chain of dependencies is defining a new method on String, or Object, it's available everywhere. Plus, in Rails usually you don't even have to require anything, as the framework magic does that for you.
The fact that there is no proper import/module system is easily one of my most disliked features about Ruby, it makes it incredibly hard to understand where things come from.
Also, Ruby doesn't only allow you to add new behaviour to existing classes. It also allows overriding existing methods. And that suffers from the exact same problems, i.e. it could be in some transitive dependency, and suddenly your strings will be doing things you didn't expect them to do...
F fuckoff(x) "lol"
echo(5.fuckoff())
echo("Hello World".fuckoff())But you get optional typing too for speed
I guess I was trying to describe the frustration of switching languages and ecosystems while also trying to ship, thought I could do it, but I couldn’t.
I suppose I would have had a much better time and stayed in C-land (and gotten much better performance) if I had tried golang instead of clojure.
I switched to the JVM because I was annoyed enough at the things I had to deal with in Ruby that learning how to do things in a new way was worth it for me in the long run.
> I suppose I would have had a much better time and stayed in C-land (and gotten much better performance) if I had tried golang instead of clojure.
But Go is a completely different language than Clojure? By what criterion are you deciding which language to focus on? Mostly I'd think that people who choose Clojure do so because they want to use a homoiconic, dynamically typed, functional programming language and not something that looks a lot like a cleaned-up C.
Ruby (and rails) for web applications is very hard to beat ergonomically.
Go also feels this way to me, the development experience is very simple, deployment is even easier than ruby/rack (assuming no docker), and you get the added bonus of using a lot less memory on the server.
I've been doing java for years and the key I've learned about the JVM is to stop tuning GC parameters.
The JVM has incredibly good heuristics provided you let them work. The most you should generally do is pick the algorithm and the heap size (maybe the pause time) doing more should only be done if you've got a large amount of evidence (GClogs) to support such changes. Way too many people shoot themselves in the foot trying to hand craft GC settings.
Beyond that, I've found that application changes are almost always more effective than GC changes. Use the JVMs fantastic telemetry tools (Java flight recorder) and find out what's really going on with your application. Again, grab the data first, optimize, and then remeasure to see what's next.
I've managed plenty of apps with 30+gb heaps and 0 tuning.
In general, I don't really encounter these problems and neither do most other people who write for the JVM.
I'm sorry, but that sort of message is a pretty strong indicator of programmer error.
Are you using try-with-resources to properly close out things?
Are you creating threads in an unbounded fashion? (new Thread)
Are you keeping references to objects for longer than needs be?
Have you used Java flight recorder to identify the root cause of you memory pressure or used eclipse MAT to find large long lubed allocations? (Or both?).
Are you using finalizes? Those have been STRONGLY discouraged (and for good reason) for decades now.
One of the many features of Java is that it is advertised as being easy enough that any idiot can write it, and managers then hire idiots.
My condolences, that sucks. I'd still suggest using the profiling tools I've suggested to find these issues and create change requests to said libs. People will think you're a wizard if you simply learn how to use Java's telemetry. It's an invaluable skill.
> Except using finalize for closing pipes for subprocesses, that was by Java standard library itself
Java uses finalizers as a last resort, not a first line. Finalizers are due to be removed from the language so don't count on them being around for long.
> And keeping references longer than needed, that is every library that implements clever caching, ever.
Sure, caching is legitimate when the cost of creating exceeds the cost of long lived objects. Have you measured to see if the caches you have are improperly sized? Have you made pull requests/change requests to said libraries to fix it if it is?
> Regarding try-with-resources, RAII was invented for a reason. Java did not copy it, because they seemed to seriously believe that GC is a good substitute.
RAII was, few languages have it. The languages that don't have it generally don't have it because tracking the lifetime of objects in a GCed language can be extremely difficult. Consider, for example, how java would handle a destructor for an object allocated in one thread shared with multiple other threads.
Do you know how C++/Rust/D solve that problem? Through a system of reference counted smart pointers which have to be handled very delicately. Java allows such objects to be freely moved.
There are pros and cons to GCs (like most language features) so instead of just defaulting to "man, GCed languages suck because the finalizers aren't called when I want them to be!" perhaps it's a better route to learn the idiomatic way of handling resources and use that?
People who want to hire me to fix their Java app think already think I am a wizard. The problem is, they stop believing me the moment I tell that Java is not a good language completely stop listening when I say that I will not work with it as long as I have other options.
> Java uses finalizers as a last resort, not a first line.
At the time it was the only option. Or, reading the JDK source revealed the undocumented feature that destroy() closes the pipes too.
> Have you measured to see if the caches you have are improperly sized? Have you made pull requests/change requests to said libraries to fix it if it is?
Wasn't the point of using JVM and pre-made libraries that they would work out of the box? Anyway, I have found out that terrible libraries are terrible because their authors either genuinely want them to be that way or because they do not recognize good code when they see it. If you have managed an open source project you should know that pull requests out of nowhere are mostly pure nonsense and tend to be ignored.
> Do you know how C++/Rust/D solve that problem?
I do, thank you for asking. As far as I understand, Java does not allow moving objects any more than copying a shared pointer or moving unique one does, but it pretends that objects have no ownership.
> perhaps it's a better route to learn the idiomatic way of handling resources and use that?
It does not solve the problem. The actual solution would be more like "magically make the CTO's nephew and all authors of the zillion libraries the app depends on learn the idiomatic way of handling resources and then rewrite everything using those".
If people hire you to "fix their Java app" and you then subsequently bury your head in the sand and insist that this cannot be fixed (or you will not fix it) without rewriting it in another language - either you have been lying to them, or they were not listening to you. Regardless, you were probably not a good hire.
It seems that you're trying to write Java as if it were C++. That will not work, so stop doing that.
> "magically make the CTO's nephew and all authors of the zillion libraries the app depends on learn the idiomatic way of handling resources and then rewrite everything using those"
Crap code is crap code and exists in every language. I'm not sure what your point is. Most battle-tested libraries in Java properly use try-with-resources. If you work for a crap startup that uses crappy libraries, that's not a problem of the language itself. All this criticism of "the CTO's nephew" is a deflection of any conversation about benefits of drawbacks of individual languages since you just try to measure a language by its worst users.
To be honest, most of these cases have been what could be called bait and switch. I think it tells something about the language that they can't find employees if they are honest about using it. Other times it had been because recruiters just decided that I'm a Java person and can't work with other technologies.
> It seems that you're trying to write Java as if it were C++. That will not work, so stop doing that.
If you can read my thoughts, James Randi has million dollars for you. I have only contempt, and no you didn't get it right.
I don't think anyone should use finalizers in Java. It's been deprecated since Java 9 and even before that, it was widely considered a bad practice.
FWIW, I haven't ever seen anyone use finalize() in Java. I don't doubt that it happens occasionally, but you can't blame a language for some of its users being incompetent.
Almost every decision in Java feels like it was chosen to waste memory. Even using less than int size ints is hard, because it makes you write explicit casts everywhere, which doesn't prevent bugs, but then makes overflow wrap silently, which encourages them.
I've since abandoned janet for web app side projects and gone back to ruby entirely but yeah, I still love lisp but it makes more sense to align my work programming language with my side projects that way I can ship my side projects faster, at least that's the thought now.
As a Clojure fanatic, I too enjoy your writing and hoped to stir up some interesting discussions around Clojures weak-points by submitting this story.
According to the article, the author moved to Janet, because it is Clojure-like without the JVM overhead.
[0]: https://github.com/babashka/nbb [1]: https://babashka.org/
He analyzed his needs and wants.
This is a good example of what "right tool for the job" analysis looks like.
Though honestly, if OP wasn't in "java land" (his phrase), I'm not sure why Clojure was even on his radar.
I greatly respect people who reject the hype and just go with the language that fits their needs, even if it isn't hyped like C, Ruby or Python but provide plenty of what you'll need.
Clojure is really appealing as a language and has enough of a following that it doesn’t seem like a risky bet. It also looks pretty self contained. It’s only when you need to push it that you find yourself with confusing JVM stack traces and all the nuances of that ecosystem. When you do you start to figure out the whole thing is not quite as elegant and simple as it first appeared.
Today it would be something else, like rust or zig or something.
But yeah it's difficult for me to build my tools and ship finished web applications at the same time, it's compounded by learning a new language.
This is something I've struggled with for a long time. I still struggle with this, the urge to create a new web framework when I would have shipped much faster if I had just used some boring old one for example.
the pitfalls of old boring tech have known solutions, while the new tech may run you into trouble you don't know you can run into?
well, gee, that was totally worth the 50 slides that i had to scroll through (and i don't know why the website was so slow)
looks like a web-dev is coping with the realities of web-dev, nothing more
C libraries are wonderful tools when you need a low-level library to run hot, but if you need, I don't know, a database adapter or a queueing library and you don't want to write one, I'd much prefer pulling something out of maven than trying to integrate a C library. There's just a lot of infrastructure the jvm gives you pretty easily.
If you're just messing around with fun side projects, there's nothing wrong with using something you enjoy for no other reason than your own personal enjoyment.
That was my thought too. If it's a side project, I guess you should just always use Python because you will never need anything else. But I already kinda know Python and it doesn't interest me much, so I write my side projects in the flavor of the day that has my fancy at the moment.
I had to write some proof of concept code that talked ISO-TP on the CAN bus and guess which language other than C had libraries for CAN and ISO-TP? Python. Fortunately we found a serial to CAN gateway and I was able to do it in Ruby over the serial port. Soon we'll have to rewrite part of it in C/C++ on some embedded board, probably ESP32 because it has CAN and WiFi built in. But I think it's perfectly doable in micropython as well. ESP32 CAN bus support for micropython is currently being worked on.
The java-world is an inscrutable place of gigantic IDEs, maddening dependency graphs, perverse build systems. Everything is just-so-poorly-compiled enough that it's a nightmare to work with. It sucks all the joy out of programming in my experience.
And it's not like this is a slow language, Java is like 80-90% as fast as C if you know what you're doing, and for most use-cases, and isn't brain-dead in its handling of threads.
I'm a C++/Python developer, by the way, although I've had day jobs writing Java too. I honestly think this is way off base; it's a great language which has constantly evolved, and the JVM is a fantastic piece of technology. Rich Hickey was super smart in targeting the JVM, it was a great decision, and to my mind OP hasn't a clue of what he's talking about.
The idea is that there's a "cluster of C-based ecosystems" of languages whose open-source philosophy, programming culture, etc. "is just fun". In contrast with the JVM ecosystem which, I think, just isnt.
Or that of Python where the package managers _globally_ install dependencies and you need virtual environment shenanigans to not break everything???
It also seems that the JVMs original purpose of running the same code across operating systems/architectures has also been superseded by containers, better cross compilation (llvm) and possibly wasm as well.
Maybe at some point in the future, the JVM will mostly be relegated to legacy systems and most new software (assuming AI isn’t writing all future software) will target wasm or require some container runtime.
I mean sure, most modern JVM apps deploy to containers, but developing in container images is quite another thing and requires you to understand and debug a lot of issues (e.g. mounting, caching, networking) that you'd like not to care about (especially if you're not particularly an infrastructure/ops person).
This might also come down to editors and things that I didn’t cover in the post too.
There’s probably a happy path for clojure/jvm via an IDE that I didn’t try (I use neovim) or something else I might have missed in my clojure years.
C (and C++), on the other hand, is a trash fire of macros obfuscating code, header files (WTF they're still here in 2022?), libraries which are hard to import, barely working IDEs (IntelliSense in latest Visual Studio takes 30+ seconds to update itself after every change in my small C++20 pet project), people redefining primitive data types in a hundred different ways, unusable compiler error messages etc. Every time I have to work with C++, I'm getting angrier by the hour - and my codebase is only a simple personal project!
While IDEs were actually born out of Xerox PARC workstations and Lisp Machines.
Languages without such workflows, stuck in the days of teletypes are what suck the joy out of development experience.
> The Janet language is implemented on top of an abstract machine (AM).
Greenspun's tenth rule comes into my mind.
For comparison, the JVM has been developed for almost 30 years now, hundreds of thousands of hours were invested in it, runs virtually on every platform on a planet. It has advanced JIT and GC capabilities, and can compile to native.
Even Clojure has been around for about 15 years now, and it has successfully carved its niche in the Java world.
Janet on the other hand... is a 4 years old hobby project? No disrespect, but they are not in the same league.
Just a side note: it is probably the easiest thing in the world to implement your own LISP - it is a fun thing to do, you can start very small and get something working within hours. So if you are into it, you should probably roll your own too!
When you start with "no disrespect", with high probability what follows is disrespectful. Did he make the claim Janet was in the same league? He starts by clearly mentioning this is in the scope of his side-projects, so enjoyments trumps marketability.
clojure was cool, I used it mainly for datomic, but the gains really dont justify relying on public slack channels and mailing lists to hire and find devs.
I really see this from an economic point of view. I could be mistaken based on my own experience dealing with clojure and datomic in particular. I'm just not sure the gain is there anymore as other databases built on conventional tech stacks increasingly more or less take away the gains.
It describes how no technical improvement alone can get an order of magnitude improvement in software development efficiency, so the ways available to increase development by an order of magnitude are learning to use already-built components, shrinking feedback loops so we can be sure we are building the right thing, and growing the profession.
It would be nice though to understand what the author believes the issues with the JVM are since they they aren't expounded upon in the article.
The problems with the JVM are more cultural or political and not necessarily technical, although at the time clojure on the jvm did require quite a bit of memory and the cold startup times weren’t great for a simple, “watch this file in development and restart the server” workflow.
Fortunately, no one in the Clojure ecosystem works like that, thanks to the REPL :) You fire up the server, send code straight from the editor to the server and evaluate everything on the fly, no restarts needed.
(defonce current-count (atom 0))
(defn handle-request [req]
(swap! current-count inc)
(str "Hello " (-> req :params :name) @current-count))
(run-server handle-request :port 8080)
Now imagine it with more complex state, or with things that require multiple states to do something. Reloading the server each time would mean you have to manipulate the state into the right "shape" each time before you can test your change, while if you can retain your state even after changing the code, you'd avoid that.
This is why I'm still stuck in Clojure and unable to go back to writing code in other languages. The productivity gain is simply too high to throw away by not using Clojure (or other languages that are driven by the same ideas).
Microsoft calls it "edit and continue". Not sure what Oracle/IBM calls it but it works.
It's not guaranteed to work and there's edge cases where it silently corrupts stuff. Function pointers of course don't update, for example. The Clojure way is cleaner.
https://docs.microsoft.com/en-us/visualstudio/debugger/how-t...
https://dzone.com/articles/hot-swap-java-bytecode-on-runtime
There have been tech demos showing that LLVM can do this for C++ compiled to machine code. I don't know if it's really working anywhere, but it certainly can be made to work if someone puts in enough effort.
I once actually established a debugging connection from eclipse to a telephony server running on JVM that was actually serving calls and live-replaced the dialplan handling code, stopping a ddos without making things worse. It even works for that case. Not something I'd advise doing, but I love that this can actually work ...
In the former case (it's useless) the stop-the-world-and-restart approach doesn't hurt anything, in the latter you would need to fix the program anyways.
Although it is nice to have them written down and more explicit
Not sure how it is better than Ruby. But maybe the author wants something fresh to play with - that I can totally relate to. Good luck with the new journey anyway!
I guess he also never saw the price of ISO C document that someone has to pay so that GCC actually supports C the proper way.
Talk about being precise on the details.
In fact, one large banking system I worked on in the late aughts completely revamped their megalith of a back-end for a major component of their system and the only two things they didn't change were using AIX and Oracle.
I’ve always wondered if there’s a way to know at compile time whether a value is being mutated, rather than throwing a runtime exception.
And in any case, I'm wondering about compiler-writing resources, not compilers.
I daresay you didn't even read what I wrote and just typed "hurr, Rust is good." I've seen a million comments like this, and it's largely the reason I'll never bother.