Embracing Common Lisp in the modern world
juxt.pro
juxt.pro
1. SBCL, the most popular open source implementation of Lisp, is seeing potentially two new garbage collectors, one of which is merged and open to beta testing. One of them is a parallel collector written by a university student (!!) which blows my mind.
2. SBCL has better and better support for deploying Lisp as a C-compatible shared library, using SBCL-LIBRARIAN. It makes it play nicer with other applications in C and Python, without the baggage of process wrangling and thick RPC (IPC/servers, serialization, ...).
3. Coalton is another exciting development that allows a Haskell type system and "Lisp-1" functional programming in Common Lisp. That means type classes (or traits), something Lisp hasn't really had a proper notion of, and full type inference. Persistent sequences based off of RRB-trees were recently merged [1], and interestingly, they're implemented purely in Coalton [2]. That means Clojure-like seqs.
It's interesting to see users of Lisp generating the above ideas and libraries, not a special in-group of committees, "official" developers, etc.
[1] https://coalton-lang.github.io/reference/#seq-type
[2] https://github.com/coalton-lang/coalton/blob/main/library/se...
Now I haven't coded extensively in clojure, so a lot of it stylistic unfamiliarity. It does hold back people from adopting clojure.
One reason why I'm excited never to use the JVM again is the huge memory footprint it uses. The best practices for production java is to figure out how much ram your process will need max under normal (not leaking) usage... and set both max and min ram to that (Xmx and Xms). It's a real hog on ram, and when running under clustered tools like k8 with can do memory overcommits it really reduces your flexibilty.
I'm super psyched for all these tiny go and rust programs that run and use like 50 mb ram or even less. Running a cluster of 50 containers is really no heavier than a single Xmx1g java process!
If you want to see Java going into where Go and Rust hardly matter, see real time Java used in embedded deployments by US and French militar, in battleship weapons control systems, and missile tracking systems.
For interop? Yes. For debugging a JVM in production? That, too. But the same can be said about ClojureScript and ${JAVASCRIPT_HOST}.
I picked up Clojure after a few years of using Common Lisp. Everything I know about Java and the JVM is thanks to using Clojure and deciding to make use of Java libraries when the ecosystem lacks a good library for something I need (e.g. WebAuthn).
My complaints about Clojure usually boil down to lack of performance (when writing idiomatic Clojure as opposed to "Clo-java") and lack of static typing (most Common Lisp compilers do a way better job here). Although I complain a lot about performance, it's ironic many Clojure users will point out to me that Clojure is still faster than Python, Ruby, etc., because I never use those languages and beating them in performance IMO is a very low bar to clear, unless you consider alternative implementations like Truffle Ruby (but then most of the Python/Ruby/etc. ecosystem is incompatible with your implementation).
With that said, Clojure is the only language I know with something like clojure.spec out of the box, and every time I use other languages, I miss being able to treat schemas as data and run transformations on them, especially when dealing with data that has a lot of optional fields (but they are optional only subject to specific conditions). Avoids a lot of repeated subtypes and keeps code concise. I also have an easier time testing Clojure code compared to other languages, in the sense that I rarely have to maintain intricate hierarchies of mock objects and state. Most of my Clojure code tends to be small, pure functions, and stateful things happen at the boundary between e.g. backend and database. So the most complex fixtures I usually have to write is setting up a database.
> there's a lot of baggage like emacs + cider...etc.
I think you were trying to use Clojure in its very early days, when people were still hacking atop of inferior-lisp-mode in order to provide basic Emacs support. While Emacs is still the majority of editor usage in Clojure, the next significant chunk is VS Code users with the Calva plugin. There is also an IntelliJ plugin and it offers (IMO) the best experience for Java interop.
In the recent 2023 survey, Cursive and Calva are pretty much neck and neck, and between them just edge out Emacs. So it's pretty much an even split these days between Emacs and more "modern" options.
https://www.surveymonkey.com/stories/SM-_2BH3b49f_2FXEkUlrb_...
I've been working on a library that targets the JVM, JS/TS, and the .NET CLR. Clojure has let me share about 85% of the code across the three, though to your point the remaining 15% required quite a bit of platform specific knowledge. I'm fortunate to have worked with all the over the years, so Clojure has been a nice power tool that has let me accomplish a lot while working solo. It's not the right tool for every person or every project, though.
Since you've worked with Python, you might enjoy Basilisp. It is essentially Clojure with Python as the host platform.
That was also around the time people had started hyping the JVM as a general language host. Seeing how people struggled on the Clojure mailing list made me rethink that proposition.
I worked on python, ruby, Javascript and Perl before Clojure and I seem to recall they had stack traces too. Am I misremembering?
The Clojure stack traces are certainly longer. Which is because it’s a hosted language - the Java stack trace plus the Clojure stuff. I suppose the extreme indirection in Java might be part of the problem, where code passes through so many intermediate classes.
Doesn't matter if it is the browser, UNIX, Java, CLR, BEAM,.... eventually that underlying layer will catch one up.
And where is this quote from?
The blog post itself is not very informative, but the quote is poetic, memorable and perfectly captures the sentiment of some Common Lisp enthusiasts towards Clojure.
There are at least 4 different options for Clojure fans that dislike Java. BB, ClojureScript in node and browser, ClojureCLR (.NET), ClojurERL. (and a few more).
If you love static types, use TypedClojure.
It is like many things in Clojure a la carte.
It shows Clojure with 4x the share of CL though. Lisp-1's hurt my brain though, so I haven't dabbled in it much.
No worries!
(defn cl-funcall [function & args]
(apply function args))
Having variables that shadow functions, that are later used somewhere causes rather inscrutable jvm backtraces to appear in Clojure.
(defn f [name list]
#dbg
(concat list (list name)))
(f 'alice '(bob))Incidentally elisp does a lot better in new Issue count too, although clojure narrowly beats it PR count. On the whole though, I think if you define mainstream to be number of people using it, then elisp probably edges clojure.
Calling Clojure a Lisp triggers some people.
To me Lisp means programs in list syntax with prefix notation.
C is more than foo{bar}.
A programming language has dimensions like: syntax, semantics, pragmatics, implementations, standards, practice, tools, social/community, libraries, code, ...
Clojure has also successfully spawned in the last years enough variants/tools/libraries/practice, ... , that it is now a diverse language family on its own.
side note: only two years ago we were complaining about the lack of support for VSCode or the Jetbrains suite, now we have working plugins for them: https://lispcookbook.github.io/cl-cookbook/editor-support.ht... in addition of Emacs, Vim, Atom/Pulsar (very decent support), Jupyter notebooks, Sublime, Eclipse (simple support), Geany, enhanced terminal REPLs, Lem (multi-language editor built in CL)…
I wrote that support mostly over a weekend, and judging by the lack of bug reports, either my code is flawless (doubtful) or nobody uses it.
Don't get me wrong, it's a great start, and I hope it gets polished. But as of 3 months ago, it still doesn't work well, so much so I'd never actually recommend it seriously to someone who wants to learn Lisp.
Have you tried using VS Code (or any other editor you mentioned) for any "serious" full-cycle Lisp development before, and do you believe it passes the minimum bar of stable and recommendable to another professional programmer or enthusiastic hobbyist? Do I need to try again?
Lem is remarkably featureful and usable for CL development though! Both its ncurses and SDL2 frontends. I only miss a couple keys from Slime: C-c C-y (print the function the cursor is on to the REPL), C-c ~ (sync the REPL with the current package (namespace)), then a better imenu, and that's it. Simple git support is available for power users too 8-)
edit never mind ... it's bound when invoked from a .lisp source file ... very cool! I think I will use this a lot! Thank you for sharing it ...
What I want — what I really really want — is an Allegro CL experience. Unfortunately, nothing provides it, nor does it look like anyone has any interest in providing such an experience.
This isn’t just a CL problem. Janky debugger integration is widespread. It’s the pathetic that only commercial developers provide a polished debugging experience. Everyone else is satisfied with a basically a CLI tool running separately in a window. That’s like. 80% solution. We can do better. We should do better.
Eclipse can also produce reasonable debugging for Java at least.
But yes other than that poor debuggers do limit the choice of languages.
Unfortunately PHP is better served than most interesting languages.
Eclipe was born as a rewrite from Visual Age for Smalltalk into Java.
See the difference?
(I just corrected mine, which was in this same comment. Corrected info below.)
I had tried out Allegro Common Lisp (which is from Franz, Inc.) and LispWorks Common Lisp, some years ago.
Maybe sometime after Peter Seibel's book Practical Common Lisp came out.
I had also tried out GNU Lisp at some point.
It’s been like 20 years ago. :)
The Lisp development product of Franz Inc. is Allegro CL. https://franz.com/products/allegro-common-lisp/
The Lisp development product of LispWorks Ltd. is LispWorks. https://www.lispworks.com/products/lispworks.html
Both are kind of expensive. Both offer also academic licenses.
"Franz Lisp" does not exist as a product since the 80s.
Both Allegro CL and LispWorks are sold since the mid/end 80s and are still being maintained and sold.
(ps: see slime-breakpoints for a nice graphical addition: https://github.com/mmontone/slime-breakpoints)
Saying that you just have to watch a bunch of tutorials betrays just how janky it is.
For example, in CL debugging, how would I reliably set a breakpoint at some nested expression in a function body, run to it, and step through the code "expr-by-expr" while verifying a bunch of values in a watch window? In VS, it's one click to set breatpoint, one keypress or two clicks to run to it, one click to step to next line, and one double click and typing an expression to set up a watch that would show updated value of that expression every time execution pauses while taking care of scoping, etc.
This is a common complaint about CL, but my experience is that the utility of debuggers is vastly over-stated. It’s a nice trick in Java or C/C++ or JS (in Chrome), but I've never found a debugger to significantly improve my workflow and the “repl-driven development” paradigm of CL solves the problem differently by increasing iteration speed.
Having to modify code to add/remove breakpoints is pretty much the definition of jankiness. Can Emacs+SBCL show a list of all breakpoints, for example?
> This is a common complaint about CL, but my experience is that the utility of debuggers is vastly over-stated.
I disagree. A debugger gives you insight into the state of the program. It makes no sense to burn neural cycles trying to track program's state manually when a debugger can visualise it in multiple views and forms. Frankly, during development, I always run my code in a debugger, just to be able to confirm my assumptions on a granular level whenever I want to.
> the “repl-driven development” paradigm of CL
In my experience, trying to substitute a proper debugger with REPL leads to unnecessarily small functions, which then lead to very deep call stacks and undocumented assumptions about pre- and post-conditions of all those tiny functions. However, CL is specially bad about this anyway, since every `let` binding adds a new nesting, making long linear functions unreadable (either due to deep nesting, or due to predeclaring all variable as in ANSI C). Scheme with its `define` is much better in this regard.
Sly's UI seems to be similar to that of GDB, which is exactly what I am talking about. I would suggest trying out and comparing GDB and one of its numerous GUI frontends to understand why a GUI with multiple views displayed simultaneously (i.e., you can keep track of different parts of state on different parts of screen) and automatically (i.e., display doesn't require interaction) improves debugging experience.
As regards windows and gui, I also dont understand. I can certainly have multiple windows displayed in Emacs, even in a tty terminal. What buffer each of those windows serves is up to me. Certainly I can have each state sent to a different buffer. One of the things I personally value about Emacs is that I don't have to run it in some Windowing gui system to get any of the supposed benefits. Heck I can even use a mouse to adjust window sizes in just my tty. I dont get what GUI gives you except a higher overhead
https://learn.microsoft.com/en-us/visualstudio/debugger/debu...
There is no feature here that is not available for debugging Common Lisp in Emacs, which has much more powerful debugging tools, and with less overhead
For many Common Lisp implementations you don't need that. The debugger & source interpreter & compiler & break loops are always available. One also does not need to "instrument" code for a debugger.
> In my experience, trying to substitute a proper debugger with REPL leads to unnecessarily small functions
That makes no sense. The REPL includes a debugger and does not enforce a function size. You can call&debug functions of any size from a REPL. Stack traces can be detailed or overview. From within the REPL, which in case of break is a break-loop, which is a REPL with added debug commands.
> undocumented assumptions about pre- and post-conditions of all those tiny functions
The idea of tiny functions is made up by you. Common Lisp's debugging tools don't enforce tiny functions. Just the opposite, many CL code bases have large functions, which are also even larger after macro expansion.
pre- and post conditions can be explicit parts of Common Lisp code, since it has :before, :around and :after methods.
> every `let` binding adds a new nesting
Lisp uses nested lists. That's one thing to learn from day one when programming in Lisp. Nested code shows indented blocks.
> or due to predeclaring all variable as in ANSI C)
Not predeclaring variables is bad practice anyway.
> Scheme with its `define` is much better in this regard.
No, Scheme's "define" has to be at the top of a block, according to Scheme standards.
It is possible to patch a running CL system by simply setting a breakpoint and then revaluating a function.
I am unaware of any other language that is possible in. This Lisp feature is only made possible through a debugger.
I know, because I've done it.
It's also possible to change a running CL system without setting a breakpoint and/or re-evaluating a function.
> I am unaware of any other language that is possible in.
Start with Smalltalk and then search for more.
The function FOO has a BREAK.
CL-USER 76 > (defun foo (a)
(+ (break) (sin a)))
FOO
CL-USER 77 > (foo 10)
Break.
1 (continue) Return from break.
2 (abort) Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
CL-USER 78 : 1 > :bq
INVOKE-DEBUGGER <- BREAK <- FOO <- EVAL <- CAPI::CAPI-TOP-LEVEL-FUNCTION <- CAPI::INTERACTIVE-PANE-TOP-LOOP <- MP::PROCESS-SG-FUNCTION
CL-USER 79 : 1 > :n
Call to INVOKE-DEBUGGER
CL-USER 80 : 1 > :n
Call to BREAK
CL-USER 81 : 1 > :n
Interpreted call to FOO
CL-USER 82 : 1 > :lambda
(LAMBDA (A) (DECLARE (SYSTEM::SOURCE-LEVEL #<EQ Hash Table{0} 8220104A8B>)) (DECLARE (LAMBDA-NAME FOO)) (+ (BREAK) (SIN A)))
CL-USER 83 : 1 > (setf (third (fifth *)) '(* a 4))
(* A 4)
CL-USER 84 : 1 > :lambda
(LAMBDA (A) (DECLARE (SYSTEM::SOURCE-LEVEL #<EQ Hash Table{0} 8220104A8B>)) (DECLARE (LAMBDA-NAME FOO)) (+ (BREAK) (* A 4)))
CL-USER 85 : 1 > :p
Call to BREAK
CL-USER 86 : 1 > :ret 2
42
> For example, in CL debugging, how would I reliably set a breakpoint at some nested expression in a function body, run to it, and step through the code "expr-by-expr" while verifying a bunch of values in a watch window?
I ... don't ever do this when debugging lisp? Sly has support for setting breakpoints, but I'm used to adding (break) expressions; the "breakdot on the side" doesn't work as well with Lisp as C (With idiomatic C you very rarely want to set a breakpoint in the middle of a line, and it's rare for a single statement to be more than 2-ish lines; I should also note that IIRC setting breakpoints in the middle of a line in VS was not the best UX).
Sly has a keyboard shortcut for stepping, but I don't have it memorized because I hardly ever use it. Conversely its F10/F11 in VS, which I still remember despite not having used Visual Studio for at least a decade, since I did it a lot!
Conversely, I have the shortcuts for recompile the single file I'm in and recompile the single function I'm in memorized, since I use those all the time when debugging. I believe VS added hot-patching functions at some point near the end of my usage, but it wasn't part of my workflow like it is with Lisp.
(The lack of) watch windows is a deficiency in Sly. While you can view all locals in the current stack-frame while debugging, it doesn't let you watch arbitrary expressions, and if you have too many locals it can be hard to focus on what you want.
I should point out stickers (C-c C-s C-s) let you trace evaluations of arbitrary subexpressions, and this has some overlap with how watchpoints are used in VS.
You can't watch expressions, but when the debugger comes up, and I think if you're stepping through too, you can hit "e" for eval and do any expression you basically want. Any variables used would need to be in scope, etc ...
(this is with sbcl and slime ... I'd bet sly supports similar)
Just look at the screenshots. What's missing? Literally everything is missing, because it's just a CLI in a window. Want to add a breakpoint? Type in the window. Want to know how many threads are running? Type in the window. Want to pause a thread? Type in the window. Want to inspect a variable? Type in the window. Want to drill down through a complex data structure? Type that 50 character line correct in one go bitch. By god! Don't you dare complain!
The irony of course is that all of these commercial products are using the exact same debugging infrastructure of the OSS counterparts. It's just that one group finished the job, and the other doesn't bother to finish the job.
The sly debugger is better than most debuggers, but calling it the most powerful debugging experience is a stretch. Just try the debuger in a Smalltalk implementation (Squeak/Pharo/Cuis)
Tucked away in the McCLIM project is Clouseau, which you can quickload and use as a normal user: https://codeberg.org/McCLIM/McCLIM/src/branch/master/Apps/Cl... One small cool thing it does is if you inspect a complex number it will also draw a little x-y graph with its vector representation. (Though trying it out again just now it's overlapping with the text... maybe I should file a bug, but I've only now just learned they moved off github, and I'm not going to make a codeberg account. Friction wins this round.) It does take a while to first compile and load all the dependencies, especially 3bz, another weakness of at least our free Lisps; AFAIK there's still no equivalent of make -j for compiling systems.
I'm a happy vim user (though there is some jank with slimv, admittedly, but it's mostly prevalent around multiple thread situations) and setup the command ,ci to call my own clouseau-inspect function; it just inspects a symbol with clouseau instead of slimv's inspector. Also have a janky watch/unwatch pair of functions that just refreshes the inspector every second. (https://github.com/Jach/dots/blob/master/.sbclrc#L113 if curious, some other junk in .swank.lisp and .vimrc too, and there's https://github.com/dense-analysis/ale/issues/4061 to call sblint on your project...)
But more graphical tools like this are what I hope to one day see more of and are how the free Lisps can close the gap in this area with the commercial Lisps. I believe there's not much Allegro can do that poking around SBCL can't do, but for many things it's just nicer to have a GUI. Want to explore all the symbols and values in a package? Easy enough to script that, but not as nice as just having a table of symbols, and even nicer if you can set watches on some of them, or even set conditional breakpoints on value changes. None of the tools need to be tightly integrated with a single IDE either, because all the stuff necessary to debug Lisp is in the running Lisp itself. It's just that the GUI situation continues to suck.
LSP has gotten more popular with other languages and editors, sometimes I wonder if the acronym was made as an inside joke because it's basically how Lisp + Slime/Swank have worked...
I have been intending on trying the latest free IDE from Franz. Two days ago I was in a fun catch-up video call with Jans Aasman from Franz, but we were talking about AI more generally, not Common Lisp specifically.
I joke that the parallel GC is bringing SBCL to 2000s Java. It seems comparable enough to Java's Parallel, though SBCL aims for less space overhead by default, so you need to adjust either GC to have a fair comparison. The concurrent one is based off a collector for SML#, Haskell has a similar low latency "Alligator" collector too.
(Disclaimer: I wrote the parallel collector, and I want to be clear that latency and throughput differ here, because someone didn't before.)
1. It is optimized for throughput over pause-times
2. It is non-parallel
Kudos to things like the cookbook tho !
The documentation culture is the one thing I miss from Perl when I go to... well, any other language. Considering CL has DESCRIBE it's really something we need to get serious about.
For a lot of the commonly used libraries, the doc is sufficiently decent, though sometimes unintuitive to find. For core stuff I like the hyperspec as much as the next lisper (and opening it on a symbol is just ,h for me) but there's also http://clqr.boundp.org/ that not everyone knows about, it's a great little reference to have printed out at my desk. And there's situations like how part of pretty much every Lisp, available through ASDF, is UIOP, a great little portability library. Its doc is at https://asdf.common-lisp.dev/uiop.html and it originates in the source repo from a Lisp script taking doc info from the source and applying a .texinfo template to it. Pretty bespoke. But this is also just part of the cost/benefit of CL: it's very unopinionated, unlike say Java or Clojure, and so lets individuals and teams do what they want which naturally isn't going to be the same worldwide. I like certain doc (especially at least one example of using the thing) in repo README files, others don't, nothing forces one way or the other.
Once the confidence for Lisp is there though, you're right that it's not a big deal. It's very nice to jump-to-source for everything, including SBCL internals, and modify things even. I can fix some bugs in a dependency library, right now, without having to file an issue/pull request to the library maintainer and waiting for a new release, and that modification can even live in my source tree in the interim as just a bit of code that runs after loading the main library and then redefines one of its functions or whatever.
I disagree that any effort needs to be made on attracting lone wolves, less even than trying to make editor support better for beginners; the loners will come on their own in time. Not because CL can only be used by loners (million+ lines of code projects made by teams demonstrate otherwise), but because it's a very good multiplier for that situation.
Some things that I have in mind that would take alot of effort but WAAY less than making another editor competitive with Emacs+SLIME/SLY:
* Rewrite the spec in org-mode and use babel for interactive learning
* Integrate the above with the so called de facto standard libraries
* Make excellent documentation for SBCL and ECL implementations (take CCL as an example)I didn't let schooling interfere with my education.
That is an amazing system, that even now, could probably hang with the best of them. :/
[0] - https://en.wikipedia.org/wiki/ML_(programming_language)
Wait what?! When was the last time programmers chose a language because it was "green"? What does it even mean for a language to be "greener" than the others?
Yeah, in that case Assembly/C is the greenest language but should we switch to them?
> The inefficiencies from extra human time would likely offset any potential gains from the pennies you'd save on CPU costs.
I'm not sure I follow what you mean here.
We've not switched away.
The point isn't about the absolute numbers, it's just trying to illustrate that the numbers are so small that it might as well be a rounding error unless you're talking about industry or society-levels of impact, which Clojure does not really have.
That's just wrong. Electricity used by data centers today add up to the whole consumption of a largish country.
If you can save even just 5% of that, it's an enormous difference. Companies like Google and Netflix spend millions to save much less than that in computing.
But no, that's not how it works at all. Nearly half (about 40%)[^1] of datacenter's total power draw on average is from cooling, which is usually shared across every server in the datacenter. Next, about 30%[^2] of the average server's power draw comes from its CPU (this obviously varies, even at different times on the same server), most servers sit at about 50% utilization on average. So already we're looking at 50% of 30% of 60% = 9% total of energy consumption going to the CPU with very rough, over-estimations.
Now Clojure makes up about 1.25% of code, ish[^3]. Let's round that up to 1.5% to be generous. So a reeeeally rough estimate has Clojure code taking up 0.135% of total datacenter power usage globally assuming averages for pretty much everything. I had trouble finding benchmarks for Clojure vs. Common Lisp, but Clojure is basically just Java at runtime, so this[^3] will do. Benchmarks are kinda silly, but we're talking about really average performance. The benchmarks I could find had Java beating SBCL on most benchmarks, but let's say you stand to gain a generous 20% efficiency by using SBCL over Java. That leaves you at a 0.027% gain in efficiency with a lot of generous rounding and assumptions.
Now, of course most code is not CPU constrained... you get the idea. The efficiency gains are not much at all.
[^1]: https://davidmytton.blog/how-much-energy-do-data-centers-use...
[^2]: https://www.researchgate.net/publication/355862079_A_Review_...
[^3]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
The point is that if everyone used more efficient languages, a very large reduction in energy costs could be expected (not just in data centers by the way, every device takes power, and the ridiculously inefficient apps people are running probably cost a lot of energy). It's the same argument as to why you should have a ecofriendly house or whatever: you try to do your part and hope the others will do too. You may be cynic enough to think that makes no difference, but you're just blind to your own unjustified prejudices.
Because that's what I was talking about in the first place? I was responding directly to the text under the linked video making an ecological appeal to using SBCL over Clojure. Both are very efficient languages, so that particular appeal seems silly.
> The point is that if everyone used more efficient languages, a very large reduction in energy costs could be expected (not just in data centers by the way, every device takes power, and the ridiculously inefficient apps people are running probably cost a lot of energy).
Sure, I agree with that. But that's not what was being talked about. If you have data or sources you can share that show studies that model how much energy could be saved, I'd love to see them.
> It's the same argument as to why you should have a ecofriendly house or whatever: you try to do your part and hope the others will do too.
So don't write load bearing applications in Python. But most code is not load bearing or even executed that often. The energy usage of SBCL vs Clojure is like making the decision to unplug your toaster when you're not using it, and it's not even clear which is more efficient.
> You may be cynic enough to think that makes no difference, but you're just blind to your own unjustified prejudices.
I looked at the data and came to a conclusion. I was talking about choosing one pretty efficient language over another, you were talking about something else.
> Everything you said is a waste of your time.
You have no right to tell me what is a waste of my time. I found it quite interesting, I just wish we could've had a conversation made in good faith instead of whatever this was.
If only it were that simple ! Clojure has no provision for compile time type checks, unlike Common Lisp running in SBCL. On the other hand Common Lisp running on JVM (see Armed Bear Common Lisp) could theoretically achieve those results because, but if we are including calls to host systems then Embedable CL could better it since it runs on top of C. The optimized Java in the benchmarks you quote in fact looks more like C or Cpp code. Im not informed enough to make a judgement if idiomatic Java even looks like that. Idiomatic SBCL (as opposed to just idiomatic Common Lisp) code does look like the versions achieving best results, because people often use SBCL for compiler optimization
> The benchmarks I could find had Java beating SBCL on most benchmarks,
Most of the benchmarks are actually quite similar - ie within the margin of error. Most of the times when Java did beat SBCL by a somewhat more significant margin I noticed that Java's submission used multi-threading whereas that particular submission for SBCL didn't, so the difference isn't so much as Java vs SBCL, but algorithm a vs algorithm b.
Also, as far as I can remember, it is worth mentioning that there was some controversy over some SBCL submissions. One of the submitters had solutions which saw SBCL beating C, Cpp, Zig, and Rust on some of the numerically intensive benchmarks, however their submissions were rejected later for using a SIMD library which at the time was not yet part of SBCL, but now it is.
https://www.reddit.com/r/Common_Lisp/comments/riedio/quite_a...
(That comparison no longer seems to show Lisp?)
Unoptmized Java looks pretty similar to C/C++, so optimized Java looking similar doesn't surprise me.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
idiomatic SBCL ?
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
> Idiomatic SBCL (as opposed to just idiomatic Common Lisp) code does look like the versions achieving best results, because people often use SBCL for compiler optimization
I guess we are interested in real life application performance, not in low-level benchmarks or benchmark games.
Case 1a: A DC is using 60kW for equipment and 40kW for cooling
Case 1b: Code efficiency of 5% improves the equipment to 57kW for equipment;
I'd expect cooling to drop to around 38kW rather than remain at 40kW to remove that 57kW of heat load.
You at least need to factor in the percentage of an individual server's power draw that goes to the CPU, which is about 30% (again, on average).
So a 5% equipment power draw improvement would need at least a 15% code efficiency improvement (0.05 / 0.3 = 0.1666), assuming you're CPU bound on whatever your code is doing. I factored in the average CPU load (50%) to account for this, but it's heavily application dependent.
I agree that it's not 1:1 from CPU to equipment.
A human who spends his day writing code is going to consume (at worst) the exact same amount of food as if he spent only half his day coding and the other half on something else. He might even consume less food while coding, if his non-coding activity is something physically intensive (i.e. burns more calories). I don't really see how the implication (that saving human time saves energy too) can possibly be true.
It means more energy efficient.
May be we should, at least for some things. For code that runs a very large number of times the energy savings will add up. Data centres do use a lot of energy. So do client machines in aggregate.
As someone who is neither a Common Lisp nor a JVM programmer, I’m genuinely curious to hear from experts whether the old trope of “JVM is slow and bloated” is even remotely true any more. Based on hearsay, the JVM often competes in language performance closer to C++, and has probably the best garbage collectors in the world as well as the best developer and operational tooling. How much do other platform developers leave on the table by ignoring the JVM these days?
As an aside, even the trope about us running out of energy, or energy usage directly corresponding to planetary destruction is becoming less and less true. With solar PVs and the sun, we truly do have a shot at a future with limitless energy and resources.
I am mostly a Java developer, but have had the opportunity to play with Common Lisp and many other languages.
While it's true the JVM is very performant, its performance actually comes at a cost: it has a JIT, Garbage Collector, a whole lot of machinery for Threads/VirtualThreads, JMX etc. all of which comes at a cost... thanks to multicore processors and lots and lots of RAM being available, this translates in really fast performance for Java applications... so that cost may be advantageous to you.
Common Lisp applications are much lighter than Java applications from what I've actually measured given its simpler model... it basically compiles Lisp code to machine code directly and then runs that (Java runs bytecode, and then at runtime the JIT selects which code to optimise, which continues to happen for the lifetime of the application). I would bet money on any application written in Common Lisp being significantly lighter, and hence "greener", than the equivalent Java application, while still running at nearly the same speed, or even faster.
For short running programs, Common Lisp is incredibly fast at starting up and getting stuff done - it competes with C and Rust in that regard, not with Java.
> How much do other platform developers leave on the table by ignoring the JVM these days?
Common Lisp tooling is integrated into the language itself. You can inspect the machine code it generates, ask it to instrument every function so you can profile, and so on without even needing any external tools - it's all built-in... but of course, using it from SLIME or SLY (in emacs) makes it a lot more pleasant. If you think Java tools are better, I believe you may not know Common Lisp very well.
Here you say “it comes at a cost” twice, but you don’t say what the cost is! Or am I missing something?
You can actually tell the JVM to not JIT... try that and see just how slow your code runs... it may give you an idea of how much it costs in terms of energy as well if you're able to measure that (I don't know how much that cost would look like, to be honest, I can only imagine it's a significant proportion of the total).
That is the beauty of languages with multiple implementations.
Also lets not forget the influence of Lisp/Scheme people like Guy Steele on how Java turned out.
"We were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp."
Full Lisp would have been better, but apparently Lisps in desguise like Dylan and Julia, are the best mainstream is willing to take on.
I don't think it's even close. Benchmark game is one (biased) reference. And it's even worse because optimized Java is not canonical Java. Canonical C++ is pretty fast.
I agree.
1. Benchmarks tend to do one thing, then repeat that one thing 25000 times. Non-server programs don't do that[1] so you don't get to amortise the time spent on JITting a code snippet across 25000 executions.
2. The runtime basically requires exclusive use of a core for short periods (GC, JIT). If you're running on a system that costs per core, you're using (worst case scenario) an extra core for each Java program. The benchmarks happily give multiple cores even to sequential (non-parallel) benchmarks.
[1] Even many server programs don't do that.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
2. The benchmarks game happily gives multiple cores and shows that "cpu secs" measurement:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
2) "one (biased) reference" is simply name calling.
3) What specifically are you saying is "biased" about these measurements?
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
The GUI situation of the free lisps is pretty much a non-started for anything portable, just curious if there's a project bundling Common Lisp with a browser window, like Rust Taurus.
At this level, there's no reason to fret about download sizes anymore if you're comparing to something like Electron, so no real need to "shake the tree" or really limit the lisp image size. Modern lisps aren't that big compared to most anything else anyway.
Do you have some examples and what language features are terrible?
"Write tests!" and "use spec" were not great suggestions at the end of the day.
Interestingly the clojure community at large does not really value or use it so much. I think even Rich Hickey commented on that at some point.
I'm not sure if the recursive hierarchical conceptual inference as well as many other things would have been possible without Common Lisp.
There is also FOIL, which runs JVM (or CLR) in a separate process and provides a bridge for calling into JVM from lisp. It was written by Rich Hickey (the future author of Clojure), who observed that the reflection overhead was similar in magnitude to the serialization overhead!
My eyes just rolled so far back in my head I can see my skull.
> "Aimed to counter Microsoft’s .NET framework"
First of all the alliace came to be during the 1996 - 2001 timeframe that predated .NET.
When Network Computer came out there was no .NET.
When Oracle adopted Java into their database ecosystem there was no .NET.
When IBM refocused from Smalltalk into Java, there was no .NET.
Secondly, .NET was only born with C#, after Sun's lawsuit due to J++ language extensions (J/Direct reborn as P/Invoke, COM interop, WFC reborn as Windows Forms, events).
Had it not been for the lawsuit, the updated COM Runtime being researched would have most likely used J++, as shown on the Ext-VOS paper.
Also 20 years later, Microsfot saw they had to embrace Java on Azure, and they are now an OpenJDK contributor with their own distribution, talk about strange ironies.
> I think he simply meant that Java was marketed and therefore prospered as a counter to Microsoft's dominance more generally.
i.e. ".NET" being used casually (and incorrectly) as a catch-all for anything Microsoft was doing in this space roughly around that period ...possibly suggests there's been an anti-M$ bias!
Windows hegemony should have been used instead.
:chefs-kiss:
Even then, the ecosystem is a Faustian bargain.
The bloat of the JVM, and the implication of Java down the road, made me nauseous.
What languages do you work with that would make Java so painful to stoop to?
The sexp-based collection literals are very convenient, and coupled with the threading macros make writing and reading clojure code very productive and fun.
Clojure has a lot going for it.
The biggest issue I have with it is the same issue I have with any 'hosted' language. If it doesn't own the runtime, it risks being broken or held-back by changes it has no control over. Clojure is conceptually and syntactically simple, so it probably fares better than other JVM-hosted languages, and the JVM itself has historically been very stable at the bytecode level, but it still a risky thing to require your platform to stay stable for your language ecosystem to survive.
I love you Clojure, but I'd rather invest in a language that controls its own destiny.
Aside: this is also an argument, for example, for even using python and rust (one of the most depressing things that led me to quitting Google last year was a manager that demanded C++ for performance reasons over Rust despite being off-g3 and it being an utter nightmare to bikeshed about C++ libraries) when someone tells you that you “need” to use C++ for performance.
Apparently, you can compile SBCL using ECL, so if you really want SBCL itself instead of just a Common Lisp implementation, that might be a path to it.
Slightly earlier he did discuss the general alliance against Microsoft. I thought it was funny he didn’t mention Netscape as part of the alliance. At the time they were considered far more important than IBM or Oracle for purposes of displacing Microsoft on the consumer side.
It’s fine in the grand scheme of things, a younger person who was not there trying to reconstruct what happened and slipping up a bit. But for someone who cares about the details of this sort of history I didn’t want to absorb any similar errors about Common Lisp which I’m genuinely curious to learn about.
A couple of C# features were born in J++, namely P/Invoke (J/Direct), events (influenced by Delphi before even Anders came to MS), and Windows Forms (WFC).