Why I Chose Common Lisp
blog.djhaskin.com
blog.djhaskin.com
A while ago, I did some private work for someone, using SBCL, and sent her the native binary I built, then forgot about the whole thing.
The client came back with some new requirements much later, when the project was concluded, and the source code was already lost by that time.
I vaguely remembered how I did things, so I spawned a REPL from the old binary, went into the relevant CL package, wrote a new function that overrides the old behavior, and the client's problem was solved.
I did all those without the original source code. It was a small thing to fix, but I can't imagine making the same fix so swiftly with any other tech stack, when the source code is lost. I was deeply impressed.
Get backups
I will also keep the source I write and, if I feel like it, take a look at it years later and no one will know except me. BFD!
A Lisp image would then know/track which versions of what code it has loaded/compiled/... Version information would be stored with the other source files.
Could you share some examples of how the old systems did this?
Files are versioned. The Lisp machine file system has versions (same for DEC VMS and a few other file systems). Nowadays the file version is attached to the name&type. Thus editing a file and saving it creates a new file with the same name&type and an updated version number. I can edit/load/compile/open/... files by specifying its version number. If there is no version number specified, the newest file version is used.
A site shared a namespace server with system registry and several file servers.
Code, documentation, etc. is defined as a system, similar to what ASDF does. A system has major and minor version numbers and other attributes (like being experimental or being released). Each time one does a full compile on a system, its major version gets updated and it tracks which file versions it used for that compilation. Whenever one creates a patch (a change of the sources) to a system, the minor version gets updated. After 12 full compiles and 30 patches to the latest version we have system FOOBAR 12.30. This data is recorded in the system directory, thus outside of an image (called a World in Genera) in the file system.
Now I can use that system FOOBAR in another Lisp and "Load System FOOBAR :version Newest" -> This will load system FOOBAR 12.30 into its runtime. Then new patches may appear in the system. I can then in a Lisp say "Load Patches FOOBAR" and it will look into the system directory for new patches to the current loaded major version and load those patches one by one. This changes the running Lisp. It then knows that it has loaded, say, FOOBAR 12.45, what files it has loaded and in which files the respective code (functions, ...) are located.
If I want to ship a system to the customer or someone else, I might tag it as released and create a distribution (with sources, documentation or not). The distribution is a file (also possibly a tape, a bunch of floppy disks, a CDROM, or similar). A distribution can contain one or more systems and its files (in several versions). On another Lisp I can later restore the distribution or parts of it. The restored systems then can be loaded for a specific version. For example I can load a released system version and then load patches for it.
A saved image knows which versions of what systems it includes. It also knows which functions sources are where. It may also have corresponding documentation versions loaded.
Later a source versioning system (called VC) was added, but I haven't used that.
Make sense: no, unless you have very special requirements. It would be very different from common practice. Images are still much more fragile than files outside, they are not designed to last across implementation releases.
(Note: some implementations might hold full code of every function in memory, that's not unreasonable.)
Also the Mesa / Cedar had a similar approach, but based on dynamic libraries, which Niklaus Wirth took great advantage on how Oberon and derived systems worked.
Incidentally, due to Rob Pikes admiration for Oberon, not only had Inferno the ACME editor from Plan 9, already influenced by how Oberon worked, the whole Limbo experience on Inferno is quite similar as well.
Unfortunately we still lack this kind of full stack OS/REPL experience on modern systems.
By all rights, it should have been _awful_. But in reality we were able to iterate quickly and actually make stuff, because the technical foundations of the system were rock solid, the team was good, and because the truly integrated Smalltalk development enviroment enables a super-tight development cycle. A joke I have with a former colleague from those days is how all modern languages are still fundamentally no different from the punch card days: You take your sources off to the batch processing center, and the results come out at the other end, while Smalltalk is future techonology from the 80s.
We do have Pharo, Allegro and such, but their spotlight opportunity is gone.
Do you think the paradigm is a dead end?
It's pretty cool, and also an obviously terrible idea as you leave behind all standard tooling
The process you describe is possible in JavaScript and many other languages as well. Updating a JAR is trivial, but doing it in a running system is uncomfortable.
The first thing to start with is versioning at the function, class, package, or module level. Combine that with versioning of data, and you've got my interest :)
Another huge benefit of vim and vim-slime is it is immediately valuable when you use/learn any new language. So long as the language has a REPL/console/interpreter that can be opened from the terminal or terminal emulator in any form (e.g. CL, ruby, python, bash etc etc etc) then vim + vim-slime will be a brilliant ~IDE. (Possibly the only thing I haven't been able to do but wanted to is 'send' code from neovim to the javascript console in chrome, which would be pretty awesome!)
A side note: I found doom-emacs very similar to vim, only needed ~10 or so new keyboard shortcuts to be productive in emacs. (I still much prefer vim, but I'm not so down on emacs).
slime has some issues for me (obviously not OP) and I am not convinced lisp and vim are a good pair. lem is getting pretty good and improving by the day, find it much better to work with than vim when it comes to lisp and vim is my primary editor.
I am sure I have left some productivity on the table not investing in workflows like cider etc but I have gotten a decent workflow using just vanilla tmux, a repl pane and vim-surround
The % matcher in vim does so much heavy lifting, I've never felt limited by a lack of slurp and barf
I actually wrote my own tiny plugin to send snippets to the repl using nc and I'm still happy enough tearing the clojure repl up and down and copying stuff in by hand because dealing with repl state can be pain. Even though I have at times had repls open for months, there is a freedom in just tearing it all down and up again.
Clojure itself has plenty of functions to load in files or snippets to help as well
Is there a tutorial you followed or a video you found useful? What was your starting fund of knowledge?
Then 3. use it. This is shockingly easy, open two panes in your terminal with neovim on one side and REPL/interpreter on the other. For example I have neovim with my ruby file on the left pane and a rails console on the right (but on the right could be SBCL, python interpreter, or any other interpreter). In neovim, move the cursor to the line you want to run and press ctrl + c twice in quick succession. It will 'send' that line to the interpreter on the right pane and run that line!
Note: The first time you do this you may be asked which pane vim-slime should 'send' the code to, with the numbers displayed over the panes. For example in kitty I'm usually sending to pane 2, so I press: 2 enter enter. If it was pane 5, I'd press 5 enter enter etc.
If the line of code is immediately proceeded by another line(s) it will run that/those as well (for example, a multi-line Active Record query). It will do the same if there's one or more lines immediately above the current line. This takes a tiny bit of getting used to as you may unintentionally run lines immediately above/below the line for a short while.
That's all there is to it!
A few tips
- As explained above, ctrl + c ctrl + c will run the line under the cursor. But you can also select and run any code you want by selecting it with vim's visual mode and ctrl + c ctrl + c to run that selected code. For example, if you want to run part of a line, select it in visual mode and ctrl + c ctrl + c and it will run! Same for say a few hundred lines of code: select it all in visual mode (e.g. v ctrl + f ctrl + f then j or k to get to the exact line), then ctrl + c ctrl + c will run everything you selected.
- Rails specific: The rails console has a pager set to 'on' by default (this would necessitate back and forth between panes in order to press 'q' to quit out of the pager). So I turn it off by adding one line (IRB.conf[:USE_PAGER] = false) to ~/.vimrc or just .vimrc in the project directory.
Let me know if you have any questions/troubles.
This is interesting -- I've worked with people who swear by common lisp, emacs, and SLIME.
I'm happiest with ruby and vim, but I have not tried vim-slime (nor even heard of it before, so thank you!).
But FWIW, my strategy for running larger bits of ad hoc code on the ruby/rails console is to:
1. Add the code to a persistent local file (e.g. "ops_console_tools.rb")
2. scp the file up to the target machine where I am running the irb/pry console
3. In the console, run `load '/PATH/TO/ops_console_tools.rb'`
4. Run the new code: `Ops::User::CustomReport.run(90.days.ago..)`
And it helps to include some code to clear constant definitions which would otherwise complain to STDERR if you update and reload the file multiple times.
None of this is as awesome as SLIME of course, but it's pretty tolerable with a bit of readline-style up-arrow and command-history conveniences.
Disclaimer: Of course, running ad hoc code in prod is frowned upon. But we're extolling the virtues of CL in this thread, so I'll confess to breaking best practices in environments where it's permissible! Also this process gives you syntax highlighting while editing without requiring config on target host, and you can include the file in version control for greater formality.
I wonder what it was about babashka that didn't work for the author. When I need a quick script with batteries included, I use babashka and it works great.
Running native-image on an already established, not-written-for-it codebase is a nightmare. I just tried again some months ago on the linked code bases. native-image wouldn't budge. Kept getting hung up on I don't even know what, the errors were way too opaque or the app just misbehaved in weird ways.
I still remember many years of reading comp.lang.lisp, where the #1 complaint of newcomers was that "common lisp cannot produce a native executable". I remember being somewhat amused by this, because apparently nobody expected the same thing from other languages, like, say, Python. But apparently things have changed over the years and now CL implementations can produce bundled executables while Clojure can't — how the tables have turned :-)
In the CL world it is usually relatively easy to create an executable. For example in SBCL I would just run SBCL and then save an image and say that it should be an executable. That's basically it. The resulting executable
* is already native compiled, since SBCL always compiles native -> all code is AOT compiled to machine code
* includes all code and data, thus there is nothing special to do, to change code for it -> the code runs without changes
* includes all the SBCL tools (compiler, repl, disassembler, code loader, ...) thus it can be used to develop with it -> the code can be still "dynamic" for further changes
* it starts fast
Thus I don't need a special VM or special tool to create an executable and/or AOT compiled code. It's built-in in SBCL.
The first drawback: the resulting executable is as large as the original SBCL was plus any additional code.
But for many use cases that's what we want: a fast starting Lisp, which includes everything precompiled.
Now it gets messy:
In the real world (TM) things might be more complicated:
* we want the executable to be smaller
* we want to get rid of debug information
* we need to include libraries written in other languages
* we want faster / more efficient execution at runtime
* we need to deliver the Lisp code&data as a shared library
* we need an executable with tuned garbage collector or without GC
* the delivery structure can be more complex (-> macOS application bundles for multiple architectures)
* we want to deliver for platforms which provide restrictions (-> iOS/Apple for example doesn't let us include a native code compiler in the executable, if we want to ship it via the Appstore)
* we want the code&data be delivered for an embedded application
That's in the CL world usually called delivery -> creating an delivered application that can be shipped to the customer (whoever that is).
This was (and is) typically where commercial CL implementations (nowadays Allegro CL and LispWorks) have extensive tooling for. A delivered LispWorks application may start at around 7MB size, depending on the platform. But there are also special capabilities of ECL (Embeddable Common Lisp). Additionally there were (and still are) specialized CL implementations, embedded in applications or which are used as a special purpose compiler. For example some of the PTC Creo CAD systems use their own CL implementation (based on a ancestor implementation of ECL), run several million lines of Lisp code and expose it to the user as an extension language.
But, if what you mean "executable" is "small compact executable like the one build by C/C++/Pascal, without extra Lisp runtime attached", perhaps you better look at something else, well like C.
It's always amusing to watch programmers arguing for superiority of their favorite language(s) over others, often bashing language features without clearly understanding their purpose.
And it is especially amusing to watch Lispers trying to argue with each other. "I chose Scheme", "I picked Clojure", "I moved to CL"... etc.
Bruh, I move from one Lisp to another based on my current needs. It's harder for me to choose new shoes than switching between Lisp dialects. Once you learn any Lisp to a sufficient level, the mental overhead between Lisps becomes almost negligible - it feels like practically operating the same language.
Sure, each Lisp is unique and they all have different flavors, but seriously, shouldn't we celebrate the diversity and be happy that we have at least one Lisp for every platform?
@jwr This isn't meant as criticism of your comment. I'm not arguing at all with what you wrote; Clojure is my favorite Lisp flavor as well. I'm just taking a sentence out of context and using it as a cue for my rant - don't be mad, we're cool.
Here's one for clojure.core/lazy-cat
https://suno.com/song/d012fa48-d5c1-46f4-8561-9a031cfb8925
Makes me sad that Clojure has mapcat and lazy-cat functions, but they've never made any effort to create mapdog and lazy-dog variants. I firmly believe that is the unique and only factor that has prevented Clojure from becoming a mainstream language at the top of the RedMonk chart.
I am a lifelong vim user (since elementary school in the early '90s), and I developed common lisp using vim for over a decade. I still use vim for nearly everything, but emacs as my Lisp IDE. Before evil-mode, I used the mouse and menus for 90% of what I did, and it was still an improvement over the best vim had to offer at the time (vim-slime existed back then, but would crash or hang regularly).
Author's vim setup is fairly good, but Emacs/slime is still better. They stopped using emacs because of RSI, but their vim setup defaults to "v a ( C-c C-c" to accomplish something that is "C-c C-c" in emacs/slime. They have altered it to be "v a ( <space> g" which begs the question of "why not remap keys in emacs?"
Would love to know more about the problems you faced.
In my experience whenever I faced such issues - it has been because I am not using it well.
For CLOS kind of things I have found https://github.com/camsaul/methodical library quite well and the performance is better than default multimethods in core clojure implementation.
it would be nice to know what exactly isn't working out and what the problems with native-image was.
Coz i think clojure is as close to perfect, imho, as a language can go without selling out.
https://github.com/clj-easy/graal-build-time
This initializes Clojure classes at build time and it mostly works for pure Clojure code.
Doing complicated things (e.g. depending on native library, etc.) requires some tweaking. For example, a few packages may need to be declared as initialized at build time or run time, depending what they are doing. And any unresolved classes need to be added to reflection-config.json.
All these are easily discoverable if one talks to people in the Clojurian slack channels. Clojure is a small community, so it helps to be part of it, because there are not a lot of open materials on the Web.
except...
> mostly works
> requires some tweaking
> discoverable if...
I know nothing about Clojure but from your caveats I think I can see why he spent hours banging his head against a wall.
Reminded me Prof. Knuth trolling in "The Art Of Computer Programming" with an exercise for the reader to prove Fermat's Last Theorem. (:
Problem solved.
Compare it to something like Go or Rust where there are no hoops and they're both well supported on Windows and Mac. I haven't actually used it but I believe Zig has very good cross platform support too.
I could be mistaken or recalling outdated information of course, but that is what I remember from the last time I looked into it
I grant that it might be described thus if I started out with that stack, but trying to retrofit an older code base with it is, I have found, next to impossible. You have to code around the myriad gotchas as you go or you're never going to identify all those landmines going back over it after the fact. The errors and bad behaviors are too difficult to identify, even for the `native-image` tooling.
If you don't do what everybody is doing to solve a problem, then of course it is not a "solved problem" for you.
No, you don't need to code specifically for native-image. What are the landmines that you need to code around? Since you have not successfully compiled native-image by following common practices, you obviously don't know.
I especially like its github readme and the FAQ there, provides a good amount of context about the project: https://github.com/janet-lang/janet
> If I had heard about Janet when starting this hunt, I might have stopped there and not gone on to CL. Nice syntax, small, fast executables, C FFI, a fun intro book. It checks all my boxes.
The feedback loop one gets with it is insanely fast, even faster than Python (certainly there are exceptions even with Python ...) That's a blessing and curse for me - the tighter a feedback loop, the harder for me to get out of a problem I am stuck on. :) But, so far, I felt like I can write a thing and not worry about running it ... type a thing, quickly get it running it in a running REPL loop. If a mistake happens, I can fix the issue right there, instead of just a long stack trace. For what it is worth though, I have been doing this in Emacs for a long time (well, for small functions), but didn't think much of it until now.
We are cursed to use the lowest common denominator of choices of programming.
If everyone in a code base wrote in a different programming language, the code base would suck.
However on a serious note, "the industry" fears even writing the entire codebase in a 'non common' language because of it, let alone your strawman idea of multiple languages.
It’s actually surprising that there’s no typing discussion in this thread.
Clearly there was another implicit requirement - maybe it had to be a niche language?
Even CL people are using Discord now? People really do seem to love to converge to a single place.
Also, comp.list at Usenet.
Because that's the usual convention, like comp.lang.c, .python, etc.
Further, the on-topic #commonlisp channel on Libera comes with a cozy off-topic channel named #lispcafe for general chit-chat about any imaginable topic. Webchat link for #lispcafe: https://web.libera.chat/#lispcafe
Except Discord servers cannot be described as "a single place" even, as they're all isolated from each other.
Instead of spread out across multiple open IRC networks/channels, developer communities converged into silo'd, closed/proprietary Discord servers. It's a shame.
A popular recommendation these days is Practical Common Lisp (by Peter Seibel): <https://gigamonkeys.com/book/>.
It hit the front page a while ago too: https://news.ycombinator.com/item?id=41401415
Also unless we're about Allegro or LispWorks with their own additions, the Java ecosystem tends to have more choice of mature libraries, I think ASDF isn't kind of spoiled by choice as Maven Central.
But to each their own.
As for Quicklisp, I may not be up to the latest and greatest, yet I doubt they are at the same level as Maven Central.
Just comments on the Scheme/Racket parts...
> I looked at Scheme, but that community seemed to still be fractured over the r6rs/r7rs mess.
Things were fractured before R6RS, with very little portable ecosystem code, and then R6RS didn't solve that, but AFAIK, people got back up on that horse, started over, and have been embracing R7RS.
> Also, there wasn't a big enough ecosystem to suit my liking.
There's a reasonably-sized ecosystem, but three things:
1. Unless things have changed recently, much of the good stuff is still in the package system for a particular implementation (like Racket, "https://pkgs.racket-lang.org/", or Chicken, "http://eggs.call-cc.org/5/").
2. It's nothing the size of Python or JavaScript.
3. Don't believe any claims of "batteries included"; you often have to roll your own basic packages for real-world work. (But this can actually be a blessing, even for startups that have to move fast, depending on what you have to do, and how capable you're willing to rise to be.)
> I'd already tried Racket in school and didn't like it. The runtime was a bit slow and bloated for my tastes.
Although CS professors are some the greatest friends of Scheme (having designed and made much of it), CS professors can also be the worst enemies for real world use of Scheme.
Before Racket, most people who had heard of Scheme knew it only from school, in problem set homework for the dense SICP course, or from whatever pet intro CS textbook their professor wrote. That's a good way to never want to try Scheme again. (And it got worse as many CS departments became optimized Leetcode->FAANG hiring funnels.) Then people never saw Scheme used, nor even described by, real-world programmers, for real-world things.
So Racket (PLT Scheme) comes along, and half of the handful of people thinking of Scheme as a language for real things gravitate to Racket, because they are doing some real-world things. And overall it's one of the best programming languages out there.
But still, although the Racket CS professors include some great programmers, Racket is determined by CS professors, who do PL and CSE research, and write and teach textbooks. So, Racket's use in intro CS classes seems to perpetuate the tradition of CS professors ensuring that students will never again want to touch Scheme after they get a passing grade for the class.
I find the "I tried it years ago and it wasn't as nice then, so I'm not gonna look at it again" attitude quote off-putting. It's the same reason Java still gets so much hate after over a decade since Java 8.
I agree that CL and Closure probably has more real world shipped products. However, if you are considering a newer and smaller ecosystem like Janet... you owe it to yourself to look at Racket with fresh eyes.
This is important give the history. R6rs was an attempt at an actually-usable scheme spec. R5rs isn't usable as described because there's way too much undefined stuff to actually get up and running in a semi-portable way. This problem exists BEFORE discussing the SRFIs which are quasi-standards except that every implementation supports a different subset and they aren't documented outside of the spec documents which themselves generally aren't fully specified.
Scheme the standard killed itself and Racket survives because it completely avoided the illusion of compatibility entirely.
My TXR Lisp comes with Vim support: a pair of syntax files and also a tags generator.
1. Standalone Executables: The biggest current obstacle right away! But I believe this (as in compilation to standalone, small executables) is coming with the next version (Julia 1.12) in an early form, so maybe stabilized and reliable within this year? There does seem to be a lot of momentum in this direction.
2. Vim Workflow: vim-slime works well to my knowledge, and the overall support (eg. treesitter, LSP, etc.) is pretty good, even if VS Code is the "main" supported editor.
3. Windows/Mac/Linux Support: mostly Tier 1 support [https://julialang.org/downloads/#supported_platforms]
4. Larger Imperative Ecosystem: FFI with both C and Python are pretty standard and commonly used.
5. Runtime Speed: Crazy fast as well
6. Multithreading: Base language support is already pretty good, and there's OhMyThreads.jl [1] and data chunking libraries and many other supporting libraries around multithreading.
7. Strong Community: I'd expect Julia and CL communities to be on the same order of magnitude? Complete assumption though, in both directions. Web presence is mostly on the Discourse [2] and Slack, and the JuliaCons are pretty well attended.
8. Ecosystem: Since package management is mentioned, I'll shout out the built-in Pkg package manager, the seamless virtual environment support, and the generally quite good versioning in the ecosystem, all of which add up to a really good experience. As for particular libraries, JSON is the only one I know the answer to: JSON3.jl is a solid, standard choice. I don't know if SQLite.jl [3] would be the recommended option for SQLite or something else, HTTP.jl does the job for HTTP requests but I believe isn't particularly fast or sophisticated, and I could believe there's a subcommunity within Julia that uses "functional data structures" but I wouldn't even know where to look. But, for the ex-Clojurian, may I present Transducers.jl [4] as worth a look?
[1] https://juliafolds2.github.io/OhMyThreads.jl/stable/ [2] https://discourse.julialang.org/ [3] https://github.com/JuliaDatabases/SQLite.jl [4] https://github.com/JuliaFolds2/Transducers.jl
Refutation: https://gist.github.com/digikar99/24decb414ddfa15a220b27f674...
Beyond that, wrapping C code from Julia is neither nicer nor worse than from CL. Wrapping C code is basically done everywhere except for a few outlier languages (Go comes to mind. It IS possible, but it means using cgo, which is its own world).
I liked Julia well enough, but the compile times were slow enough to be painful for me. All the best though to Julia :)
Agreed, I only included this because the author mentioned it explicitly as a requirement. From a "allows plugging into some other, large-community imperative language, like Clojure does with Java" perspective - in terms of library access - the combination of having well-polished interfaces to both C and Python is pretty powerful though.
> the compile times were slow enough to be painful for me
I think this, and the developer experience in general (eg. linting, IDE support, etc.), has been the biggest reason the excitement for Julia dampened over time, despite it being a wonderful language in theory. It's been getting better, from "painful" to just "somewhat annoying" for me, but not quickly enough to turn it around (IMHO).
> You can't make it a CLI script, because it compiles the whole code WITH dependencies every time
The "with dependencies" part is mostly untrue nowadays, with constantly better pre-compiled caches for dependencies with every release. The overall issue of compile times getting in the way of interactive development still has some truth to it, but much less than the comment implies.
> https://viralinstruction.com/posts/badjulia/
1. The subheadings in the ToC are mostly based on comparisons with the best-in-class: for eg. "Julia can't easily integrate into other languages [compared to C]", "Weak static analysis [compared to Rust]".
2. Seeing this actually gave me hope about Julia's progress, based on how many of these issues have been largely mitigated or have been actively worked on in the last three years since the post.
3. As a side note, the author of the post is still an active user of and contributor to Julia, so I think this kinda falls under the "There are only two kinds of languages: the ones people complain about and the ones nobody uses" banner. As in, the complaints are there because they like and actively use the language enough to want it to be the best in every way.
> Even though Julia 1.6 was a lot faster than 1.5, it still took too long.
I agree - I think pre-1.9 Julia experience sucked, and overselling the language in that period hurt its reception a lot. (I've mentioned elsewhere in the thread that the developer experience is still one of the weaker points of Julia.)
> (in CL a hello world weighs ±20MB):
In Julia 1.12, with the experimental --trim feature, a hello world is <1MB. Still too early to tell how that'll translate to real programs though.
> false ? 1 : 2 : 3 : 4
This is hilarious and awful at the same time. There's no beating CL in this - I've learnt that every language with syntax unfortunately develops "wat"s like this over time when well-intended syntax features interact.
> A few months ago, I tried to write a program that should receive UDP packets over IPv6 multicast.
> It didn't work. I never figured it out. This works in Java and Python.
> This might be unfair or untrue, but I get the feeling that it doesn't work because no one has seriously tried to use the language this way.
I don't think it's either of those: it seems like networking was and remains a weak area in Julia. For eg., though the language itself is blazingly fast, there have been a bunch of reports about how HTTP traffic performance is several time slower than ostensibly slower languages like Ruby. The reason is probably what the quote says too, there just isn't as much of a userbase or demand for this side of things.
> my packages seem to really like breaking when I try to load them about once a month
There's no source for this one, and no info on what "breaking" means or what the packages do, so I can only say this isn't a common experience. It's very easy to "pin" dependencies to preserve a reproducible state of dependencies and replicate it as well, which is greatly useful in a language used for science.
> I migrated from Lisp to Julia for the ecosystem. It hasn't been worth it from my point of view. I'll migrate back to Lisp eventually. [on a post] about lisp-stats
I'm not very surprised, given the lisp-stats context - it seems to be a common assumption/misconception, because Julia gets compared to Python and R often, that it's a data science and stats focused language. It's great for greenfield stats work, pleasant in many ways compared to those two, but the ecosystem is not particularly focused specifically on it. I'd suggest choosing Julia for the ecosystem if you're doing science science - quantum computing, diffeq modeling, numerical optimization, many others - but on the data science side, what Julia offers is consistent syntax, performance without C reliance, while retaining niceties like DataFrames and Tidier that other languages offer.
Agree on the rest, but multithreading in Julia is a let down.
The manual (https://docs.julialang.org/en/v1/manual/parallel-computing/) claims it's composable, but that's only true if you stay within Julia with the paradigm of Tasks. As soon as you interface with C/C++, you get a SegFault, as the Julia runtime expects to adopt [1] all threads calling into it. This is not always viable.
Julia should offer the option of C-style pthreads (or the Windows equivalent) and let others build abstractions on top of them.
[1] And that option was only added recently.
Unfortunately, this isn't possible when interfacing with proprietary, closed-source binaries. Especially when the C FFI is defined by standard, makes no mention of threading, and each implementation has different quirks.
The .net VM is similar to the JVM in perf, but iirc, the clj startup time is more dominated by Clojure itself (like var initialization).
I never heard of anyone else using it (though I'm sure somebody tried), so I'd bet against it having a better AOT experience. Just my $.02, anyway.
Probably the only reason why anyone would ever pick Common Lisp for a new project in 2025.
Maybe I’m crazy, but that is what I like from Lisp
Q: why I chose common lisp
A: I was looking for a lisp to begin with
This is almost like a satire. But I've found it rather funny. It presents question and provides an answer which is contradictory at first but still makes sense.
> A: I was looking for a lisp to begin with
> This is almost like a satire
How? It's no different than "Why I chose Arch Linux? I was looking for a Linux to begin with."
To even think that's satire is to completely miss the point.
Meanwhile, "why I chose Arch Linux" is more likely to be interpreted as "...over other types of Linux" because most HN people are already familiar with the decision of "which Linux distro will I use?" But if you gave that headline to someone who doesn't have much familiarity with Linux they'd probably expect the article to address why they chose it over Windows or Mac.
(I don't think "why I chose Common Lisp over other Lisp dialects" is an absurd premise for an article like the person you're replying to does, but I can at least kinda see where they're coming from)
I feel like there's been a boom in "editor hipsterism" in the last 10 - 15 years, while everyone has forgotten the variety of novel editors that were made in the 80s and 90s (I've forgotten them, too, I just remember seeing ads and reviews in magazines as a young programmer).
For context, I do have a basic understanding of vim because I run it on servers, but my knowledge doesn't go far beyond search and replace.
I mostly use Emacs because of org mode. It's way better than anything else trying to fill this hole. Otherwise I'd probably just use VSCode. But I don't want to add yet another editor to my regular use.
it helps that vim bindings are adopted in many places so learning and using them ports well to browsing and even managing windows (vimium and aerospace respectively)
secondarily, while i don't think using the terminal is generally better than GUI I tend to work in the terminal anyway, so keeping text editing there makes sense.
Did you ever try Opera 5.12 mouse gestures? That was a feature that completely contradicted your statement. The gestures were direct, unambiguous, and easier to memorize than any set of Ctrl+key shortcuts.
Sadly, that feature has never been fully replicated. The closest I found is Vivaldi, which I am using now, but it is not exactly the same.
Massive wasting of time? with vim you can do something in seconds that with an IDE you would last minutes if not ours.
Check out:
- entr to run commands on modifying files/directories
- plain Makefiles to run your code: git://bitreich.org/english_knight
- LSP and alike tools for your language
I feel like you only read half of that sentence.
entr to run commands on modifying files/directories
Alt-Tab to the command console that I always have running.
plain Makefiles to run your code
I have no idea what the advantage is here. F5 to run my code, including scripted deployment.
LSP and alike tools for your language
I don't know what this means.
I can do more or less the same thing most folks can with an IDE; I just use external tools. I wouldn't claim that Vim is somehow superior. It's just what I use. Every now and then, I noodle a bit on a personal editor that is to ed what Vim is to vi. At some point, I'll migrate to it.
I think there is a bit of a different philosophy that the editor folks have. I can't speak for them, but I can speak for me. I like to feel closer to the code base. I like to have more of it in my head. The best analogy I've found is that using an editor like Vim or Emacs is closer to driving with a manual transmission and with tight steering controls, compared to driving with an automatic transmission with partial self-driving features found in modern cars. There is definitely something to be said about things like adaptive cruise control, lane keeping assist, GPS navigation, etc. But, if you talk to a manual transmission enthusiast, there is a thrill of feeling closer to the road and being more engaged. Both folks arrive at the destination in the same amount of time. But, if you ask each about their experience, they will have much different views of the drive organized in their head.
And yet I get down-voted for expressing a well-reasoned opinion against vim and Emacs.
I've been using vi/Vim for the past 30 years. I take it to every project and job.
I've rarely used an IDE that didn't allow custom key bindings, often with the ability to select a set from a drop-down list to match other IDEs. I've been using mostly the same keyboard shortcuts across IDEs for over 20 years.
if you talk to a manual transmission enthusiast, there is a thrill of feeling closer to the road and being more engaged
Funny you should say that, because I regularly enrage these types by pointing out that if they can't stay engaged as a driver with an automatic transmission, then the problem is with them, not the car. This is a quasi-religious ritual with these people, and a very low-effort way to get a sense of superiority over others (i.e. every driver on the road before ~1970 had experience with a manual transmission and literally anyone can learn in a few hours. It's not a skill to be proud of).
I agree that it is a skill that is easy to learn. The same is true of IDEs. This isn't about skill or superiority, but comfort. Some folks are more comfortable being closer to the machine or the road, as it were. Others are more comfortable having some automation between them and the machine. I think that the better to consider this a matter of personal preference.
The IDE adds a layer of abstraction, and abstraction can be leaky. If you are comfortable with the abstraction, and with the opinionated choices the IDE makes, that's fine. If you are not, that's also fine. All that I ask when I'm bootstrapping a project with a team is that projects be arranged such that they are IDE / editor agnostic. Use standard build configuration / build tools that have appropriate plugins for IDEs, and can also be run in the terminal / command-line. Then, the individual developer can choose to use whichever editor or IDE that developer is comfortable using.
The down-votes that I continue to get in this thread tell a different story.
Calling people whose editor preferences differ from yours "obsessed" "editor-hipsters" is not a well-reasoned opinion, nor is saying that people who drive manual are engaged in a "quasi-religious ritual" to feel a false "sense of superiority". Those are just insults. Hence the downvotes, I suspect.
No one here is forcing you to use any editor, you've only got people trying to explain to you what they find valuable in the tools you're attacking. You're coming off as someone with a major chip on his shoulder. You may use whatever editor you like, but you should consider extending the same courtesy to others.
Exactly, that's your editor and you are used to it. There is no reason at all for you to consider to change editors.
I use Sublime Text, it's my editor, it has been for a decade and I am used to it.
I don't have Copilot or similar, and I'm glad I don't. Not interested in such a thing.
Because you've never truly done it. Like someone who has seen all three sides, I can tell you this: I have never, even once, even for a second, ever regretted my time invested in learning Vim and Emacs. Vim is hands-down the best mental model for navigating through text - I use it everywhere - in my editor, in my terminal, in my browser; heck, I use it system-wide - in my window manager. It's immensely empowering - being able to control things without losing context - your fingertips are in control of everything, you don't even need to shift your hand to touch the mouse or arrow keys. It also liberates you from learning myriad key combinations for every single app, it gives you freedom from having to learn, remember and having to perform weird dactylar dance, where sometimes you can't even reach the exact keys without looking down at your keyboard, not to mention the ergonomics.
And then Emacs. OMG, Emacs is so amazing, you just have no idea. The things you can do in Emacs are hard to describe in words - you just need to see it.
> 90% of my code editing time is spent reading and thinking, not writing or modifying
I spent most of my time taking notes. Emacs is the best tool for that. Matter of fact, I find Emacs is the best tool for any kind of text manipulation. I don't even type anything longer than three words in any app anymore. I'm typing this exact comment in Emacs right now. Why wouldn't I? I have all the tools I need at my disposal - spellchecking, dictionaries, translation, etymology and definition lookup, access to various LLMs - chatgpt, claude, ollama, perplexity, and others, search engines - here's a real, practical example: I would type a search query once and it sends requests to Google, Wikipedia, GitHub, YouTube, etc. I then can pick up the YouTube url, open the video and control its playback while typing - I can pause, mute, resume, speed up the video. All that with the emphasis of the main task at hand - taking notes. Done without leaving the window where the notes are being typed, without having to switch your focus - your mind remains "in the zone". I'm telling you - that's some blackmagic fuckery for staying productive and happy. It's enormously fun when you have complete control over the things happening on your screen.
> I've always found it weird
There's nothing truly "weird" about it. If you are a computer programmer, you do want to be in control of the computing happening on your computer. It's rather weird when there's the opposite - when computer programmers become merely "users", when they are told that "you're holding it wrong" and "users don't know what they want". I for one do exactly what I want - I want the shit on my computer to work and work on my terms, not anyone else's.
Yes, we've all heard the saying that Emacs is an operating system with a text editor built in.
I for one do exactly what I want - I want the shit on my computer to work and work on my terms, not anyone else's.
That's exactly what I get out of the IDEs I've used for decades. This whole argument is so weird, disingenuous, and full of vague strawmen. Just because I don't see the value in investing the time that you have doesn't mean that I'm for giving up control or whatever you were trying frame me as.
Anyway, I'm done here. This site is full of toxic people who are offended that someone doesn't choose the same editor and habits that they do, and any dissent is to be punished.
Perhaps my usage of pronouns in the last paragraph was confusing; I apologize for that. Since we're talking on a public forum, the "you" wasn't aimed specifically at your persona; I meant it in a generic sense, talking about a "proverbial" programmer.
> This site is full of toxic people who are offended that someone doesn't choose the same editor
I don't know where you're getting this kind of vibe from; my comments are free of toxicity. I think this particular website and the world in general is full of all sorts of people. And if you genuinely try to find helpful and beneficial thoughts and inspiration, you may find some - even within the words of criticism and discouragement. Conversely, someone who sees toxicity whenever they don't get conformance with their views may reap only bitterness and dissatisfaction.
Perhaps you should reflect on the source of your anger; just don't waste energy searching for it in my statements - after all, I merely showed you possibilities. I never forced you toward them - the choice to step through or turn away remains entirely in your hands.
I used IDEs also all the time. Makes sense to me.
> Anyway, I'm done here. This site is full of toxic people who are offended that someone doesn't choose the same editor and habits that they do, and any dissent is to be punished.
Yeah, that's weird. You can use all kinds of editors you like. In the case of Lisp, there are arguments that good support for editing Lisp and for interaction with a Lisp runtime is useful (since Lisp is often used interactively). But several IDEs and editors can do that.
Just wanted to give you the impression that are others, who like IDEs and similar tools. ;-)
For serious work, a GUI editor (Sublime is my choice) beats the pants off vim. The only situation I use a terminal editor is when I'm editing config files on servers, and vim sucks at that too - nano is far superior for quick and dirty edits to files. I simply do not think there's a use case where vim makes sense any more.
You will say... oh but my fingers are in the home row! It doesn't matter. I can use the cursor keys just as fast and without having to look at the keyboard, their spatial positions are burned into my brain. And I don't need to change modes to use them! Your way is not superior to my way.
However, the point of all this discussion is that Lisp doesn't provide good enough language servers for modern editors, so Vim and Emacs integration is much better. And that's an orthogonal issue to the fact that some editors allow you to do things one way or another.
They use the "language server" model since a long time, before Microsoft's LSP existed. Thus the need isn't really there, unless one wants to develop with Microsoft products (or similar) and get the needed extensions for Lisp into those Microsoft driven standards. If there were a real need and a real benefit, there would be some better adoption on the Lisp side.
I don't like vscode extensions advertising to me every 5 seconds, auto-downgrading the free versions of extensions, auto-installing aux tools every 5 seconds, having a 400 MB RSS chromium runtime (remember Eight Megabytes And Constantly Swapping? VS code is much worse; and it's also just a plain text editor); nerfing the .net debugger and breaking hot reload on purpose in VSCodium; telemetry, .... it's so noisy all the time. You are using this? On purpose?!
VS code is basically the same idea as emacs, just the MVP variant and with a lot of questionable technology choices (Javascript? Electron? Then emulate terminal cells anyway and manually copy cell contents? uhhh. What is this? Retrofuturism?) and done with the usual Microsoft Embrace-Extend-Extinguish tactics (nerfing pylance, funny license terms on some extensions that the extensions are only allowed to be used in their vscode etc).
So if you didn't like emacs you probably wouldn't like vscode either.
Also, if you use anything BUT emacs for Lisp development, what do you use that doesn't have a jarring break between the Lisp image and you? vim seems weird for that use case :)
emacs is very very good for Lisp development.
On the other hand, VSCode for Lisp is very flaky and VSCode regularily breaks your Lisp projects. Did you try it?
Because of your comment I tried VSCode again and now about 20 extensions (one of them "Alive", a Lisp extension for vscode) complain about now missing
"Dev container: Docker from Docker Compose"
>>./logs/20250112T181356/window1/exthost/ms-vscode-remote.remote-containers/remoteContainers-2025-01-12T17-13-58.234Z.log: >>>> Executing external compose provider "/home/dannym/.guix-home/profile/bin/podman-compose". Please see podman-compose(1) for how to disable this message. <<<< >a239310d8b933dc85cc7671d2c90a75580fc57a309905298170eac4e7618d0c1 >Error: statfs /var/run/docker.sock: no such file or directory >Error: no container with name or ID "serverdevcontainer_app_1" found: no such container
... because it's using podman (I didn't configure that--vscode did that on its own, incompletely. Also, it thinks that means having a docker/podman service running as root has to be a thing then (instead of rootless podman). Funny thing is I use podman extensively. I don't wanna know how bad it would be if I HADN'T set podman up already).
So it didn't actually fix anything, but it removed the error message. I see. And there's no REPL for the editor--so I can't actually find out details, let alone fix anything.
I had thought emacs DX was bad--but I've revised my opinion now: compared to vscode DX, emacs DX is great. You live with VSCode if you want to.
And note, vscode was made after emacs was made. There's no excuse for this.
I think this now was about all the time that I want to waste on this thing, again.
How is this a problem in 2025? shakes head
>VS Code managed to by-pass the qualitiy and amount of extensions/plugins in a fraction of time Emacs took decades.
Yeah? Seems to me these vscode extensions are written in crayon. Bad quality like that would never make it into emacs mainline. And it's not even strictly about that! I wonder who would write a developer tool that the developer can't easily debug its own extensions in (yes, I know about Ctrl-Shift-P). That flies about as well as a lead balloon.
For comparison, there's emacs bufferenv that does dev containerization like this and it works just fine. Configuration: 1 line--the names of the containerfiles one wants it to pick up. Also, if I wanted to debug what it did (which is rare) I could just evaluate any expression whatsoever in emacs. ("Alt-ESC : «expression»" anywhere)
PS. manually running "podman-compose up" in an example project as a regular user works just fine--starts up the project and everything needed. So what are they overcomplicating here? Pipes too hard?
PPS. I've read some blog article to make socket activation work for rootless podman[1] but it's not really talking about vscode. Instead, it talks how one would set up "linger" so that the container stays there when I'm logged out. So that's not for dev containers (why would I possibly want that there? I'm not ensuring Heisenbugs myself :P).
[1] https://github.com/containers/podman/blob/main/docs/tutorial...
Haha, yeah, sure, but of course, no! Similar shit has been said so many times since 1990s. Yet both Vim and Emacs still have vibrant communities, have dedicated conferences, they get mentioned almost every week - here on HN, and every day on Reddit.
Emacs, in experienced hands, absolutely kicks everything out of the ballpark; it's just hands-down the best tool with unmatched text manipulation capabilities. Anyone who says otherwise simply is unaware what you can do in Emacs.
Can anyone in the grand community of VSCode users claim to have a workflow that involves:
- Reading a pdf where the colors match the current color scheme? The scheme that automatically adjusts the colors based on time of the day (because Emacs has built-in solar and lunar calendars)?
- Where they do annotate the said pdf in their notes, where you can jump to the places in pdf from the notes and vice-versa? Where you can scroll the pdf, without even having to switch windows, because you're in the middle of typing?
- Where you can open a video and control its playback - pausing and resuming it in place, directly from your editor, whilst typing?
- Where you also extract subtitles and copy some text chunks for your notes? Where you can run LLM to extract summary for your notes of the said transcript?
- Where you can resume the video-playback at some position in the transcript? Where you can watch the video and chunks of the transcript text get automatically highlighted - the karaoke style?
- Where you can simply type 'RFC-xxx' and despite that being a plain text entry, Emacs intelligently recognizes what that is and lets you browse the RFC entry, in-place, without even googling for it? Or similarly have plain-text of e.g., 'myorg/foo#34' and browse that Pull-Request and even perform the review with diffs and everything?
- Speaking of googling, can you type a search query only once and let it run through different places, finding things in Google, YouTube, Wikipedia, DuckDuckGo, GitHub, your browser's history and personal emails? Or any other places, since it's highly configurable?
- Do you use translation, dictionaries, thesaurus, etymology and definition lookup for any words and phrases, in the midst of typing? I have bound "auto-correct previous typo" to a double tap of the comma key - it's super convenient. Can you do something like that in VSCode easily?
- Do you edit code comments and docstrings in the code, treating them as markdown - with all the syntax highlighting, preview, and other perks?
- Do you have embedded LaTeX formulas directly in your notes?
And that's just a tiny fraction of things I personally do in Emacs - it's just the tip of the iceberg. There are tons of other interesting and highly pragmatic Emacs packages people use for various kinds of tasks. Speaking of packages - my config contains over 300 different Emacs packages, and I still can restart and load it under a second. Can you imagine any VS Code user having installed even half of that many plugins? Would that still be a "workable" environment?
I'd say the mental model for developing the 'Emacs way of thinking' lies, first and foremost, in realizing that Emacs is not "an editor that uses Lisp for configuring it," but rather a 'Lisp Machine' - calling it that would be a stretch, of course. Allow me this simplification here. Emacs is a 'Lisp REPL' that has a built-in editor. That means, instead of focusing on the features of the editor, it's better to study Emacs Lisp and the ways of using it to shape the editor's features.
The great way of developing knack for Lisp is to start with figuring out mainly two things: first, is so called "REPL-driven development". For Elisp that basically means learning the ways of evaluating symbolic-expressions: https://www.gnu.org/software/emacs/manual/html_node/emacs/Li...
Second thing, even though some experts would argue for being optional, I still believe is very important - structural editing commands: ways for quickly moving symbolic expressions around, expanding and transposing them, etc. - you do want to control those (seemingly pesky at first) parenthesis. There are multiple different ways of dealing with those - paredit, smartparens, parinfer, etc.
Then, learning how to effectively search through built-in help, finding the functions, commands, variables, etc. - the stock features already good, although the keybindings and navigating through them might be confusing, there are different packages that can help - consult-info, helpful, etc.
And then, one basically has to get annoyed by small inefficiencies in their workflow. Examples? Do you often have to copy&paste the url of the current tab in your browser while typing? Seemingly simple activity, still requires you to: switching to the browser, focusing on the address bar, copying the url, switching back to the editor, pasting the url, sometimes, it doesn't end there, you need to switch back to the browser, copy the description of the page, switch back to the editor, paste the description, wrap things in a markdown-type of link, etc. Most people would say: "what a nonsense, it's not a big deal, like at all...". But if you think for a minute, how often do you have to do that? Every day, during the lifetime of your computer use. These kind of small inefficiencies are not really distractions for the brain, in fact, that all is over-stimulation of neurons. You may think that you've done that so many times, it actually happens so quickly, you don't even think about it, yet, it still forces your brain "out of the zone", even though might be just for a second.
Another personal example: at some point, I got annoyed by my own typing - I make typos all the time, so I needed a quicker way of automatically fixing typos as I type. First, I wrote a command that does that, and then I bound it to a key - to a double tap of the comma. I'd be typing, and typing, then a typo gets highlighted, I'd quickly tap comma twice and it gets fixed with 90% of predictability. Sometimes it would guess a wrong word, and then if I keep pressing the comma, it would cycle through variants. It feels great beyond words, and it is so instinctive and fast, I don't get distracted at all. Imagine a guitar player with a performance so wild that strings get snapped every few seconds. Imagine that instead of having to replace the entire guitar, there's a special mechanism on it that quickly installs a new string and tunes it up immediately, so the performance goes on without any hiccups. That's what using Emacs often feels to me.
So, my personal advice - simply be annoyed by small things. Then, put them on your TODO list. At some point, you'll get annoyed again and again. Note it each time. And then, try to find a better way to do that thing. The rate of getting more productive is proportional to the level of accrued annoyance.
Same with emacs, now that they've spent some time on performance.
VSCode sits in a weird limbo. It's not an IDE, and it's not an excellent editor. The plugins are usually rudimentary but there's a lot of them. There's no community, instead there's one of the nastiest corporations on the planet faking one.
Re VS Code, I wouldn't say it's in a weird limbo - on the contrary, I think that a "smart-ish, extensible editor" is exactly what lots of people actually want. IDEs are too heavy for many folk, vim/emacs too hard, Notepad too dumb. Things like VS Code hit a sweet spot, hence their popularity.
Power tools aren't as common as regular hammers and saws and so on, i.e. 'less popular', but among professionals few would prefer the unpowered ones just because the threshold to bring one out is lower.
I also think the discoverability in emacs, despite the nice point-and-click interface, is always going to be hampered by the idiosyncratic vocabulary. If you don't already know emacs, `Buffers > *scratch*` isn't going to mean anything to you. And if you do know emacs, you're just going to use the chord. So who's the audience for the point-and-click? People who know what scratch buffers are, but don't know how to use emacs? I don't imagine there are many of those.
> Power tools aren't as common as regular hammers and saws and so on, i.e. 'less popular', but among professionals few would prefer the unpowered ones just because the threshold to bring one out is lower.
Agreed, but I suspect many more professional programmers use VS Code than vim and emacs put together. Though I suspect even more use IDEs.
Sure, that might be the case. I'm not as sure it is good just because it might be the case.
I get the impression that the VSCode has a rather fractured, limited, malware-infested[1][2], JavaScript-churn inundated plugin ecosystem. You have lots of choice, but I'm would be just as hesitant using a random extension as I would downloading some random executable and running it. On the other hand, I'm genuinely surprised (in a good way) how OCD some Emacs users are with the code I publish to MELPA.
[1] https://arxiv.org/abs/2411.07479 [2] https://www.bankinfosecurity.com/visual-studio-code-has-mali...