Qlot: Common Lisp Library Manager

Really unfortunate that Guix never took off. Guile is so much nicer than Nix slop.

The nixpkgs upstream CL infrastructure is also recently updated and IMHO excellent: https://nixos.org/manual/nixpkgs/stable/#lisp

Related:

Ocicl – An ASDF system distribution and management tool for Common Lisp - https://news.ycombinator.com/item?id=35973000 - May 2023 (32 comments)

I also bet my money on ocicl, because of its systems.csv for locking versions of dependencies.

faré the guy who briefly took over asdf development before quitting common lisp entirely to do scheme, or whatever he's doing right now, spent significant amount of time turning asdf from a 10kb file into 3000kb file in order to solve problems that only he himself cared about, or possibly only google cared about because faré worked for google at the time (building portability library that ignored and force-superseded existing solutions, and trying to eliminate corners cases of SAT solving). there was a period where he was harrassing (this is a subjective interpretation of the events, but this is my side) implementors to incorporate his recent asdf changes, that were breaking left and right to the point that multiple common lisp maintainers of multiple common lisps were considering pulling asdf from their repos entirely. things have stabilized, because faré quit and there's no longer volatility. we're at a point where there's just no concensus about relying on asdf for anything, and while some people go all in on 30% of available functionality (as opposed to customary 5%), equally many people, the majority, use it as a glorified (load)(load)(load). this is an unfortunate side effect of activist developers in small communities: they can have outsized long term effect on how things are done.

My thoughts as soon as I read the README: what’s wrong with asdf?

The only time I've had CL code break due to lack of versioning was when I was depending on stuff written by the author of Qlot, so I either avoid or locally pin stuff written by him.

Most of my dependencies are really quite stable.

[edit]

I thought of a place where things broke that aren't related to Fukamachi; sometimes foreign libraries have changed to the point where they won't grovel any more, but I don't think that's something that Qlot aims to solve.

> It's kind of impressive that Common Lisp is the only language I use where I can't expect code written today to work next month due to Quicklisp lacking versioning outside of the date attached to dists.

I think the lack of version pinning is a positive force in stability: reducing the pain of breaking changes increases the likelihood that library authors will break things.

Honestly, I just don't think it bothers people much.

When I need a specific version of a library, I clone it to my local projects directory and checkout the version I need. Problem solved. Quicklisp is just downloading a snapshot of the upstream repo anyway.

For me, if I could change anything about QuickLisp, I'd add something like Python's `virutalenv`. I suspect I can hack something up by messing ASDF's source registries, but so far I haven't wanted it bad enough to implement it.

Lisp macros don’t actually make code harder to read and think about and, unlike the way widely used tools like Flink and Spark do code generation, it’s easy to see what the generated code looks like.

In practices, images are just a deployment tool in CL and most people don’t actually do “version control” by saving and restoring images or using hot code reloading. However, these features enable a development experience that I find much more pleasant than any other language I’ve tried.

The “downsides” you list are actually benefits that non–lisp users try to downplay. Especially macros. Macros give an individual the ability to write things in Lisp that would take a team in any other language. There’s nothing that says a team cannot gain the same multiplier.

I'm actually learning Common lisp for working on my side projects, and those are the reasons that really make the whole thing stick out and amazing.

* Live image and hot-patching is great. You edit it within the code, so if you're making a shooter game, and you're in a "draw HUD" function, you can just edit the code to be what you want and send that function to the running image, to update while you play. Any time you spot a bug, you can isolate, correct, and verify, while still running the game.

* Macros can be thought of as a way to make the code more readable. CL is almost thought of as a "programming language to make programming languages in, so you can solve the problem".

For instance, if you think about any amount of boilerplate. Macros allow you to write it once (because you have to) and then use it in different places at different times. CL makes boilerplate nonexistent.

I do this for wrapping up ncurses functions: to use colours, you must turn on and off certain bits and flags. You can imagine that there's complications when you want to draw a box, or colours text, or coloured underlined text, and there's all sorts of flags and state involved. To write it in C or Python would be a few multi line functions. In lisp it can be made as simple as (with-colours a-colour (draw-text) (with-underline (draw-text))).

Macros enable communication, rather than hinder it. From the mundane ones defined as part of the language like defun and cond, to frequently helpful patterns like with-foo, to the more expressive ones like pattern matching or even domain-specific ones that can even let non-programmers contribute usefully.

No one versions their save-lisp-and-die blobs like source code, they're effectively binary releases. You could version them as releases.

Hot code reloading is not incompatible with logging, documentation, or normal file-based source control. You may have a misconception that we're just raw-dogging it with typing directly in the REPL window like you're more or less forced to with e.g. Python. Sure, sometimes, but the vast majority of the time no, edits are made to files in a text editor (and often saved) and sent to the REPL as complete forms. It's not challenging to keep track of state, either, especially when you can inspect everything whenever you want, trivially reload an entire system (what I often do after editing more than one file at once), and define more complicated state transforms ahead of time with things like update-instance-for-redefined-class. Rather, it helps avoid keeping track of so much state in your head, because when you're developing something and want to test something out, you just do it, you don't need to restart the application and rebuild the state to just what it was before the point where you wanted to test something different. Additionally, if a company isn't using some sort of hot reloading tool (like JRebel for Java), they're easily wasting 20% or more of developer time. (https://www.jrebel.com/sites/rebel/files/pdfs/rw-mythbusters...)

Besides that: I like dynamic typing, SBCL is default-fast unlike dog-slow Python and Ruby, optional types for speed boosts are there with compiler advice and ready disassembly views to help get and verify the speed (they also add a little bit of compile-time correctness checking, and options like Coalton are readily available for a more Haskell-like experience), the experience of conditions and restarts is way better than exceptions for normal development and debugging, debugging itself is more pleasant despite missing some tools because of the conditions framework and fully dynamic no-compromises hot reloading, code I wrote years ago continues to function, code that was written before I was born continues to function, Lisp's expression of OOP doesn't make me want to rip my hair out, and I rather like the uniformity of s-expressions over having to remember all the l-value/r-value/hidden constructor/operator/conversion/initializations/etc. details of other languages.

It looks good to eyes that took a few days getting over their babby duck syndrome.

Has a useful numerical tower containing rational and bignums and an extensive math stdlib to act on them.

A powerful optimizing compiler (available to the user at runtime!) despite the deep dynamism. And pragmas (declarations) much better integrated with the language than in C/C++.

Gradual typing (missing recursive and user-facing parametric typing, I know...).

A stable standard with more than one quality implementation

A pretty good stdlib (missing some stuff due to its age, but well designed in what's there) with some hidden gems like the bitwise manipulation tools or compiler-macros.

https://iterate.common-lisp.dev stomps other languages' idea of imperative iteration.

CLOS/MOP is so flexible the sky's the limit (too flexible, some might say).

I could continue all day, especially if you want the bad and half-baked parts too, but my point is that there are other good points than the "esoteric" features.

Macros are fine and indeed part of the reason to use CL, just don't abuse them.

> what's your reasoning behind choosing this language over the more mainstream options?

Well, I could lie and list all of the great features of Lisp that others will have mentioned already, however the real reason I like Lisp is the S-expressions.

It may sound silly, but I really like the structure S-expressions give a program, which gels much better with the way my brain works than more 'conventional' syntax. Other Lispers tend to say that as you go on using Lisp the brackets disappear, but I like the brackets. They give me a sense of structure that I don't get from C, Pascal, Python, or Haskell. I dunno, maybe it's just my autism talking, but there you go.

Common Lisp is the standard, it's ANSI. SBCL is a very robust, very performant runtime, with a pretty sweet price tag.

Sure, you don't want everyone to invent their own personal DSL:s for everything in production, but when you actually need macros the alternative is usually insane amounts of code that is rather inscrutable and quite hard to change across the code base.

And your developers can use the macro system to improve on their personal tooling, eliminating any concerns about it being detrimental to the team while still improving their working environment. You can also use the susceptibility to macros for static analysis and formatting tooling that makes the team's code more homogenous than it would otherwise be, like doped up linters.

You might not want to hot reload in production always, but when you actually need it, it could give a business edge or save the company from an impending catastrophe. It could also help improve working conditions in development and test, testing out patch ideas faster than subsequent builds would allow.

> For example, Lisp macros are amazing. BUT...

> this is not good for communicating code in big

> teams.

I don't know what this means, how is this any different than any other function ? You document it, use it.. I think whoever has said that macros are a problem in teams is imagining problems.

> Or hot code reloading, which sounds like a

> powerful concept except that companies would

> rather log and document their codebase. It also > makes it challenging to keep track of the

> state.

You do the same you would for every language, you have test suites that exercise the code that you need and they need to pass. All code needs to pass on both the local system and CI, again a non issue.

> If you force teams not to use macros

This would be like forcing C++ programmers not to use objects, you could do it but you'd be doing it wrong.

There are many things to pick on common lisp for, but these are not them.

Start with CLOS being to too customizable, the loop macro being its own DSL, and the REPL/live image based development making you annoyed that other repls are considered equal.

>Those who still use Common Lisp or some variant of Lisp (other than Clojure), what's your reasoning behind choosing this language over the more mainstream options?

One programmer can be amazingly productive due to how expressive and flexible the language is. I find my "developer velocity" (eyeroll) is much higher in lisp than in almost any other language.

>Or hot code reloading, which sounds like a powerful concept except that companies would rather log and document their codebase.

I think this is a benefit primarily realized during development cycles. You never "CTRL+C, edit, recompile, run". You just recompile the single function or statement and things just keep on trucking. If your production environment is containerized you'd obviously rarely be hot-reloading, but how often are you changing prod, anyway?