Writing Small CLI Programs in Common Lisp (2021)
stevelosh.com
stevelosh.com
The guile manual is pretty nice, but having something like this will always help people get started. For people that don't know about things like how non-local exits work getting started and doing it correctly can really take some time to figure out.
I'd love to see either provide native support for additional file descriptors, bash `<()` style. I often wrap a shell around commands just for that.
https://www.gnu.org/software/guile/manual/html_node/Pipes.ht... https://github.com/andrewchambers/janet-sh
when i try to remove guile on my fedora machine, then the following get removed too:
abrt
akmods
cargo
clang
cmake
gcc
gdb
golang
kernel-devel
make
rpm-build
rpmdevtools
rust
vcpkg
strictly speaking it is just make and gdb that depend on guile, but gcc depends on make and many other packages depend on gcc.
it appears that on debian make does not depend on guile.
but since i mainly use fedora, all my machines are likely to have guile installed.
Guile is noticeably faster as well, but that is not really surprising considering the implementation differences.
AFAIK, there isn't an official and hassle-free way to generate a statically-compiled binary with SBCL the way Go and Rust can. Like you develop on a debian 12 system, then move the binary to your debian 11 server only to be confronted with the notorious 'incompatible glibc version error'. It's super annoying.
In ECL, I think it's possible to use musl and bundle libecl and mucl libc into the final binary, but I'm not sure.
The caveat is ECL output startup time is around 500ms (as opposed to the much more performant SBCL output)
But I've never had trouble building against older glibcs.
I heard projects (like the Kandria game) build their releases on an old glibc (but latest SBCL and libraries), so it can be shipped to any more recent system.
I started making my own freestanding Linux Lisp because of this exact issue. It's nowhere near as performant as something like SBCL but it's small and has zero dependencies. Once compiled it will literally run on any Linux of the same architecture.
https://github.com/lone-lang/lone
I'm taking a break from this project at the moment but eventually I'm gonna add a feature that lets me put a Lisp script into the ELF itself so I can just copy it with the scripts included and have a totally self-contained freestanding Lisp executable.
People forgot this lesson when macbooks got popular for development, then had to relearn it by upending the ecosystem into containers.
$ time ecl --eval "(quit)"
0.052 secs
$ time ecl --eval "(require 'asdf)" --eval "(quit)"
;;; Loading #P".../lib64/ecl-21.2.1/asdf.fas"
0.301 secs
This is a bad plan for target systems that use glibc, because static linking is not recommended and not supported.
The NetBSD backwards compatibility mechanism is to not change the arguments to an existing syscall, if there is a need to change something then a new syscall will be created and aliased to the old name in the system include files. Old binaries use the old syscall, those compiled after the new one has been added will use that. Languages that don't parse the system .h files like Go need conditional code for NetBSD.
I've written more about this here:
s/backward/forward/
The commlaint here is that the older glibc on Debian 11 doesn't run programs built against a newer glibc under Debian 12.
glibc has very good backward compatibility, with careful symbol versioning and whatnot.
Backward compatibility means that a newer glibc (like in Debian 12) will handle programs requesting the older glibc (e.g. from Debian 11).
Not in my experience. Working through exercises in Practical Binary Analysis, the first problem I needed to solve was running programs compiled against an older glibc on older Ubuntu on a new Ubuntu version with newer glibc, which boiled down to hunting down the glibc so from the older Ubuntu. Without it, the programs crashed on startup. That problem wasn’t even planned by the book author, but it sure was a good thematic fit. :)
I find a lot of the static binary hand wringing basically: did you ever solve how to ship more than 1 file to production? Did you ever need to virtualize any resource on your compute?
Then you're at a minimum using a chroot and containers.
If you aren't shipping to production, if you're really just copying files around between your laptops and you want them to run - you have glibc.
The moment we end up in a world where a "small CLI program" requires docker.. I give up. Time to retire from software and go raise sheep or something instead.
I think I just use linux enough that all the macos bugs I don't experience.
This is a real problem with Linux (not MacOS or other BSDs from what I understand) because it doesn't have a stable ABI (the ABI is basically glibc). It's a design choice, and a pain in the ass for people who want to distribute things as binaries. Which, well, it's not something I do, but, it has burned me using other people's things.
Could docker/containerization solve this problem? Yes, in the same way that a shotgun would serve for eliminating mosquitoes...
I agree with you about containers.
The article was about writing small cli programs, and compiling them yourself. I guess I assumed you would be able to rebuild in this discussion trivially - you wrote the program.
I get that with Go too. I now make all my release builds on the oldest version of Linux I have.
I got the glibc errors when I moved a binary I compiled on the newest Linux Mint to my VPS running some ancient debian.
There is no official, hassle-free way to generate a statically-compiled binary with C for GNU/Linux.
A C program compiled on Deb 12 might not work on 11, if it requests a new version of some function, or a new function.
Go and Rust can, but they are silos.
Glibc dropped support for static linking years ago. You can't deploy a new library to close a library security hole, for programs that have statically linked it.
Rust and Go are doing a stupid thing.
Of course you can. You just recompile affected programs. Just like you need to do in the case of header-only C/C++ libraries like much of widely-used Boost. And if the library is not strictly header-only, but still has significant code in headers, as is often the case in C++, if you only swap the dynamic library, the program is still broken, or much more likely: it becomes broken, even though it originally wasn’t.
A while back I wrote a number of Emacs scripts so that I could listen to Discord messages, as they were streaming in basically, and then 'do Lisp' on them. For example, take the data that was streaming in and format it into a web page with a few links I could then investigate. I may get back into that.
First, you have an API or something which is piping text into your terminal - in this case, into eshell.
Like for example let's say it's churning out lines like this:
"big vote in berlin" https://cnn.com/vote-in-berlin
I move my cursor over the text - remember, this is in emacs in eshell, I can modify the text there - and change the line to read:
"big vote in berlin" (opyn https://cnn.com/vote-in-berlin)
I move my cursor after the parenthesis and type C-x C-e.
In my .emacs file I've defined this function:
(defun opyn (arg) "Open local file in Firefox" (interactive) (let ((filearg (concat "file:///home/julian/Lang/Python/Something/" arg ".html"))) (shell-command (concat "chromium " filearg))) (message "opened file") )
Now it opens that file.
But probably what you want to do is process the data in the text. So here's my emacs function for this (not the code for the data processing itself which was done in Python, separately - but see how you can go combine those together).
(defun refresh (arg) "Rerun local file creation & open in Firefox" (interactive) (let ((filearg (concat "ruby project_refresh_emacs.rb " arg))) (shell-command filearg)) (message "refreshed file for you") )
That's pretty much it. Hope that helps.
(defun opyn (arg) "Open local file in Firefox" (interactive) (let ((filearg (concat "file:///home/julian/Lang/Python/Something/" arg ".html"))) (shell-command (concat "chromium " filearg))) (message "opened file") )
(defun refresh (arg) "Rerun local file creation & open in Firefox" (interactive) (let ((filearg (concat "ruby project_refresh_emacs.rb " arg))) (shell-command filearg)) (message "refreshed file for you") )
An alternative is also Babashka which is excellent for this ! https://github.com/babashka/babashka
Writing Small CLI Programs in Common Lisp - https://news.ycombinator.com/item?id=26493588 - March 2021 (61 comments)
i ended up heading toward something that was more engineered to run a simple repl function call via SWANK, because i'm increasingly settled to having a single, long-running SBCL image rather than starting up individually compiled or interpreted lisp scripts. Feels more lispy somehow -- and definitely better for TDD.
However, I’ve settled on a pattern that works pretty well for the few small tools I write: https://github.com/fiddlerwoaroof/dotfiles/blob/18cecfc93bcf...
That allows adding just 1-line to a module to add a pretty complete CLI and then a string per parameter to properly document options (assuming an existing API using keyword arguments).
It's also not hard to compile & link a static ELF binary with Nim.. I do it with MUSL libc on Linux all the time. I just toss into my ~/.config/nim/nim.cfg:
@if musl: # make nim c -d:musl .. foo static-link `foo` with musl
cc = gcc # --opt:size trades more speed
gcc.exe = "musl-gcc" # NOTE: This also works as a ..
gcc.linkerexe = "musl-gcc" #..per-module foo.nim.cfg
passL = "-static -s"
@end
EDIT: https://nim-lang.org/ has much more detail and there are over 50 small CLI programs at https://github.com/c-blake/bu as rather extended examples. Using that musl approach I get tiny little 100 kB ELF executables that would work on practically any Linux you can `scp` them to with none of the boilerplate/language limits/hassles or speed-limits of, say, Go and run-time start-up times on the order of 100 microseconds. There are all kinds of great libs in Nim, too like @V1ndaar's https://github.com/SciNim/Measuremancer or all sorts of goodies over at https://nimble.directory/
If your program can generate its own man page from the option data, you don't need an actual man page file. man takes input from a pipe, e.g.
# useful use of cat: shows man is not accessing a file
cat foo.1 | man -l -
$ util --man
If you still need man util, that could be a proper, verbose man page.
The ability to use the same language all the way up and down my stack, to import code I use in my program from my script, and get Type-checking means more often than not I’ve got a hash bang with npx ts-node.
Especially useful for testing server code while skipping the HTTP nonsense.
I hate that I’m largely living in a JS world, but I’m fast and effective with it, and the standard library, ecosystem, and tooling has matured enough that I can deal with the quirks.