Tutorial Series to learn Common Lisp quickly
github.com
github.com
- the idea that almost all language constructs are user-implementable with macros is quite impressive (but I don’t buy the argument that “real” macros are only able in s-exprs)
- and the REPL-driven development model was a really beautiful and seamless experience.
But… every time I try to use it on some code, I find too many warts and inconsistencies that I just didn’t want to spend too much effort in making it work. It’s a beautiful language in it’s own ways, but the other parts are too ugly (to me, of course).
I’m fine with it being a lisp or not; I just want a clean language with macros baked in (and becomes the basis of most language features), and with a REPL-driven development model as a first class citizen (with interactive restarts and all).
Unfortunately it seems a pipe dream… so I’m stuck here.
[0]: https://mikelevins.github.io/posts/2020-12-18-repl-driven/
Clojure is also an option, but of course that comes with a heavy JVM burden.
This does seem cleaner. I have some questions though:
- What about the interactive environment like the one in Common Lisp? It's not enough to only have a "REPL", I think.
- There are so many Lisp dialects. Is this one going to remain obscure?
- What's the library situation?Never used a "real" Lisp REPL, but Conjure[0] seems like it ticks a lot of boxes.
>Is this one going to remain obscure?
There are only a ~dozen mainstream languages. Once you get outside of the popular zeitgeist, you have to appreciate exactly what you are getting. It takes a lot of dedication to become fluent in a language/ecosystem, so it is no surprise that people are reluctant to switch to a novel platform. Clojure is far more likely to ever become mainstream (owing to the huge JVM ecosystem), but even that seems to have only limited industry penetration.
Hm.
https://en.wikipedia.org/wiki/GraalVM
"GraalVM is a Java VM and JDK based on HotSpot/OpenJDK, implemented in Java..."
https://en.wikipedia.org/wiki/Clojure
"Other implementations of Clojure on different platforms include: - Babashka,[82] Native Clojure scripting language leveraging GraalVM native image and Small Clojure Interpreter.."
I guess you mean that some particular implementation of JVM isn't needed. But in general Clojure has "j" in its name, it wouldn't make sense without JVM at all.
Think how say CPython needs GCC to build itself, but to use Python as a user you don't need GCC, only the compiled CPython interpreter.
This is the same for Babashka Clojure, you only need GraalVM to build it, but as a user you only need the compiled Babashka interpreter.
There's also Self-hosted Clojurescript which only needs a JS VM. And there is Clojure CLR which only needs a CLR VM. And you've got Clojerl that only needs BeamVM.
But Babashka is probably the one you'd want to try, since it is dependency free, just download the binary and put it on your PATH.
It only adds special forms and doesn't change the runtime semantics at all, so you can drop it into any lua runtime for a strictly better dev experience, or target luajit or whatever but that's not relevant to me.
I haven't used its macro system yet but it looks fine and pretty much like clojure's. I only ever wrote a handful of macros over years of working in clojure so ymmv anyway. Also it has pattern matching, which is ridiculously powerful and smooth in lisps, and it still boggles my mind that clojure doesn't seem aware of that.
Again I haven't used this language yet for anything that didn't require lua, so I'm not sure how that'll go. I'll definitely consider it next time I need a lightweight scripting language though.
The Rhombus project in Racket^1 is an attempt to fix a lot of things about Racket and S-exprs but one of the goals is creating a syntax that is able to be manipulated as easily a s-exprs.
> By “macro-extensible,” we mean that Rhombus users should be able to extend the syntax of Rhombus in much the same way that Racket users can extend the syntax of Racket. Complex syntactic extensions such as object and class systems, static type checkers, and pattern matching should be implemented as libraries while still providing a surface syntax familiar to users of these features in other languages.
1. https://github.com/racket/rhombus-prototype/blob/master/reso...
Rhombus takes inspiration from Honu!
Metaprogamming can be done also semantically in other languages. Elixir for example has also macros and a good part of the core of the language is written with them. When writing a macro you then get an AST in input (represented with tuples in Elixir) and give another AST as output. I think Rust has something similar.
The reason why s-exps are better to write macros is that the AST has the same form has the code you write in the editor. This way textual code and code as data have the same representation, making it easier to reason and manipulate the code.
[1]: http://julialang.org/ [2]: https://www.youtube.com/watch?v=CRiD12Y75wM [3]: https://timholy.github.io/Revise.jl/stable/ [4]: https://kristofferc.github.io/OhMyREPL.jl/latest/
There are macros in non-s-expr languages. Usually then one needs to manipulate an AST data structure. That has consequences in those cases:
a) the macro forms need to be parsed according to a syntax, usually a predefined syntax
b) the AST data structure adds some complexity, which makes simple things more complex, but can also be a help
For Common Lisp a) means that the code enclosed in a macro does not need to conform to a predefined syntax and the macro is responsible to parse it. A typical example is the LOOP macro: (LOOP for i from a to b by 2 when (> a (foo b)) maximize (sin b)). That's a whole different sublanguage, which does not look like the typical nested lists based notation. It uses symbols as syntactic words in a complex syntax, which otherwise would not be valid Lisp.
You can see what it does: one can add arbitrary syntax. Which is good and bad at the same time.
b) means for Common Lisp that there is certain meta-level model, where the language can be used to program itself via procedural macros. Now one has to deal with code, where one can program on a meta-level which looks like the normal level. That's also good and bad. Good because one can apply all the knowledge and tools on the meta-level, too. Bad because it means that a programmer may see in a debugger code-generating code and the programmer sees code constructs which have code generators behind it. In a Lisp interpreter (one which processes Lisp source code at runtime), this will also be visible at runtime. In a language which uses AST data structures, the code in AST representation looks very different from the written code. In Lisp the code as data looks the same as the written code.
The experience is widely different from programming in other programming languages and not that easy to learn: programming a language on a meta-level in itself.
Most programming we see seems to happen without that self-programmability capability.
When I write Emacs Lisp, I don't have a REPL. I just put point at the end of a random s-expression in a random Emacs-mode buffer, hit C-x C-e, and it gets evaluated. Or I do "eval-print-last-sexpression" to get the output on the next line. If I put my emacs in an org-mode code block, I'll get the output nicely formatted by org-mode. I can get close to this in Common Lisp with SLIME, but I've never found it as tight as Emacs Lisp.
But Emacs lisp is still puny. Common Lisps had much higher aspirations than SLIME and a crappy UNIX-based Lisp executable. Allegro and Lispwerks still run on the philosophy that your IDE is just some live code in your lisp image, and you extend that image via the GUI. The same philosophy pervades (non-GNU) Smalltalk implementations.
This is the sort of thing that still makes Lisp (and Smalltalk) absolutely stand out among programming languages, and it will be a crime when it is inevitably forgotten and we are left dumbfounded by the banality of REPL driven development.
https://www.masteringemacs.org/article/evaluating-elisp-emac...
I should look into contributing I guess
That actually is repl driven development. It does not mean that you type everything in on a repl prompt. It means you interactively and incrementally work on a program.
I don't think that's exactly what you asked, however, and I agree that a true Emacs running entirely on CL would be preferable.
GNU Emacs has by far the strongest community and ecosystem though.
For a terminal editor, it’s impressively usable.
If you just mean writing an implementation of Emacs (or something Emacs-like) in Common Lisp, that's not very hard, and it's been done a few times. See, for example, Lem[1] and Hemlock[2].
Heck, I wrote one for MacOSX in about 2001 or 2002.
If you mean a drop-in replacement for GNU Emacs, that's a lot harder. Besides the UI and the editing infrastructure, you need to write a bug-compatible implementation of GNU's elisp, or you lose the whole GNU Emacs ecosystem. That ecosystem is most of its practical appeal. That's a whole bunch of work.
[1] https://github.com/40ants/lem-pareto/blob/master/lem-pareto-...
The problem is after the tutorial is done, all the libraries (look (like (this))) and the language is so powerful it is difficult to know if this 10kb contains a life-changing insight into the nature of programming or a half-hearted implementation of half of something that was a bad idea to start with. Knowing the syntax and technical semantics of the language doesn't help with that.
The topic has been done to death, but the lisps seem to have a social organisation problem that the community never quite managed to get a grip on.
I think this is the worst part of learning to program in CL: the self-flagellating, cynical CL users and the haters that parrot this old, barnacle-ridden trope: it's too powerful! Macros! Nobody can understand it but gods and strange people on the Internet! You will have to listen to these people constantly invade every forum, discussion group, and chat room on the Internet to tell you how you shouldn't be learning CL.
I like powerful languages. They have neat ideas. If they help you get more done in less time with less code, great. Not every software project needs to be a tedious, boring slog that takes 30 developers 8 months to do what you can with CL in a week.
Keep learning CL if that floats your boat. I had a blast learning it and it's a great tool to have and a constant source of inspiration for ideas.
Using Clojure changed my life somewhat, as at some point I rewrote 2000 lines of C++ (to handle splitting and coalescing NAESB cycle pipeline nominations) in about 27 lines of pattern-matching Clojure, and just no one, no one would take it seriously. A usable UI in Swing to manage it wasn't much more, and was cross-platform. Not a fun experience overall, I ended up quitting that company, doing iPhone dev for a bit, then moved back into G&G.
Other than that rewrite experience, I just used Clojure like I used to use Perl. Easy to bang something out in, and performant native Java libraries for things you might not expect, like low-level SQL Server (2005 at the time!) and Oracle (8 at the time) drivers.
I'm back mainly as a C++ developer now in visualization and HPC.
They don't _have_ to. They can lead (and often do) to clean implementations. But we tend to like 'cleverness'. The larger the team, the more detrimental the effect.
That's why we ended up with Java. And more recently, golang. As much as I like Go on its niche, it's a straightjacket. I'd often think to myself "this whole file I've been working for a week on would be 5 lines of scheme or lisp". But then if I get run over by a bus the company could theoretically replace me the next day (they couldn't because there's far more knowledge required than coding, but they think they could).
Maybe not so much of a fallacy when you look at how developer management currently works in most mainstream companies.
It really does seem more geared to going slowly, and making each tiny step be recorded in some system.
To be fair, I've seen much bigger unrecognizable blobs in Java than I have in any Lisp. A language that's too rigid just makes people reach for other workarounds, e.g. craziness in the build system, using lots of reflection, configuring things with XML, etc. It doesn't make it any easier for someone to come into a project and understand what's going on. At least with Common Lisp you can just expand the macros to see what's going on.
These people are actually the reason I went and learned CL. I couldn't understand why some programmers would hate a programming language so much. Now it's one of my favourite languages.
then once you want a more consistent macro language reinvent Forth on your macro assembler;
then once you want a complete stdlib reinvent Lisp on Forth
Right. Some people prefer them to (asd) look(int a, bool b, char *c,int[] d) { &like->this;};
If you’re unfamiliar with Common Lisp, it’s important to understand that it’s a multi-paradigm language. The presence of closures, key functions in the standard library, and the syntax often push you naturally to a functional solution. But you shouldn’t feel bad about writing a LOOP that mutates a bunch of state under the hood. This isn’t scheme!
I think you omitted a word there: one should not feel badly about writing a state-mutating LOOP. That's okay, and often the best solution to a problem.
Heck, I wrote a state-mutating TAGBODY just yesterday …
Till now when I hack on something I use Steel Bank Common Lisp.
Only for the most basic of web apps do I switch to Rails.
These set of tutorials are excellent. Thanks for sharing.
I wish there was something similar for scheme. Most scheme books focus on teaching you compsci with the language and not on teaching you how to build practical things with it.
It's quite hard to onboard someone to a scheme codebase, despite scheme's reputation of being a simple language.
Realm of Racket [0] seems like it might be more OP's speed.
[0] https://www.amazon.ca/Realm-Racket-Learn-Program-Game/dp/159...
https://github.com/hylang/hy/releases/tag/1.0a4
That's Lisp over Python -- you can do data science properly in Lisp now using all you favourite Python packages.
I repeat you can do data science properly in Lisp now.
https://www.wolfram.com/data-science-platform/
I suppose Hylang is quite slow (and possibly has leaky abstractions). Ideally the main C/C++ libraries like libtorch should be wrapped in Common Lisp and not Python.
My preference is using py4cl to use pre-trained TensorFlow models in Common Lisp.
I wrote a short book on Hy that has some deep learning examples, but I don't use Hy that often any longer. I am looking forward to the 1.0 changes.
If you're forced to use Python it's usually because of one of 2 reasons: Either you need to work with colleagues on something (in which case almost certainly they wouldn't accept a lispified version of it) or because you need some of the libraries that are already implemented.
In the latter case, you might say Hy is a good option. But why bind yourself to the slowness of Python when you could just as well use some fast Lisp (e.g. Common Lisp) that interacts with a Python interpreter similar to nimpy [0] from Nim? Not sure if such a thing already exists for CL, but it should be quite feasible to write and then you get access to all your Python library needs while actually using a better language for the rest of your code.
But I'm all ears on the real advantages Hy provides aside from being neat.
couple of solutions exist for this
I'm not lisp enough to make a macro, but supposedly, isn't let under the hood making a lambda function and calling it? That's within the confines of possibility in Python.
It has a rather fast rate of development, which makes coding in it a bit of a moving target.
It’s really impressive work. I guess it’s a reimplementation of a framework he wrote in ADA years ago, so it’s really well developed and thought through.
The macros are orthogonal to this. Macros are really just an example of a more general thing: features that are used to effectively implement parts of the language definition are often lifted up and also made available to users. This effectively allows the language to be extended in a way that very closely resembles the language's built-ins. (This is not always the case; lifting more such things would be a good avenue for extending Lisps.)
`the value-type form) => result*`
Looking at its use, you might think it lets you declare types of things for type-checking purposes. i.e.
(the integer (operate-on 1))
... would make the program assert-fail if (operate-on 1) were somehow not an integer.
The real story is it can do that, but the spec actually says behavior is undefined if the value is not the specified type. So check-failing is one of the many things it's allowed to do.
In practice, depending on your compiler settings, `the` can act like asserts or it can act like places the compiler's allowed to paint racing-stripes on your code to make it go faster by skipping dynamic type checks and letting data structures mangle up if the types are wrong. ;)*
(compress (reverse (explode 'ABC)))
==> 'CBA
Compress & Explode manipulate of list of numbers, of course. In my latest Brain-Fart production I think have finally solved all problems. List of numbers, whose first number is 34 aka \" , is printed as "string" in ppretty-printer and editor. No need to implement strings *EVER*.
Hint: all Quantum computing companies, Google for ITA Software, the NASA for James-Webb's planning and scheduling system, smaller companies (Kina), solo devs that find they are more productive than in Python (like myself).
Plus a whole bunch of supporting libraries, tools, and random toys.
I've used it in preference to anything else since 1988. Lisp is an exotic niche option in the programming language space, so a Lisper needs to be prepared to work in other languages, and I often have. I've been fortunate, though, in that I've been able to work mainly in Lisp for more than half my career since 1988. I use it every day in my current job (working on the radar stuff and some other projects at the same employer).
So If primarily Linux user (sysadmin/devops/ occasional embedder programmer), wanted to learn lispy like language, that is useful which one would you pick?
Emacs one? some common lisp or scheme ?
And as a practical matter if you want to learn Common Lisp you’re probably going to be using GNU Emacs. Thus you’ll probably end up picking up some Elisp too. Both have their share of idiosyncrasies, but a great deal of what you’ll learn applies to both.
That is not true - cl-ppcre generates a chain of closures. Experimental performance is in the same ballpark as typical "bytecode" interpreting regex implementations.
However, the "compiler macro" feature in Common Lisp allows for generating the chain of closures to be performed at compile-time, and invalid syntax (in string literals) can be detected at compile-time.
(Disclosure: I wrote another regex library at <https://github.com/telekons/one-more-re-nightmare>, which does do native code compilation.)
One nitpick: the Google Docs code does not display favourably on mobile. It might help to publish in HTML if you're interested in targeting those readers.
can you please talk to us like programmers? your audience is programmers.