Practical Common Lisp (2009)
gigamonkeys.com
gigamonkeys.com
The main advantage of Common Lisp is standardization. There's very few languages that can run old code across so many platforms and implementations. But that also holds the language back a bit in terms of progress. Libraries feel like they have been falling a bit behind. It's also one of the few practical languages that has a great multi-paradigm support.
The main advantage of Clojure is implementing lots of modern functional programming ideas. It's main disadvantage is being excessively tied to the JVM. That allowed initial rapid growth, but it brings some side-effects such as ugly stack traces. Aside from JavaScript, there are no alternative platforms.
Racket will now run on top of Chez Scheme. That's another major Lisp worth considering.
Lastly, Julia shows some of the multiple dispatch ideas in CLOS can lead to really efficient code generation when paired with LLVM or a similar backend.
I wish someone put all these interesting ideas in a blender and implemented them on top of Common Lisp, or as a new language. Something functional, with great typing (like Shen), static analysis and an LLVM-like backend are really worth pursuing. We need better languages to address modern challenges, like merging symbolic and probabilistic AI.
- "more interactive" resumable exceptions, better code reloading functionality
- multiple values (cumbersome to use for superficial reasons, but with much better semantics than in e.g. scheme, for sure and probably also Lua)
- compiler macros
- readtable macros (although the design leaves to be desired)
- more sophisticated pretty printer
- more "target platforms"; with clojure it's basically jvm or, to a lesser extent, JS. If you dislike both you're out of luck
- there's a treasure trove of old but interesting material that uses Common Lisp (like PAIP by Norvig)
- much nicer for writing highly imperative code
Common Lisp disadvantages:
- Common Lisp has no eco system to speak off (but at least there's less churn than in clojure land)
- the language is frozen/dead
- career suicide, even compared to clojure
- no high quality free implementations (for example sbcl has a rather poor garbage collector compared to java, go and other widely used languages)
- single precision is de-facto the default floating point type
- built around cons cells which are a terrible data structure: slow, cumbersome and brittle; there's OK support for vectors and slightly worse support for multidimensional arrays and hash-tables and very poor support for generic collection manipulation or efficient functional data structures
- no out of the box support for laziness
- no concurrency or parallelism abstractions
- no good standard pattern matching library
- brain drain: very few smart people doing interesting things left (most left by the 90ies)
- often feels plodding and cumbersome compared to clojure (destructuring, basic string or sequence operations)
> multiple values (cumbersome to use for superficial reasons
How so?
> Common Lisp has no eco system to speak off
False. https://www.quicklisp.org/beta/ https://cliki.net/
> the language is frozen/dead
False. Extensions are being done through libraries and in the implementations.
> career suicide, even compared to clojure
I only ever get positive attention for having been a professional Common Lisp (and Scheme) programmer. Never had trouble finding freelance or full-time work.
> no high quality free implementations (for example sbcl has a rather poor garbage collector compared to java, go and other widely used languages)
How do you go from "SBCL's GC is slower than Java's" (which is a nonsensical statement, because different JVMs have different GCs), to "no high-quality free implementations?" There are three Free Software ones to choose from: SBCL, CCL, and ECL.
> single precision is de-facto the default floating point type
False. Where do you even pick up a ridiculous idea like that? http://www.lispworks.com/documentation/HyperSpec/Body/t_shor...
> built around cons cells which are a terrible data structure
Ignoring how dumb labeling the linked list a "terrible data structure" is, you do realize that data structures are something that you implement in a language, not something that you have to build into one? There are libraries for all kinds of data structures.
> no out of the box support for laziness
There is no reason to make it built in at the language level when you have closures.
> no concurrency or parallelism abstractions
That is handled by implementations and libraries.
> no good standard pattern matching library
Just a bunch of really great ones.
> brain drain: very few smart people doing interesting things left (most left by the 90ies)
You don't seem to know anything or anyone from the Common Lisp world. The best techniques for using the language (advanced use of closures, read macros, the condition system, and applications to domain-specific languages and transpilation) have only started coming out in the past 15 years, from people like Edi Weitz, Doug Hoyte, and Tobias Rittweiler. The only thing relevant from the 1990s is Paul Graham's On Lisp. Norvig's PAIP is very overrated.
Dough Hoyte wrote Let over Lambda which is strong on the same breathless prose gracing its blurp:
> Let Over Lambda is one of the most hardcore computer programming books out there. Starting with the fundamentals, it describes the most advanced features of the most advanced language:
but IMO much weaker than PAIP on the actual "hardcore" programming. I can see why it would appeal to you more.
Wrt single precision being the default floating point type in practice:
* *read-default-float-format*
SINGLE-FLOAT
* (cos 355/113)
-1.0
* (cos (/ 355 113.0))
-1.0
* (cos (/ 355 113d0)) ; get a more accurate answer by using double float literal
-0.9999999999999645d0
The only relevant example in your list is the fact that irrational functions coerce rational arguments to single floats (http://clhs.lisp.se/Body/12_acc.htm). Why do you think that dividing an integer by a single float should produce a double float?
If you don't want this (and I almost never do), you either need to explicitly specify that you want double precision, using d as an exponent marker in your literals or ensure READ-DEFAULT-FLOAT-FORMAT is bound to DOUBLE-FLOAT (not it's default SINGLE-FLOAT) when you read them (which can go wrong more easily, so the former is typically better). The coercion problem is harder to work around.
So in the light of these facts what exactly is false and ridiculous about my claim that single precision floats are the default floating point type in practice in Common Lisp?
IEEE 754 specifies a binary representation. There is no "normal" floating point literal notation. There are numbers in decimal or scientific notation, nothing about which will tell you what kind of internal representation the number should have (not even whether it should be floating point or not). You have to add syntax extensions (this is how Common Lisp does it: http://clhs.lisp.se/Body/02_cbb.htm) or pick an arbitrary representation. Is setting a variable to pick a representation more troublesome or "can go wrong more easily" that setting any other kind of variable for you, or is there something special about that particular variable? You might as well complain that the default Common Lisp reader/printer does not handle C99 hex float literals.
If an unadorned numeric token such as 2.7 generates a single-precision float (32 bit IEEE 754), that is a problem. There are good justifications why the default representation should be the 64 bit double.
The 32 bit single precision has too small a range and precision for general use; it is suitable mainly for saving space in large arrays.
We shouldn't have to use an extra suffix to indicate that we don't want that type, but rather the common double.
Even C gets this right.
That is good, but availability of Libraries are better.
Let's face it. Most people can't use even basic programming features well. When programming went mainstream we traded art for plumbing. What matters now is being able to talk to http end points, working with databases, message buses and myriad of other backend interfaces.
To have widespread adoption you have to do library stuff well, and then may be other features will matter.
Perl is an interesting use case in this regards. The language is very 'functional programming language like' in nature but has a very practical focus for common people, while providing the powerful features for advanced users.
Mature native code compilation (including runtime assembler), type declarations that lead to optimizations in generated code, SBCL compiler can use declared types for compile-time type checks, read/compiler macros, extremely easy interface to C (JNI is a pain in the ass), multi-paradigm and doesn't prematurely optimize like Clojure (I have no need for STM or immutable data structures in 99% of the things I do), very powerful and flexible debugger, CLOS, more sophisticated interactive development.
Clojure is a no-go for me primarily because it's tied to the JVM (or worse, Javascript) and the Java ecosystem but also because it prematurely optimizes with the sort of decisions it has made in the design space (immutability, STM, crippled reader, not Lispy-enough debugging and so on).
Common Lisp is much more flexible in the sort of problems it allows you to explore and attack. Even the special features of Clojure (STM, immutable data structures) can be used in Common Lisp via existing libraries.
Finally, for me Lisp == Lisp Machines and Clojure is a big step back in that regard (and not really modern at all) since it de-emphasizes (by lacking things that Common Lisp has) the interactivity that made Genera so iconoclastic and powerful.
However, I have started playing with Lisp (SBCL) a bit in my off time and am finding it interesting. I have a desire to write some CLI applications and think Lisp might be a better fit than Clojure. I was drawn to it for a lot of the reasons you note:
1. Native compilation
2. Reader macros
3. C FFI
4. Debugger
But it's also a bit like learning how to write with my left hand; my brain has been warped by Clojure to eschew place-oriented development and reduce mutability to a pinpoint. Lisp is completely, unabashedly mutable (which gives a lot of room for improved performance vs. immutable) and I'm struggling with it.
I have enjoyed exploring it a great deal. The project's readme, explains it better than I can:
"folio 2 is a collection of small libraries that provide support for functional idioms and data structures in Common Lisp and a common set of APIs for working with them.
"It's a direct descendant of the older and simpler folio library, with a greatly expanded and reorganized API, and support for more data structures and procedures.
FSet is really cool and I'll probably end up using it in conjunction with https://github.com/fiddlerwoaroof/cl-edn
¹Incidentally this http://www.nhplace.com/kent/Papers/Condition-Handling-2001.h... is my favorite high-level article around one of those potentially mind blowing differences that is Lisp's condition system, written by someone who has spent a lot of time considering fundamentals of programming like what it means to be in an exceptional situation and how to handle it.
Compare and contrast with the designers of today's popular programming languages. There is something severely lacking in the minds behind PHP/Javascript/Python/Ruby and other languages where projected popular appeal and "easy" is the prime design consideration.
Are we worse off? I think so, not just for technical reasons, but because we have lost something greater that can not be easily put into words.
While being nostalgic about the early days of programming languages, I am not as pessimistic about the current state of the art. There were plenty of old programming languages that have faded away despite being important in their day (IBM's PL/1) and lots of work on new languages is pushing in interesting directions (Haskell, Idris).
Common Lisp has hung on and perhaps survivor bias has made it seem special, but to me, it is special. While working with it I feel that I am working with a language of the future from the past.
The landscape of calculation is much bigger than it was half a century ago, when I started programming. It is populated by an enormous ecosystem of processing units from tiny to enormous, embedded to networked across the planet, with single processors to high performance multiprocessors. All of this is controlled by programming and the programmers that wrestle the world of computing into submission. In this Hobbesian ecosystem some languages have prevailed for many years, like Java and C, others have survived even longer like FORTRAN and LISP, and some have simply proliferated rapidly, like Python, all for reasons that are complex and involve human abilities and computer architectures. Not all of them are beautiful, Lisp is, but all of them have a place even if only to reveal what does and doesn't work.
It's similar mind-blowing when you think about this: that people wrote and used an operating system in the 70s/80s in Lisp, where the condition handling system was in full effect throughout the whole user experience (for example when using the file system, the network, the development environment, etc). With all the pros and cons...
But, I too am not necessarily a huge fan of the ecosystems that Clojure is presently tied to. I'd love to see an SBCL implementation of Clojure or (especially) an LLVM based Clojure. I'm grateful when Clojure offers me an out from writing Java, but I also feel like I need to understand both the Java and the Clojure.
The Rackjure dialect of Racket is pretty nice by my estimation (adds some of the conveniences of Clojure, isn't tied to / the beneficiary of an ecosystem like the JVM or Javascript). But I suspect it would frustrate people who like aspects of the CL ecosystem like long-lived system images, and a very neutral disposition toward immutability and functional programming.
It’s a glue language for assembling components, most of which are written in Java.
Timothy Baldridge wrote a python based language heavily inspired by clojure called Pixie. That gives a feel for what clojure is like without the ecosystem. Basically it means reinventing many many wheels.
If Clojure offers nothing without the large JVM/JS ecosystems, then we could say the same about most languages (that aren't the size of Java/JS).
If it’s that the distinct cocktail of features and ideas in the language aren’t worth anything, then I’d respectfully disagree. But at that point we’d simply be arguing aesthetic preferences.
Any new language today has little to compete with existing languages in terms language features, the competition is in terms or the library ecosystems, online help and overall ecosystem. That takes time and money to build.
Given how large the Java ecosystem is, and what it took in terms of money and time to go there is not easy or even possible to match in any relative comparison.
So it makes sense to make sacrifices(if needed) to re use all that Java work that has been done and has been going on over the years.
Clojure is the best thing that has happened to Lisp in years.
I’m old enough to remember a time when Java and JavaScript didn’t exist. They, for better or worse, climbed Mt. Improbable to become widespread. Ditto Python. I’d sure love to believe that a better set of languages could also climb that mountain.
Clojure has certainly gotten a leg up by grafting itself onto Java’s ecosystem, but as you point out, there are sacrifices (stack traces, and inability to support things like TCO for two). Ultimately, I’d love to see a combination of two things happen:
- More libraries that work across all implementations of Clojure
- At least one or two implementations of Clojure where the mismatch between the host platform and Clojure is minimized to useful effect.
On the last point, I’d love to see Clojure hosted on Common Lisp. I don’t hold much hope for this given the toxic reaction that many Common Lispers have for Clojure, but I can see benefits in both directions.
For Clojurists, CL has the benefit of running in lots of different contexts with some useful performance characteristics in some cases (SBCL, I’m looking at you). I also believe a Clojure-on-CL implementation would have the possibility of good stack traces.
For Common Lisp, rather than Clojure just being a mindshare theif, pure-Clojure libraries would become usable to the CL community.
Sadly, I think we’ve already achieved peak collaboration between the Clojure and CL communities. The people who liked what Clojure was about already enthusiastically left, and the remainder seem to mostly harbor sour grapes.
FTFY
Unfortunately they’ve moved to slack for their communication. The clojure group on google is pretty stagnant which is too bad. Topic threads are the exception in a chat room.
Conversations in a forum like groups can serve as their own documentation. Chat logs have to be deciphered.
I like Clojure for bringing maps and vectors into first class language constructs with simple syntax and the fact that it runs on the JVM means I have access to the entire Java ecosystem of libraries and tools.
Vector:
CL-USER 102 > (map 'vector #'char-code "foobar")
#(102 111 111 98 97 114)
CL-USER 103 > (reduce #'+ #(102 111 111 98 97 114))
633
CL-USER 105 > (reduce #'+ #*10101001010101)
7
CL-USER 106 > (aref #2a((1 2) (3 4)) 1 1)
4 CL-USER 109 > (reduce #'+ (read))
(1 2 3)
6
CL-USER 110 > (reduce #'+ (read))
#(1 2 3)
6
You can use the #(...) syntax to include literal vectors in your code, but as the effects of modifying literal objects aren't defined, you should always use VECTOR or the more general function MAKE-ARRAY to create vectors you plan to modify.
CL-USER 111 > (defun maps (function sequence)
(map (type-of sequence) function sequence))
MAPS
CL-USER 112 > (maps #'1+ '(1 2 3))
(2 3 4)
CL-USER 113 > (maps #'1+ #(1 2 3))
#(2 3 4)
Common Lisp chose to define the basic MAP to always specify the result sequence I want to create or NIL for no sequence.
> You can use the #(...) syntax to include literal vectors in your code, but as the effects of modifying literal objects aren't defined, you should always use VECTOR or the more general function MAKE-ARRAY to create vectors you plan to modify.
That's not different for lists and vectors. For both I need to know which I want to modify. Common Lisp has generic operations to create and copy sequences for that: I can call COPY-SEQ to copy the literal sequence or call MAKE-SEQUENCE to create the sequence type I want.
Here (and in many other places) Common Lisp is a low-level language, which exposes these things to the programmer.
You haven't even touched on sets, which in CL are not a built in type, even though there are functions that let you pretend a list is a set, it's not really the same thing when it's not an efficient underlying representation.
Personally I wouldn't call Common Lisp a low level language. A programmable language sure. Low level implies being close to the machine architecture, which it clearly is not.
I didn't admit anything like that. I showed you that vectors are well integrated in Common Lisp.
> Personally I wouldn't call Common Lisp a low level language.
I wouldn't either.
But then I didn't call Common Lisp a low level language. I said: 'Here (and in many other places) Common Lisp is a low-level language' - which means that Common Lisp is PARTLY a low-level language - for example no detection of modification of literal data is required/provided - the programmer has to make sure data can be modified when necessary. A higher-level data-structure would probably require detecting this or would provide only immutable data types. Common Lisp has many relatively low-level features, from cons cells to directives telling the compiler to avoid runtime type checks. Having singly-linked lists is much lower-level than persistent sequences of Clojure.
Generally it's not surprising that Common Lisp has lots of low-level features, since it has been used to write much of itself and large parts of its runtime - where Clojure is a hosted language: large parts of the runtime, the compiler and parts of the library are not written in Clojure: https://github.com/clojure/clojure/tree/master/src/jvm/cloju...
If you look at an unsafe CAR operation, it is often just a single machine instruction.
Take this function which calls FOO twice. FOO is just calling CAR and inlining this call.
(defun bar (a)
(declare (optimize (speed 3) (safety 0) (debug 0))
(type cons a)
(inline foo))
(the cons (foo (the cons (foo a)))))
; Size: 14 bytes. Origin: #x1002FB5E49
; 49: 488B40F9 MOV RAX, [RAX-7] ; no-arg-parsing entry point
; 4D: 488B50F9 MOV RDX, [RAX-7]
; 51: 488BE5 MOV RSP, RBP
; 54: F8 CLC
; 55: 5D POP RBP
; 56: C3 RET
Or take structures in CL. They are defined such that can be compiled to a linear vector-like data structure with slot-access to known and static offsets. No redefinition, no meta-data information - low overhead.
Two of the original goals of Common Lisp were defined as this:
> Portability
> Common Lisp intentionally excludes features that cannot be implemented easily on a broad class of machines. On the one hand, features that are difficult or expensive to implement on hardware without special microcode are avoided or provided in a more abstract and efficiently implementable form. (Examples of this are the invisible forwarding pointers and locatives of Zetalisp. Some of the problems that they solve are addressed in different ways in Common Lisp.) On the other hand, features that are useful only on certain ``ordinary'' or ``commercial'' processors are avoided or made optional. (An example of this is the type declaration facility, which is useful in some implementations and completely ignored in others. Type declarations are completely optional and for correct programs affect only efficiency, not semantics.) Common Lisp is designed to make it easy to write programs that depend as little as possible on machine-specific characteristics, such as word length, while allowing some variety of implementation techniques.
> Efficiency
> Common Lisp has a number of features designed to facilitate the production of high-quality compiled code in those implementations whose developers care to invest effort in an optimizing compiler.
I see a "cleaner syntax in the example" but the fragments of code you don't like, the literals, are something that you won't see in production code except for a few constants maybe. The difference in ~"characters to change" for switching from list to vector is far smaller than the amount of characters spent on this thread... (edit: even over the lifetime of a programmer)
- Compiles to native
- Superior REPL experience/interactivity
- Easier C FFI
- Smaller ecosystem, not as many libraries.
- Dated tooling.
- Inferior support for multi-threading/multi-core.
- Not as homoiconic. Only lists are by default.
- Standard is abandoned. No new versions since 1994. Some CDRs till 2013.
- Leans towards OOP and imperative mutability over Functional and immutable.
- Doesn't have unified abstractions over datastructures.Common Lisp also has full language interpreters: these are implementations of Common Lisp which interpret s-expression level code - not byte code.
Common Lisp can easily dump images and make them executable. Saving and starting them is quick.
Common Lisp has implementations mostly written in Common Lisp - including the compiler and large parts of the runtime.
> - Not as homoiconic. Only lists are by default.
Plus the s-expression syntax is user-extensible - making it as 'homoiconic' as the user needs it.
Dated tooling? Not if you compare it to Clojure tooling, SLIME is lightyears ahead in terms of robustness. I've lost track of the Clojure REPLs (nrepl, CIDER, inf-clojure, ...) that still suck.
Re: "smaller ecosystem, not as many libraries", surely you are joking or not at all aware of things like the CMU Common Lisp repository. If we disregard Java libraries, Clojure doesn't even come close to CL code that has been made available over the last few _decades_.
One doesn't need CDRs or another (expensive, time consuming) standardization process when there are great portability libraries (FFI, threads, sockets, convenience macros, ..) or one can pick a defacto standard implementation that offers all of the above.
I'll raise you a car.
I found lack of TCO to be a little like a stick in the neck when it comes to both Clojure and Emacs Lisp. It prevents you from thinking in the way you have been thinking in other Lisps.
I understand Clojure depends on JVM and JVM doesn't do tail call eliminations yet, and Rich Hickey is right in thinking that method calls are too much a platform thing and we shouldn't be working around this limitation with a hack. However lack of TCO in Emacs Lisp is a little not easy to live with. From what I understand patches have been submitted to Emacs to make this happen but haven't made it to the upstream.
TCO is very important for any language that sells functional programming these days. Apart from the performance advantages, Recursion is a thought framework. Limitations are not easy to live with.
I hope some day Clojure and Emacs Lisp get TCO.
On a tangential note, JVM is a great platform, its a also a widely used platform, so changes are going to come in very slowly. So I'm not holding my breath waiting for TCO.
I started with Scheme and Lisp (Standard Lisp, ZetaLisp, CL, ...) and wrote/used TCO a lot. I tend to view the use of TCO for basic iteration as a mistake, which makes code harder to read. I tend to like either higher-order iterative constructs or powerful iteration constructs like CL's ITERATE: https://common-lisp.net/project/iterate/
I find the Clojure hack ugly.
I understand some one like you is far more experienced and has been around for a long time. So obviously you know better than I ever will. And you are right. Even with very minimal Lisp exposure, after using Scheme and Common Lisp, Clojure's TCO thing feels unacceptable.
In fact after using Common Lisp, and I'm a novice, even some one like me feels Common Lisp to be a far superior language compared to other Lisps around.
The issue is, like I said, I started learning lisp starting with Scheme, and I started recently. First book was SICP, second was 'The Little Schemer', third was 'The Seasoned Schemer'. At that point your brain is hard programmed to think in terms of recursion, and whenever I see loops my mind moves to recursion automatically. You have to trust me when I say this, when I read PG's ANSI Common Lisp book , I skipped the sections on iterations and loops, because I thought I would never use them anyway.
I have to say those Schemer books do their job really well.
May be I just need more practice, so I will develop more stuff in Clojure, now that its not even possible to do that Clojure, eventually that TCO addiction should go away.
:)
You have read a good selection of Scheme books. It's very useful to explore that style of programming and be able to really understand. Take it only as a hint, that there is more out there and that different styles and approaches exist. One can program with functions, one can program with linguistic abstractions, ... McCarthy called his paper: 'Recursive Functions of Symbolic Expressions and Their Computation by Machine' - this hints that one programs with recursive functions - but it enables also to compute with symbolic expressions - not only as data, but also as code. The evaluator as a recursive function, which implements Lisp.
There are a bunch of oldish Lisp books, but which might be read with some benefit even today. I'd would for example mention Performance and Evaluation of Lisp Systems (benchmarking Lisp), Keene's CLOS book, PAIP (writing classical non-trivial code and optimizing it), AMOP (meta-objects, extensible architecture), On Lisp (macros), Land of Lisp (fun&games), Common Lisp Recipes (Lisp lore), ... and also the Lisp Machine Manual ...
For a bunch of, often old, Lisp related books see my list: https://www.librarything.com/catalog/lispm&tag=lisp
And very likely tomorrow. :)
As for introduction, another nice book is Land of Lisp - it introduces you to most of Common Lisp by means of writing games of increasing complexity.
Another example could metadata in image (jpg, ...) files for example.
Blockchain in five minutes?