CL21: An experimental project redesigning Common Lisp
cl21.org
cl21.org
One difficulty with promoting a more modern layer to common lisp is that Clojure is already such a productive and practical language. I have written a few common lisp books, and remain a fan of the language, and an upgrade does sound good.
I am curious to hear the opinions of the heavy hitters in the common lisp world.
In this case, theirs isn't the opinion that counts.
A bit of a tangent, but I've been looking at the series library[0] recently. It makes it possible to write functional style code which is compiled into a loop for efficiency. For example:
(defun integers ()
"Returns a series of all of the integers."
(declare (optimizable-series-function))
(scan-range :from 1))
(defun squares ()
"Returns a series of all of the square numbers."
(declare (optimizable-series-function))
(map-fn t
(lambda (x) (* x x))
(integers)))
(defun sum-squares (n)
"Returns the sum of the first N square numbers."
(collect-sum (subseries (squares) 0 k)))
(defun sum-squares (n)
"Returns the sum of the first N square numbers."
(loop for i from 1 to n
for square = (* i i)
sum square))
* (disassemble #'(lambda () (loop for i from 1 upto 10 summing i)))
; disassembly for (LAMBDA ())
; Size: 62 bytes. Origin: #x100306FE84
; 84: BB02000000 MOV EBX, 2 ; no-arg-parsing entry point
; 89: 31C9 XOR ECX, ECX
; 8B: EB21 JMP L1
; 8D: 0F1F00 NOP
; 90: L0: 48895DF8 MOV [RBP-8], RBX
; 94: 488BD1 MOV RDX, RCX
; 97: 488BFB MOV RDI, RBX
; 9A: 41BBA0010020 MOV R11D, 536871328 ; GENERIC-+
; A0: 41FFD3 CALL R11
; A3: 488BCA MOV RCX, RDX
; A6: 488B5DF8 MOV RBX, [RBP-8]
; AA: 4883C302 ADD RBX, 2
; AE: L1: 4883FB14 CMP RBX, 20
; B2: 7EDC JLE L0
; B4: 488BD1 MOV RDX, RCX
; B7: 488BE5 MOV RSP, RBP
; BA: F8 CLC
; BB: 5D POP RBP
; BC: C3 RET
; BD: CC0A BREAK 10 ; error trap
; BF: 02 BYTE #X02
; C0: 19 BYTE #X19 ; INVALID-ARG-COUNT-ERROR
; C1: 9A BYTE #X9A ; RCX
[1]: https://blogs.oracle.com/projectfortress/entry/fortress_wrap...
Take princ #"Hello, ${name}\n". Why not just use print like the rest of the programming world. Okay, some languages use writeln, etc. But princ? Why not print?
A little further down, we see while-let1. Why the number? If you want to call it "Common Lisp in the 21st Century" you need to clean up the syntax.
There's usually a very good reason behind all those "unusual names" of Common Lisp. I don't think that dumbing things up so that they look more similar to everything else in a language that is already different from anything else is the way to go.
I really like the direction of this project, but I agree with the parent comment, it doesn't go far enough.
Consequently, just because a language seems superficially familiar, that doesn't make it easy to learn - for programming languages it takes weeks to understand the basic necessities, whereas it takes years to become a master, regardless of the programming language you're talking about.
Also, Erlang's Prolog-like syntax sucks, not because it's unfamiliar, but because it objectively sucks.
The specific example is talking about doing away with the format function in favor of the princ function. What is the argument against calling it print? No one is suggesting fprintf by the way.
You and the parent are also citing spoken languages, and certainly if you are a Chinese speaker then the difference between princ and print is going to be nominal to you. However, if you are a native English speaker, not so much. Really though, once you learn one programming language with standard library function names in English, then the second will be easier to learn if it uses similar names.
Back to the subject at hand, the case being made here is that most modern languages choose things a bit easier for the brain to parse; they aren't overly shortened, they aren't Hungarian notation, etc.. The reason for that is that it requires enough brain power to learn a new language, its structure, libraries and quirks without also having to memorize strange sequences of consonants.
Here is some code I wrote yesterday. Even if you never saw Ruby before in your life, don't know what blocks are, and have no idea what the array/enum functions are, you can probably figure out what this code is doing.
invoices = @organization.subscriptions.collect{|s| s.invoices}.flatten
current_invoices = invoices.select {|i| i.invoice_period_begin_date <= Date.today && i.invoice_period_end_date > Date.today }
current_paid_invoices, current_unpaid_invoices = current_invoices.partition {|i| i.paid?}
and THAT is why if you are designing a new language from scratch, you should strongly consider using intuitive function names.
Very simple, actually. You know the term REPL, right? It stands for Read-Eval-Print Loop, and originates with Lisp. Actually, every Lisp gives you read, eval and print functions which work closely together. In particular, you expect read to be able to understand what print outputs, because they work in a loop.
So print is already taken. As for other prin* functions, see here: http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec....
There is a good rationale behind those names.
And no one uses princ for interpolated string output anyway.
As for your code, things like princ or while-let1 are more-less equivalent of @, |sth|, ?, etc. in your code - i.e. there are some basics you have to learn with every language.
> Back to the subject at hand, the case being made here is that most modern languages choose things a bit easier for the brain to parse; they aren't overly shortened, they aren't Hungarian notation, etc.
Well, you don't get closer to that goal than with Lisp and the majority of its naming conventions. From make-instance to destructuring-bind to multiple-value-bind to update-instance-for-redefined-class, most of the things are named pretty well and readable for a programming language.
I can't help but see these arguments about "intuitive names" as finding just another excuse not to learn something new.
Yes, and notice how it doesn't stand for Rrepl-Evl-Princ Loop.
> There is a good rationale behind those names.
No, there is a good rationale for having separate functions, there is no logical rationale for prin1, princ and pprint.
Let's take "prin1 produces output suitable for input to read." from the link you provided. Since we are redesigning the language anyway, let's call that evalable-print or readable-print, or anything else in the world that semantically captures it's meaning. Because when you have a language with hundreds of obtusely named library functions it requires an unnecessary amount of cognitive load added to the already taxing act of programming. Functions that are easier to remember are directly related to code that is easier to read and understand.
I remember struggling with Haskell because there is a function called 'nub', which wasn't anywhere near what I would have called the function if I had named it. Hoogle says "(The name nub means `essence'.)" In Lisp it is called 'remove-duplicates.' In Python it is list(set(x)). In SQL it is 'select distinct.'
So what advantage does nub pose? Not one of clarity, but of convenience for those "in the know." (Which is not those who are learning the language.)
That's rhetorical, you don't need to reply.
Seriously, Lisp has long history and had half the "new amazing features of today" 30 years ago. There is a rationale for naming conventions. Just because you're free to call a string-spitting function "print" in Ruby or Pascal or whatever doesn't mean you can do it in a language with a proper REPL.
Lisps are a beautiful and powerful family of languages, but you are conflating the implementation with the idea. The names of functions from 30 years ago are not what makes Lisp special.
Just because CL has a long history, it doesn't mean that everything about it has to be correct. Otherwise why start a project of CL remodernization in the first place?
My 1.5cents
I don't mean that I don't understand it. I even know they come from assembler instruction names. And they still look weird to me and still have no meaning despite having read what the abbreviation expands to three or four times.
You'd be far better off shipping "first" & "rest" and whispering to the old hands that you're sure they know how to "fix" the problem in their code, than presenting new people with car & cdr and then try to wedge an explanation into a brain that just threw an exception and went "Wait, what?"
I like the distinction of car/cdr meaning pointers vs first/rest meaning "start of list/rest of list".
It weirded me out at first, but I came to really appreciate it.
(def x ((((5 4 3 2 1) 6) 7) 8 9))
(get-in x (0 0 0 1))
Some examples (from Haskell):
>>> ((((5,4,3,2,1),6),7),8,9) & view (_1 . _1 . _2)
6
>>> ((((5,4,3,2,1),6),7),8,9) & (_1 . _1 . _2) .~ 4
((((5,4,3,2,1),4),7),8,9)
>>> let foo = ((((5,4,3,2,1),6),7),8,9)
>>> foo & view (_1 . _1 . _2)
6
>>> let six = _1 . _1 . _2
>>> foo & view six
6
>>> foo & six .~ 4
((((5,4,3,2,1),4),7),8,9)
>>> foo & six .~ "abc"
((((5,4,3,2,1),"abc"),7),8,9) (let ((x '((((5 4 3 2 1) 6) 7) 8 9)))
(syntax->datum
(syntax-case x ()
((((_ p) _) _ ...) #'p)))) - both 3 letters each
- only differ by 1 letter a|d
While writing a toy lisp I looked for other names with structural qualities, actually not names, more symbols since these weren't words.
Things like Oo/oO, xo/ox or _/__.
car and cdr are mnemonics related to the particular instruction set of the first computer on which Lisp happened to be implemented, which is great for people with a background with system programming on that system, but not so good for anyone else.
They're entrenched enough in the language that they are worth changing unless you are intentionally building a new Lisp-like language from scratch with some other main motivation and the change is just a side issue -- and it is clear that having something that doesn't imply list semantics for cons cells like first/rest does is good -- but car/cdr aren't a particularly good pair.
If I tried to explain deeper, Lisp focus on recursion over cons-lists was so foreign to the mainstream paradigms, that car/cdr, as most used operations, became symbolic axioms in my mind.
[1] which goes all in on the intuitive, human, pragmatic side of things, throwing away some old lisp idioms.
As funny as it is, people whining about "car/cdr" and saying "language X did it better because it called them first/last, or head/tail" don't realize that it's the language X that lacks semantics to express anything else than lists using cons cells.
http://www.lispworks.com/documentation/lw61/CLHS/Body/f_wr_p...
If I were doing this I'd also get rid of the * around the hash keyword and either just go with the keyword 'hash' or replace it all together with an operator.
It's almost like lispers want the opposite of java, long verbose keywords and syntax and then as short as possible variable names.
Why not switch the parens to something that's a single keystroke? Why not get rid of the parens in most cases? In some cases this seems to make the parens problem with lisp even worse?
(loop for key being each hash-key of *hash*
using (hash-value val)
when (< (length key) 2)
do (format t "~A~%" val))
(doeach ((key val) *hash*)
(when (< (length key) 2)
(princ #"${x}\n")))
the former also required fewer shift-character keystroke combinations
( ) * < " ~ %
( ) * < # " $ { }
this just seems to be moving things around, not really improving anything
I'm not a lisper, so somebody correct me if I'm wrong
Many a well-intentioned Lisper eventually begins to see deficiencies where there are none and attempts to improve their implementation of choice. Common Lisp is one such target because people coming from in-vogue languages are used to the language changing every year or two. Invariably this leads to superficial changes at best which are enabled, surprise of surprises, by Common Lisp because it was designed to be extensible.
My theory is that if your language of choice needs to change its grammar rules to introduce new features you're basically admitting that you should have added macros programmatic access to the reader.
> It's almost like lispers want the opposite of java, long verbose keywords and syntax and then as short as possible variable names.
Well syntax is at a minimum in Common Lisp. It tends to avoid syntactical short-hand. The most common I know of being QUOTE, BACKQUOTE, UNQUOTE, UNQUOTE-SPLICE, and FUNCTION: ' ` , ,@ and #' respectively.
If you read the preface to the first edition of Structure and Interpretation of Computer Programs I think you will find an enlightening explanation of, Why Lisp?. To paraphrase a brief summary: there are very few ways to form compound expressions and hardly any syntax at all so that the programmer can get to the problem at hand within an hour: the language disappears.
The longer names are chosen stylistically based on the maxim that computer programs are meant for humans to read and only incidentally to be executed by a computer.
Lisps are not, contrary to popular mythology, list-processing languages. It's a symbolic language. The relationship between symbol and data allows the programmer to think primarily about the transformations to data without the abstractions becoming weak.
> Why not switch the parens to something that's a single keystroke?
You could. Common Lisp gives you access to the reader. You can write a more fitting syntax you are more comfortable with.
I've written reader macros to parse assembler instructions for an emulator I wrote in Common Lisp.
> this just seems to be moving things around, not really improving anything
That's my impression too. These sorts of experiments are worth trying from time to time in order to see if something can come from them. Maybe something will come out of this.
But ultimately people bike-shed over "car/cdr" and specialized functions for specific data-types (ie: AREF, GET-HASH, etc). You can write higher-order functions to abstract away sequence access and iteration if you want: people have and there are good libraries for it.
The standard hasn't changed AFAIK because:
1. It'd be expensive and nobody wants to fund another ANSI committee.
2. There haven't been any significant proposals the necessitate revisiting the specification.
People complain about the lack of run-time support for concurrency and IO in the specification. I don't think they realize, coming from other languages, that Common Lisp doesn't need it. The specification defines the language and a minimal ball of mud for doing practical work with it. Things like concurrency, threading, and IO are implementation issues and have been solved by high-quality open-source implementations for quite some time. We even have good libraries that provide a cross-platform layer to things like threading.
In that case, it would be useful to newcomers to introduce those facilities into the core of the language. The more batteries are included, the easier language adoption becomes.
> But ultimately people bike-shed over "car/cdr" and specialized functions for specific data-types (ie: AREF, GET-HASH, etc). You can write higher-order functions to abstract away sequence access and iteration if you want: people have and there are good libraries for it.
To me, this is not bike shedding, but user experience design. The more learning that is required to accomplish a given goal the first time, the less productive and more difficult that goal becomes. "car" and "cdr" are not intuitive to me. Why not use "first" and "rest" or "head" and "tail" for CL21 instead?
Nothing about programming is intuitive.
The problem here is that you've adopted ideas and notions from another language and are bringing that baggage to Common Lisp. If you learned Common Lisp first you might not have this very issue you describe.
For historical reasons, "car" and "cdr" are the names of special pointers to the elements of a cons cell. There's nothing about them that have anything to do with lists. You just happen to be able to construct lists by linking together cons cells. CAR and CDR simply deference the pointers in the cons cell:
(atom . atom)
Those are higher-order functions for operating on linked list and are in the Common Lisp specification anyway.
Suggesting that Common Lisp should remove them is like saying you should remove pointers from C.
Then names like "left" and "right" rather than mnemonics for "contents of address part of register" and "contents of decrement part of register" would make sense.
The bonus discovery that the (car (cdr some-cons-cell)) could be shortened to CADR via a macro was just icing on the cake. Now we have a macro that can access any part of an abstract tree. Nice.
There's nothing intuitive about programming. There's no analogy you can apply to CAR and CDR or even CONS which will suddenly make sense to your primitive mind. You can't pick up a cons cell in your handle, fiddle with it, and throw it back at the computer. There's no analog in nature which resembles it. It's an ephemeral thing that only exists inside a machine because an idea someone had made it so. So stop looking.
If that's the biggest hurdle to learning Lisp I'd be surprised if someone could be taught to program in any other language. Try teaching C declaration syntax to someone.
char *(*c[10])(int **p);
... to red-black trees, skip lists, linked lists, circular buffers, trieps, whatever.
Update: I don't mean to say I'd be surprised if someone could be taught to program in any other language in any sort of glib or pejorative sense. Of course one can learn to program without hearing about a cons cell. What I mean by that is it takes some willingness to accept abstract concepts and turn them into symbols in order to learn how to manipulate them.
If that was true, than naming constructs in programming languages wouldn't matter at all.
> If that's the biggest hurdle to learning Lisp
Its not, its just one of many small hurdles that keep Lisp as a fairly niche language, despite its strengths when viewed from a 30,000 foot level.
> Try teaching C declaration syntax to someone.
C-family languages declaration syntax is a major factor in the popularity of dynamically typed languages (newer statically typed languages that are starting to take back some ground mostly reduce both the clutter of that syntax and the need for declarations at all.)
So, other than illustrating the a similar (but larger) problem to the car/cdr problem, I'm not sure what the relevance is.
http://dictionary.reference.com/browse/intuitive
I think we're diverging off the track here. Naming is important. Programming has borrowed quite a lot from various places to name the often weird, alien "things" we work with. But a rose by any other name is still a rose.
There is no name for which there exists an analog in nature that we can apply to CAR and CDR. You can't think about them intuitively. As demonstrated elsewhere calling them FIRST and REST is a mistake. FIRST, SECOND, THIRD and so one are just English words for for CAR, CADR, CADAR, and so on.
So it's best not to get stuck in the "intuitive" trap. CAR and CDR have stuck around because people have argued about it endlessly for 40 years and haven't come up with a better name. They took on abstract symbols and the people writing Common Lisp code kept shipping systems.
> Its not, its just one of many small hurdles that keep Lisp as a fairly niche language
I don't think that's the reason for it's lack of market share. That has more to do with corporate politics and business. Read up on the history of Symbolics.
tldr; there were once only commercial implementations of Common Lisp that cost far too much and delivered little. Because money.
So a couple generations of programmers didn't get exposed to it as their first language.
> C-family languages declaration syntax is a major factor in the popularity of dynamically typed languages
Is that so? Why then are their interpreters and virtual machines written in C or some other language? PyPy is making an effort but even then... lots of C. Most implementations I've seen of Common Lisp are written in Common Lisp. Odd.
CAR/CDR isn't a problem. I've explained it five times in various posts on this thread. There's even a link to the HyperSpec (the HTML form of the ANSI specification). It's quite simple.
The real problem is that Common Lisp isn't the first language for most "newcomers." So they come packed with notions and biases that they forget to leave at the door. If C is your first language pointers aren't a big deal and you learn to read the declaration syntax. You know it's not great and that it's difficult to pick up but once you get it you move on. In a similar fashion when you pick up Lisp you figure out CAR and CDR.
Changing the names of CAR and CDR isn't going to make Lisp any more appealing to newcomers. You just have to learn what they are and move on.
Which is complete bullshit.
Or is my terribly lazy and misinformed history of the AI winter and the rise of inexpensive Unix mainframes inaccurate as to the reason why Lisp suddenly dropped out of vogue? I've only read accounts of it online. If you have a link to a better source I'd be happy to read it. :)
that's bullshit. Complete bullshit.
Lisp Machines were programmed in ZetaLisp (aka Lisp Machine Lisp) and Interlisp. Not Common Lisp. Common Lisp appeared later on Lisp Machines as an additional Lisp dialect and software was than gradually moved to Common Lisp.
The first version of Common Lisp was initially designed between 1982 and 1984. The defining book was published in 1984.
CMUCL, a free Common Lisp, appeared early. CMUCL was many years forked into SBCL, but CMUCL is still available today. KCL, which then was patched into AKCL, ... Which today is ECl, MKCL, GCL! ... appeared early. CLISP, a free implementation in C, appeared early.
A bunch of mid-priced Common Lisps was available for Windows and Macs.
The commercial ones for Unix were Allegro CL, Lucid CL and LispWorks. They were slightly more expensive and great.
There was always a choice of various Common Lisp implementations...
'Unix mainframes?' What is a Unix mainframe? Unix ran on various types of machines, but there was no rise of a Unix mainframes.
Common Lisp was early available with Lisp. Every major university had a site license for a commercial Common Lisp. I used CL on a SUN cluster. Steve Jobs shipped his first NeXT machine with Allegro CL included.
Common Lisp was never bound to Lisp Machines or only on Lisp Machines. Far from it. The whole idea of Common Lisp was to have the language available on a wide variety of machines: personal workstations, PCs, Unix systems, IBM Mainframes and supercomputers like the Cray.
Maybe I was struggling to recall Dan Wienreb's post[0] on the failure of Symbolics and somehow confusing that with other things and not making much sense.
Thanks for clarifying.
[0] http://lemonodor.com/archives/2007/11/why_did_symbolics_fail...
> The problem here is that you've adopted ideas and notions from another language and are bringing that baggage to Common Lisp. If you learned Common Lisp first you might not have this very issue you describe.
Indeed, very little about programming is intuitive to a non-programmer. You're also right about me bringing in baggage from other languages. Those are valid points, yet I would still like to see CL become more appealing to programmers from other languages, whose baggage becomes relevant to their intuitions.
> Those are higher-order functions for operating on linked list and are in the Common Lisp specification anyway.
This Common Lisp reference [1] equates "cdr" with "rest". I am not familiar with the source, so is this a good reference? If so, then I must still argue that "rest" returning the cdr of a list instead of all the elements except the first is not intuitive to me. Or maybe I am misunderstanding something. My familiarity is with Clojure, not Common Lisp, so that might be interfering with my ability to understand what you mean here.
> Suggesting that Common Lisp should remove them is like saying you should remove pointers from C.
That is an interesting point. I am not very familiar with the organization of data structures in Common Lisp.
I suspect that's the reason for libraries like CL21.
However the specification for the language known as Common Lisp defines a language that is mutable by the user. We don't need to change the specification to experiment with syntactic changes that could benefit new users.
> This Common Lisp reference [1] equates "cdr" with "rest".
That would be a common mistake I've had to overcome when I first started learning Common Lisp. Consider:
(cdr '(1 . 2))
Hint: It's not a list. So why conflate CDR with REST? They're completely different things.
(cddr '(1 . (2 . 3)))
(cdr (cdr '(1 . (2 . 3))))
Again, not REST. Not a list. It just so happens that when you have a structure like:
(1 2 3 4)
(cdr '(1 2 3 4))
(1 2 3 4)
(1 . (2 . (3 . (4 . nil))))
(Edit: trying to escape HN formatting conventions.)
From this and the author's name (Eitaro Fukamachi), I wonder if he is not a native English speaker.
This could shed light on differences in aesthetic choices about function names. Difference between "princ" and "print", or awkwardness of "while-let1" may not resonate as much for a non-native speaker.
And no, I do not miss reader macros. Perhaps more surprisingly, I don't miss CLOS at all, much as I always admired its design. It's just... Unnecessary.
What I do enjoy is STM, good performance, Java interop, ClojureScript, core.async and lots of excellent code with fresh ideas popping up all over the landscape.
When I was working with it on a non-trivial project I despised the JVM stack traces, image-less environment, and lack of access to the reader. I assume that eventually Clojure will steal these ideas and adopt it in its own way... but these old, un-modern ideas of conditions and restarts, images, and reader macros are actually really useful. I hope they do take a hint.
For many, Clojure may be a better choice as you suggest: perhaps especially for web projects. But I think for some groups and applications, CL remains a better choice.
Some of them could possible be considered for alexandria https://common-lisp.net/project/alexandria/. Admittedly I am not sure how alive development is but it certainly would be worth breathing new life into as it is probably the most heavily used CL library.
As things like STM and lazy which Clojure has, they have been implemented at the library level for CL.
https://common-lisp.net/project/cl-stm/
The beauty about a Lisp is that you can easily add so much on top of the language that will feel as if it was always there. CLOS is a good example of this.
In the case of JVM with CLojure, and of course others will disagree, I don't think accessing the Java ecosystem is a good thing. The mindset there is monolith "Enterprise Solutions" "Inversion of Control frameworks" which invariable mean systems that are so complex the authors don't even understand them and lets write XML above all else.
Also if you want CL->Javascript; Parenscript.
https://common-lisp.net/project/iterate/
In my personal opinion, the iterate loops read better than the code samples of the main page.
1. It has CLOS but it doesn't really embrace it in standard library. You have list, vector, hash-table, each with its own iterator function. You have lots of #with-xxx macro. All modern languages utilize the idea of common interface like #iterator or #dispoable. You can clearly see CLOS library and pre-CLOS library in CL. CL needs to eat its own dog food (CLOS) more.
2. It lacks basic practical API library, while another containing big complicate academic functions. Its #format function is probably Turing complete. It has function to format number in Roman numeral. But it doesn't have function to deal with datetime. No networking IO function. No Threading library.
3. Condition and Restart could take the idea from Dylan. Tying condition to available restarts, and making class hierarchy of them, is a better idea. This problem stems from CL not actually utilizing CLOS.
Also, PACKAGE should take the idea of LOCALE, allowing easier local package rename, and less symbol conflicts problem.
One of my main gripes with Common Lisp is its complexity. When I started learning Lisp, I expected to find a beautiful and elegant language, one without a need for syntax. The way C is almost an elegant assembly. Maybe I should have tried Scheme instead.
Common Lisp is anything but elegant. Instead of a regular syntax, you get all these "mini-languages", like the format syntax e.g.
(format t "~{~a~^, ~}" '(1 2 3))
; prints out
; 1, 2, 3
The loop macro is also quite powerful and complicated. e.g.
(loop :for i :from 1 :to 1000 :collect (* i i))
The fact that Common Lisp is a Lisp-2 makes treating functions as first-class as rather awkward. I could never remember when I needed to prepend a lambda with #'.
This article may be of interest http://xahlee.info/comp/Common_Lisp_quotations.html
I found much of the elegance I was expecting in Haskell. But maybe Scheme is a better lisp.
Also, Java/JVM suck ass, why would anyone willingly work with that whole mess?