Answers from 2017 Common Lisp experts | Hacker News Reader

Answers from 2017 Common Lisp experts

blog.teknik.io

104 pointsdrdre200143 comments

Wow, that is great, thanks for putting it together! I have been using CL since the mid 1980s, but cumulatively have just used the language for perhaps five years.

While I find myself using Python a lot because it wraps so many machine learning libraries and Java because of the ecosystem, there are other languages like CL, Ruby, and Haskell that all just make me feel good when I use them.

I really wish I could use CL more. It's just that in almost every domain, the new libraries are elsewhere. You could use lisp for scientific computing, for example, but it'd be crazy to do so because you'd be reinventing so much that's getting built elsewhere. Similarly with games, C++ just has a lot more libraries available (you could FFI I guess, but that's never as bulletproof).

Not sure lisp is better at any one thing than being a lisp, which is great but I want to be able to justify reaching for it. 10 years ago PG had me believing that there'd be a lisp revival one day, but despite getting more and more polished, there's so much going on in other ecosystems.

Sorry for the rant, I know the lisp community has been building great stuff for decades, lisp is great.

You don't have to reinvent a thing, just use some ffi (foreign function interface) implementation and call into whatever you want.

To add to this reply, interfacing in C with Common Lisp is pretty easy today. There are at least two portable libraries to do this.

Also, you can also interface with Java libraries on the JVM by using the ABCL lisp implementation.

>There are at least two portable libraries to do this.

What are they?

One is CFFI, widely known and well documented, with tutorial and tooling:

https://common-lisp.net/project/cffi/

The other was UFFI, older; but then there are others. In fact, there are a ton of other projects for doing FFI in Common Lisp!

http://www.cliki.net/FFI

And then you can also use the FFI functionalities provided by the particular Lisp implementation (i.e. SBCL, LispWorks, ABCL, etc.)

However, the use of a portable library like CFFI means that you can take your code that runs correctly in SBCL, and then run the very same code in CLISP (other Lisp implementation) with no change at all.

CFFI works for the following Lisp implementations or "compilers": ABCL, Allegro CL, Clasp, CLISP, Clozure CL, CMUCL, Corman CL, ECL, GCL, LispWorks, MCL, SBCL and the Scieneer CL.

That's a lot of implementations!

Sure is a lot. Thanks!

I'd guess one is cffi:

https://www.quicklisp.org/beta/UNOFFICIAL/docs/cffi/doc/Intr...

Maybe with the help of cl-autowrap:

https://github.com/rpav/cl-autowrap/blob/master/README.md

There are others... :

http://www.cliki.net/FFI

Interestingly there is a CFFI module in Python too. Maybe they got the idea from Lisp.

Um, i'm not sure. Python was designed with C compatibility as a main feature since the start, anyways.

The cffi module in python came from pypy (which broke the cpython c api) - but both the python ctypes module and cffi use libffi:

https://en.m.wikipedia.org/wiki/Libffi

Interesting info, thanks.

Are there performance implications using FFI ?

There can be, but they mostly have to do with moving things around in memory and garbage collection. An FFI has to marshall in-memory data structures from Lisp into the form C expects, and then move C's results back into Lisp's world. Sometimes your program will have to do this manually but the FFI usually takes care of simple cases like pointers and atomic types. Still, it's not cost-free because at minimum the removal and insertion of tags has to happen.

A related problem is word alignment; getting this right is important for passing data from Lisp to vector processors (and to a GPU I presume, but I haven't done that).

The other issue is GC: When the C function is running, it's important that Lisp's garbage collector not move the memory C is using. Some Lisp implementations do the laziest thing and just stop Lisp until C returns. Others are more sophisticated and keep C's heap separate -- but that requires more copying. Still others mark the memory C uses in a way that tells the GC "don't touch" but that can lead to pathological fragmentation in extreme cases.

The above makes it sound worse than it is. 99% of my use of a Lisp FFI has resulted in a performance increase because C is in general about twice as fast as CL for low-level stuff--which is one of the reasons to use an FFI in the first place.

> You could use lisp for scientific computing, for example, but it'd be crazy to do so because you'd be reinventing so much that's getting built elsewhere.

However, it has the potential to be just great for scientific computing. And it has been used for scientific computing extensively in the past.

First because of the fully interactive experience. Even a running program can be modified while running, in an easy and safe way.

Second, because of Lisp supporting easily, with no use of special libraries or special operators, the following: Arbitrary length numbers, complex numbers, fractions, integers, and arbitrary long integers. You work with them with all the built-in operators (+,-,/, etc) and all the built in functions like sine, cosine, etc. They all work correctly with the provided datatype.

Let me quote a part of the "History of Lisp" PDF by Gabriel:

---------------- QUOTE ------------------------

The S-1 was initially intended to be a fast signal processor. (...) Infuenced by S-1 Lisp, Common Lisp provides an expanded system of floating-point data types to accommodate such architectural variation. The inclusion of complex numbers in Common Lisp was also an inheritance from the S-1. This was something of a sticking point with Scott Fahlman. A running joke was an acceptance test for nascent Common Lisp implementations developed by Steele. It was in three parts.

First you type T; if it responds T, it passes part 1.

Second, you define the factorial function and then calculate

    (/ (factorial 1000) (factorial 999))

If it responds 1000, it passes part 2.

Third, you try

    (atanh -2)

If it returns a complex number, it passes; extra credit if it returns the correct complex number. (...) Gerald Sussman and his students (including Gerald Roylance and Matthew Halfant) became interested in numerical applications and in the use of Lisp to generate and transform numerical programs [Sussman, 1988; Roylance, 1988]. Sussman also spent a fair amount of time at MIT teaching Lisp to undergraduates. Sussman thought it was absolutely crazy to have to tell students that the quotient of 10.0 and 4.0 was 2.5 but the quotient of 10 and 4 was 2. Of course, nearly all other programming languages have the same problem (Pascal [Jensen, 1974] and its derivatives being notable exceptions), but that is no excuse; Lisp aspires to better things, and centuries of mathematical precedent should outweigh the few decades of temporary aberration in the field of computers. At Sussman's urging, the / function was defined to return rationals when necessary, so

    (/ 10 4)

in Common Lisp produces 5/2. (This was not considered a radical change to the language. Rational numbers were already in use in symbolic algebra systems. The developers of Common Lisp were simply integrating into the language functionality frequently required by their clients, anyway.)

-----------------------------------------------

I think any programmer that has had to work with huge numbers, complex numbers, floats and integers in C or C++ will appreciate the above information.

> Similarly with games, C++ just has a lot more libraries available (you could FFI I guess, but that's never as bulletproof).

True, however it can be done in Lisp as well. Some commercial games were written in the past in Lisp. Some famous and frankly groundbreaking games of the past, like King's Quest, Space Quest, Police Quest and Leisure Suit Larry, were written in an special interpreter (AGI) which was basically... a minimal Lisp.

Currently, on the Lisp reddit there is a long series of tutorials on this topic, called "Pushing Pixels with Lisp".

I have used Python, Java and C# in commercial projects for years, however i've started in CL some months ago and from my experience, it is much more suitable to general purpose development, including, definitively, commercial software development.

There have been voices that lament the supposed lack of libraries in CL, however i've found libraries for most of the things I would need anyways on a project, like ORMs, database access, compression, encryption, communication, etc.

There are also other voices that are scared by the lack of static typing, however CL is very strongly typed (it will strongly enforce types at runtime), and popular compilers like SBCL will accept type declarations that will also do many static-time type checks, plus these declarations will also greatly speed up the code.

Lots of interesting stuff in here.

Feel like some people could probably say things with a bit less sass though... Replying to "I have a hard time finding good resources" with "everything is terrible" isn't super useful

The teknik.io gang is not known to be lacking in sass.

Who's the gang you're speaking of? I'd like to get to know them. The aforementioned quote is AFAIR from Reddit, not Teknik, so the credit goes there; you can trace the answers to individual comments inside the Reddit thread linked in the article.

Personally, I use teknik.io as a blogging service and Git hosting.

Can you reach out to me via email? I don't necessarily want to post their haunts in public without their permission.

That's a lot of Lisp experts

It's a play on the title of the original blogpost. Quoting myself from there:

It's somewhat baffling to say that "experts", whoever they are, have gathered to respond to this question - I simply assume that the people who have voiced their opinion on the topic - regardless of who they are - know what they are talking of, and therefore their comments should be, and have been, included here.

But, on the other hand, correct, Lisp ain't dead yet. At least not any deader than the usual.

It's a year. From the "Questions" article:

> (...) and I have some questions for the Common Lisp experts out there in 2017.

That's exactly what I was thinking.

I really don't get the lispm's answer about cl21. What are the hidden treasures of the genuine Common Lisp we are ignorant of?

In my humble opinion Common Lisp is the classic example of the kitchen sink syndrome, where many pieces taken from various dialects were put together in a hurry. The results is not that bad (like, say, C++) but we know, arguably, a much more refined versions of Lisp, such as the dialect from Symbolics and Arc.

So, honestly, what are these hidden gems?

cl21, it seems, was an attempt to unify some syntactic forms rather than re-define what a Common Lisp is or should be. In that respect it is very remarkable effort. It is also a package - don't use if don't like.

The fair point that Common Lisp is a truly multi-paradigm language, so it includes mutating primitives alongside with "pure" functions and supports lexical and dynamic scooping, is rather difficult to grasp, but there is a lot of possibilities at the level of syntactic forms and embedded specialized DSLs, which, arguably, is what makes a Lisp Lisp.

It is never too much of embedded DSLs and syntactic sugar.

> What are the hidden treasures of the genuine Common Lisp we are ignorant of?

Actually Common Lisp is in many places a very well documented and designed language. Other languages have copied its numeric tower, its macros, its object-system, its error handling, ...

>In my humble opinion Common Lisp is the classic example of the kitchen sink syndrome, where many pieces taken from various dialects were put together in a hurry

The 'hurry' took place from 1981 (when the work on Common Lisp started) until 1994 when the ANSI CL standard finally was published. 13 years where hundred+ persons helped to design the language and provided comments, improvements, designs, prototypical implementations, alternative designs, ...

Also the 'various dialects' were mainly only Maclisp successors and Common Lisp was designed to be successor to them, not a summary of various dialects.

CL21 has more problems than lines of code, including security problems. As an experiment, fine. As a library that should be used? Please don't.

Edit: this thread gives a bit more detail discussing a COERCE method of CL21:

https://www.reddit.com/r/lisp/comments/6snw5d/questions_for_...

I think that a future Common Lisp could profitably default to lower-case instead of upper-case, and unify argument order. It probably shouldn't default to generic functions due to the performance implications. It could more-fully specify pathnames for current platforms. It could specify something like Go's channels & goroutines.

None of that's probably worth the bother of a new spec.

CL21 was, as you note, an interesting experiment, but I wouldn't run with it.

I remember reading the Symbolics manuals from bitsavers and how I have been impressed by clarity and unity of the language, and how they incorporated CL into Zetalisp just by moving their own stuff into packages.

At least in the documentation everything looks wonderful. Compared to Symbolics CL looks like a mess at least from reading the books.

> Compared to Symbolics CL looks like a mess at least from reading the books.

Which books, may I know?

The Common Lisp Hyperspec is one of the best reference manuals for a programming language, ever. But it is not a tutorial nor a how-to guide!

Try "Pratical Common Lisp". Or "Land of Lisp" if you want to have lots of fun.

simple but refined, guaranteed to blow your mind...

https://youtu.be/HM1Zb3xmvMc

> Which books

Symbolics Common Lisp - Language Concepts (volume 2A)

It also covers parts of Zetalisp.

I am also big fan of pg's On Lisp and ANSI Common Lisp.

That does not make sense to me. Common Lisp provided lots of improvements over Zetalisp.

The major goal of Common Lisp was commonality, and that means incorporating forms from various dialects into one and to maintain compatibility, like you said. That goal has been accomplished.

I think that carefully designed dialects with emphasis on right principles and attention to details leading by people like David Moon would be aesthetically better than a good kitchen sink. It is no coincidence that Common Lisp took most of stuff from Zetalisp, according to CLtL2.

I think it is a general heuristic that a small group of disciplined devoties would produce better artifact than an vast assembly of... general public or passionate and productive but ignorant individual. Think of Python3 vs Ruby, Go vs early C++ or Scheme vs CL. Attention to details and perfectionism works - look at Haskell syntax and prelude. (All this just to illustrate validity of my heuristic).

> The major goal of Common Lisp was commonality, and that means incorporating forms from various dialects into one and to maintain compatibility, like you said. That goal has been accomplished.

That's misleading. The 'various dialects' were mostly three very similar successor implementations of Maclisp.

Common Lisp was designed as a single successor language to Maclisp. There were successors of Maclisp for different computers under development: NIL (on a super computer and the VAX), Spice Lisp (for a new breed of workstations), Lisp Machine Lisp (developed for MIT's Lisp Machine systems), ... We are talking about new implementations of basically very similar languages, with Lisp Machine Lisp as the major influence.

DARPA wanted that these and future successor implementations stayed compatible. Not every Lisp application for the military should come with its own incompatible implementation of the language. NIL then looked a lot like CL. Spice Lisp evolved into CMUCL. Lisp Machines then got a CL implementation integrated with Lisp Machine Lisp.

From CLTL:

> Common Lisp originated in an attempt to focus the work of several implementation groups, each of which was constructing successor implementations of MacLisp for different computers.

Common Lisp was not designed to be compatible with other Lisp dialects like Interlisp, Scheme, Standard Lisp, ...

You are right - Common Dialect of MacLisp Successors. Thank you!

> In my humble opinion Common Lisp is the classic example of the kitchen sink syndrome, where many pieces taken from various dialects were put together in a hurry.

Can you give a particular example?

I have read many times about people speaking about the amount of "cruft" that was left into Common Lisp, but frankly i've yet to find any cruft. Some people use as an example the fact that there is "CAR " and "CDR", but in this particular topic:

1) you could use "first" and "rest" if you wnat, instead of "car" and "cdr" 2) Using "car" and "cdr" is good because it shows that you are explicitely wanting to modify cons cells; 3) "car" and "cdr" is to Lisp what "*" is to C; that is, part of the charm!!

Its mostly about inconsistent naming, order of arguments and standard idioms. Obviously most of these are historical artifacts, like nconc and friends or the primitives for working with hash-tables.

Car and cdr are small miracles because they give us of caddr or caadr and friends. It is beautiful accident which should be appreciated and preserved.

> Its mostly about inconsistent naming, order of arguments

Do you have some examples of inconsistent order of arguments? I'm not really doubting you, I just can't think of any examples.

One of the goals of Common Lisp was backwarts compatibility. The stuff from McCarthy's Lisp is mostly present in Maclisp, Zetalisp and Common Lisp. That's why they are Lisp languages. Every language existing for a longer period of time accumulates different design approaches. A language purist may complain, but a programmer will get his stuff ported and will get work done.

Author of the compilation here.

Sweet lords, I have made the front page of the Hacker News for the first time of my life! I had no idea that this would generate so much traction, especially as my own submission at https://news.ycombinator.com/item?id=15012678 went across HN rather unnoticed.

It's somewhat hit-and-miss which submissions make the front page. A submission has only a few minutes to get several upvotes before it's pushed off the front "new" page, after which it's unlikely to recover.

Since you're here I'll add a little to my previous comment [0], which was very short because I didn't think many people would see it.

I don't advocate completely pure functional programming as one would do in Haskell, but I do think there are lots of places in most code where the functional style would be clearer and easier to understand and debug than the fully imperative style more common in Lisp. That's why I created FSet [1], a functional collections library. It has been my observation that having high-quality functional collections is often the most important missing piece to make it possible to write more code functionally.

[0] https://news.ycombinator.com/item?id=15077088

[1] https://github.com/slburson/fset

> It's not some axiom of the universe that "thou shalt use GitHub

Love it!

Alternate Link for people who can't open the URL due to it being marked as "malicious" by some software:

https://gist.github.com/phoe/d93f968f22bbcc87070cdc583176202...

• Are there any functional programming advocates in the Common Lisp open source community?

Yo! https://github.com/slburson/fset

Reply on news.ycombinator.com