Steve Yegge: Lisp is not an acceptable Lisp. (2006)
steve-yegge.blogspot.com
steve-yegge.blogspot.com
People who actually use Lisp (opposed to people who only write about it) have no problem that there are extensions to the standard. The whole point of Lisp is that it is easily extended in many ways: the programmable programming language. Sure it brings a bunch of problems, but that's what it is and why people use it.
The stuff he writes about CLOS is mostly wrong. CLOS is not implemented as a bunch of macros. CLOS is actually a three level architecture: the core is written object-oriented in CLOS itself, on top is a functional interface and for user convenience there is a bunch of macros for easy definition of classes, generic functions, methods and some other language objects.
Then, macros are no problem. They are a feature. No, 'hygienic' is fine, but not needed. No they don't need to be rethought. They do a useful job like they are.
No, Lisp will never be massively successful (because it is a bit too complicated, too dynamic, too powerful for many users) and bending the language trying to make it 'massively successful' won't improve anything. No, I don't want a dictator. Lisp has choice, various alternatives and a more democratic approach.
Lisp is still a good choice for computing with symbols. If you don't need that, then you might not need Lisp. But that's not a reason that Lisp should change. Should a helicopter be changed in design because someone does not use it, but drives a car? Should the helicopter be changed, because more people drive cars, getting a drivers license for cars is easier than getting a pilot's license for helicopters? Lisp is different, for a reason and this difference will ensure its survival for many years. We have now fifty years that the basic ideas have proven to be useful and I don't see that this will change in the coming years.
This needs to be repeated more often - quality isn't measured by popularity. It's obvious when we talk about food establishments (McDonald's being the equivalent of Java), but for some reason, the logic is completely lost on people when we start talking about programming languages.
No, the quality of things is not measured necessarily by the amount of uneducated users.
Therefore inexperienced cooks should prepare food with butter knives?
Lisp has a cost for useage. Higher learning curve, Different way of thinking about patterns. Harder to use correctly at first. It's worth it for the "good cook" to pay the cost. Maybe not for the everyone else.
(as an aside I love my expensive set of high quality knives. and always feel handicapped when I have to cook without them. But they did cost a small fortune)
I was at the jQuery conference this past weekend, and a lot of the functional programming concepts would be way easier for people to understand if people saw them in Scheme first.
Is this true, or is it something you want to be true. I would say its a disservice to the language which in many ways was better than its peers and has since been imitated a lot (VMs, Javadoc, Parts of the platform library).
The tools in terms of breadth and quality available for java (IDEs, testing, decompiling, AOP ... etc.) are years ahead of most other platforms. (Excluding C#, what else...?).
A large part of this is due to the simplicity of the language (and the stability). Is java worse if you are developping in Vim, no doubt, but this is a strawman. With the tools vs. python|ruby|lisp is a different proposition entirely.
Lisp is the antithesis to this: it is a language laboratory - the programmable programming language. The programmer needs to develop skills to understand when and how to change the language. Changing and extending the language is a core mechanism. This makes the language unstable and puts a lot of linguistic power into the programmer. With Lisp you need to trust the programmer and the programmer has to learn how to justify the trust (by learning to effectively use macros, by learning when and when not to use EVAL, ...).
The Lisp programmer has to learn how to program languages and the concepts he may want to use for that.
No, that's just a question of marketing and man power.
A large part of this is due to the simplicity of the language
I was with you until that part - maybe the abstraction Java programmers deal with, sure. But the question "what is my computer doing?" when you run a Java program is well beyond the vast majority of Java developers, partially because it's so complicated. This is a Very Bad Thing.
Java:Starbucks::C#:Coffee Bean & Tea Leaf
Java:Starbucks::Scala:Peets
Java:Starbucks::Groovy:Mystic Mocha
Java:Starbucks::Algol60:Vesuvio Café
I have a hard time buying your explanation, since the scripting languages of today can be described as lisp subsets.
I am a fan of Perl and enjoy my life. The other usual scripting languages are quite similar, in most ways.
Still, I'd love to get a job using Common Lisp, which I haven't really touched since I studied. Why can't I do that?
Edit: I don't mean to whine, it is just that there is something wrong here. Lisp should be a, well, "contender".
Because, in the words of Drew Crampsie (paraphrased): Lispers do not look for Lisp jobs. They look for work to do and do it in Lisp.
Edit: Also, I believe well-established Lisp companies (such as ITA or Clozure) usually look for experienced Lispers. If you studied it in school, what makes you a good candidate for these companies?
Lots of the standard scripting jobs out there could be done at least as well with lisp, so why aren't many of them? (For instance, there has been good interpreting and compiling environments for Lisp since the 1980s? 1970s?)
(I didn't have any Common Lisp courses, I studied another lisp. I learned CL by myself.)
Edit: A bit of syntax.
Is Lisp popular enough? Hard to say, but people certainly seem to complain about the lack of popularity.
At least that was my experience with Scheme, I don't know (but suspect from all the whinging) about the CL side of it.
Which is one of the big advantages of Clojure of course, it comes with a lot of batteries.
Implementations use both non-portable and portable libraries. There are also several libraries that span several implementations.
Also the implementations serve different purposes and may use widely different implementation strategies - so not all libraries might apply to a certain implementation.
The choice is there for a reason. Some libraries might start on some implementation. If it is good, then it will likely be ported (if necessary) to other implementations.
Here's something I wrote about the concept a while back:
http://journal.dedasys.com/2006/02/18/maximizers-satisficers...
As Steve's article suggests, I don't think lots of more or less equal implementations is that great in the long term. IMO, having one main one and some experiments floating around works better.
I let you have your freedom to choose a scripting language.
Lisp is a family of different languages and implementations. Like 'scripting languages': python, rebol, ruby, javascript, vbscript, ... - similar problem there: why not have one main language? why these all these scripting languages which share almost NO code?
In Common Lisp I have a core language and can share lots of code between different implementations. ECL compiles to C and byte code and can be embedded. SBCL uses an optimizing compiler to machine code. CLISP uses a byte code engine written in C. ABCL is compiles to the JVM. LispWorks and Allegro CL define very large languages with lots of bells and whistles. Each of these implementations have been developed for different usages scenarios. CLISP is fine for scripting, LispWorks is better for the development of portable applications with a GUI.
There are Common Lisp implementations that are more popular than others. For free implementations SBCL is the most popular. But there is competition. CCL is also very good. Some implementations are less popular, but they survive because they use different implementation technologies. The commercial implementations like Allegro CL and LispWorks are also widely used, and provide features that the other implementations don't provide or don't provide as complete. LispWorks for example has a cross platform GUI toolkit that is used to implement its cross platform development environment. One can develop sophisticated GUI applications that can be compiled for Windows, Unix, Linux, FreeBSD and Mac OS X without changes.
Dan Weinreb gives an overview: http://common-lisp.net/~dlw/LispSurvey.html .
Ruby for example does not give me these choices. The main implementation is widely used, but lacks much of what the advanced Lisp implementations offer - which enables me to write a different class of applications in Lisp.
But Lisp was not designed to script applications. It was designed to be a language that can compute with symbols. This applied to Lisp itself lead to meta circular definitions, macros, meta level architectures like CLOS, and more. It led to the first self hosting compiler in the earlz 60s and ever since Lisp implementations have been written largely in itself.
It was designed to easily implement all kinds of notations and evaluation mechanism. Originally for example to implement symbolic mathematics and logic reasoners.
If the subset of Lisp that allows scripting is fine for you, that's great. Why complicate things?
Well, the point was that I don't see why subsets have taken over, it's as if PHP grew larger than the rest of the scripting languages for web progr... Oh, wait. :-)
But seriously, if you take the hit of a GC (which most everything do, today) then you might as well use the nicest system around. I am happy with my present tools, but a nice lisp variant might seriously tempt me.
Sigh, maybe I should just look for different work or do a startup with really hard problems.
You don't need really hard problems before the benefits of a language with macros become obvious. SQL-driven web apps are a lot easier to write if you have macros, for example: http://brl.codesimply.net/brl_4.html#SEC31
One word: Guile.
'Lisp' is a family of languages and the basic design was done by John McCarthy starting in 1958. He did not design a scripting language.
I was talking about 'Lisp', not Guile.
(I'm trying to hunt down that talk, but not having much luck.)
If you think that he is a pretty smart guy, then I would say that there are many Lisp implementations that are way ahead in terms of implementation technology - Matz's Ruby implementation is in the Lisp world several decades behind the state of the art. Ruby is one of the slowest scripting languages and has a very much ad-hoc design. It does useful things for many people, and that counts. But from the technology it is Ruby that is a huge pain in the ass. Just see how people struggle to write a decent compiler for it. Instead it is just another simple interpreter with a C core for the interpreter implementation and parts of the library. Ruby is fast where you stay in with the library functions that are implemented in C - otherwise it is a pain.
The better Lisp systems are also written in Lisp and their performance is sometimes 10 to 100 times of what Ruby offers - while Lisp offers also lots of dynamic features and the code as data paradigm.
He sounds confused.
Started with BASIC: 'I could Make Intelligence'. ???
Then he found Lisp in a magazine article and 'fell in love with it'. 'BASIC aristocracy' vs. 'Lisp democracy'???
He liked Lisp from a book, but 'unfortunately he did not have a computer that could run it'. In University he used Emacs Lisp and it did not make him 'happy'. Parentheses? No? He had no problems with it. Macros? Partially. 'Smart people just underestimate oridinarity (sic) to use Lisp'. 'We are too ordinary to use Lisp'. Aristocracy. He is happy with aristocracy, as long as he has power.
The whole video is so painful...
(He's also not a native English speaker, so i cut him some slack on that front.)
Matz designed Ruby for so-called 'ordinary' people with him to do the language design. I can tell you for sure that I neither like his design nor do I want him as the language designer. In the Lisp world the implementors role is to empower the user, not make them dependent. I need a new control structure? I write one myself. I don't have to wait for some benevolent dictator, who probably does not get basic designs right - just look at the confusion of blocks, lambdas, etc. in Ruby.
(also there is no excuse for putting confused stuff on slides, even when he is not a native speaker - lots of people (including me) are no native English speakers and I haven't sensed this amount of confusion Matz shows)
That Lisp is too complicated for some people is not news, still Emacs (which he tried) now comes with a standard library of a million lines of Emacs Lisp code for all kinds of editor extensions. Emacs Lisp code that has been written by very different people and obviously a lot of them learned enough Lisp to write sophisticated applications based on Emacs (like GNUS, calc, the various mail modes, org mode, ...).
Lisp is also not only great technically and theoretically, but some people found it practically. There weren't ten maintained Common Lisp implementations if there were no users for it. For example the talks at the Lisp Meeting last sunday here in Hamburg described several applications in Lisp: an airline reservation system (developed by a team of hundred people, fifty of them Lisp programmers), a visual reactive programming system, an aircraft analysis tool, a reasoner for the semantic web, a visual simulation system, machine learning applications and some more.
I pay my bills writing Common Lisp code, full time. The fact that you're not an employed Lisper is your own fault, IMO, it takes guts to look someone in the eye and tell them their investment, in both time and money, is safe with you and your fringe programming language. It's allot of work, a lot of responsibility, and most certainly tons of fun.
Try it.
P.S. I am both lead developer and "CTO"; I have a team of web developers who consume my backend and architecture.
Not just fringe language, but fringe anything.
The only known amongst a see of unknowns will be you.
For that matter, mainstream popularity of anything has always had a good dose of randomness in it (I bet you can model it with some sort of free energy / entropy equation). For example, it's hard to break down exactly why the No.1 hit song is No.1, or that No.7 song is permanently No.7. Usually it has more to do with the politics of the record company than the quality of the song itself.
We are also being a bit pessimistic here. Lisp is not un-popular. It just hasn't reach the critical mass yet. And it seems to be going to that direction with unraveling of Clojure.
I agree with you that CLOS and macros are not problems. Common Lisp is not primarily an object oriented language, and that's OK. That it happens to have an extremely powerful object system is a nice bonus. If you want a ground up object system, use Smalltalk or Ruby and have fun.
The rant against macros was incoherent to me. Editor support for macros is imperfect so...macros suck...or something? Macros remain the distinguishing feature of Lisp, in that no other language allows you to transform your source code so seamlessly. I'm not sure there's a compelling reason to use any Lisp over other languages if macros are eliminated.
Imagine the standard, cross-platform GUI you'd get in 1994 (the time of the ANSI spec approval).
The only thing unique to Lispy languages (and really not at all that hard to understand) is the language syntax and the way the compilation rules are late-bound. That is, the compiler is intended to be extended by the user. Roughly, you're intended to be able to say "unless (...)" will translate to "if (!...)".
The rest of the design decisions made aren't even that outlandish: you still have text which is parsed and either interpreted or compiled into machine code, which is run. More foreign (and interesting), I think, is the research being done at Alan Kay's lab now:
This is what Rich Hickey did with Clojure, more or less. So this was a pretty accurate prediction.
My impression is that you [haven't used] neither Common Lisp nor Scheme on a regular basis, and/or in a considerably sized software project. Most of the issues that you mention are "academic", in the sense that they are theoretical problems which tend not to bite you in practice
However, he is not indebted to us to keep writing so my opinion doesn't really count in that regard.
Thanks, guys!
Of course, if you feel that the post is obsolete due to progress made in Lisp since the post was written, I'm extremely interested in hearing how things have changed.
Given that Clojure is not the only Lisp, I feel the article is still relevant although comments like yours and another poster pointing out the value of Clojure add value for readers.
I've been pondering Clojure as "Lisp with Java libs! Hooray!", but if there are notable flaws in its Lispiness I'd like to hear about them.
See http://www.loper-os.org/?p=42 for more.
See, this wouldn't bother me so much if it assumed the user knew the JVM instruction set and built up abstractions around it. But no, that's not what it does. It creates another "language", meaning "obscuring abstraction" that isn't friendly to inspection.
However, it's a step back in The Philosophy of Lisp, because it doesn't allow user-defined reader macros. First let me explain what I see the philosophy of Lisp - it is the late-binding of all things. Late-binding, in this sense, means that the system by which you make the computer do the things you want it to (we usually call them "languages") makes as few decisions as possible, and lets the user overwrite and extend them. For example, the Python mailing list every once in awhile has an active debate if anaphoric `aif' should be allowed in the language [0]. This would mean adding a special symbol `it', such that:
aif expensive_function_call():
foo(it)
it = expensive_function_call()
if it:
foo(it)
However, there are two kinds of macros Lisp supports for the same reason: "syntax macros" which let you late-define how syntax is interpreted, and "reader macros", which let you define how the parser interprets your syntax (the Lisp parser is called the reader). For example, the reader of Clojure translates '(foo) into (quote foo), #{:x} into (hash-set :x), [x y z] into (vector x y z), etc.
However, the rules by which the reader translates those "special" deviations from normal s-expression syntax into s-expressions is closed off from modification. This goes against The Lisp Philosophy, and is not so in other Lisps [2]. The only case for this I've heard (in Stuart Halloway's "Programming Clojure") is that this would allow people to dilute the language into something which is no longer Clojure. This explicit early binding is a step back in what I see as the Philosophy of Lisp because brings us back to the religious tribalism of "this is Clojure, and the language decisions that were made are good decisions, and if you don't like them go fuck yourself".
0. http://en.wikipedia.org/wiki/Anaphora_%28linguistics%29
1. The analogy I like best here is to mathematics. Any mathematician can create his own definitions to explain his ideas (as they usually do), and yet, mathematics doesn't break up into dialects that nobody can understand. Instead, mathematical language evolves as people participate - if you have a pet definition you like (maybe one you thought up all on your own!) you don't need to convince Guido or some council of language makers to be able to use it. You use it, and if your paper is worth reading, maybe you can convince other people to accept it. That way, mathematics as a language evolves, not by design, but by social interaction. The formal languages of computation can (and should) do the same.
The no reader macro convention basically boils down to making it easier to design a language.
If you want an acceptable Lisp, you effectively have to combine the best features of all Lisps. This is surprisingly difficult to do and ensure portability and stability. So your comments about "direction of Java" are puzzling, as it guarantees portability and stability to the extent the JVM is. Really the only downside is compatibility with type erasure, which is unfortunate but hey.
I'm not sure how not adding a reader macro mechanism makes the design easier. Actually the point of read macros is to enable incremental addition/change of syntactic elements of s-expressions.
I wish, though, that Clojure had a versioning system similar to Curl or even .NET's module system, or even OSGi/Jigsaw. Syntactically, I'm thinking of something like Curl.
linky: http://piumarta.com/software/cola/
Thanks! I was thinking of dropping that into the post as an example of the late-binding philosophy going into the future, but I've been linking it quite a bit lately, and decided to go without it for once.
In my view of it, Piumarta's COLA is very strongly "of The Lisp Philosophy". As a matter of fact, I would call it the future of Lisp (the philosophy) :)
When things are truly late-bound, it makes as little sense to split computational instructions into these "language" silos. Allowing users (and creating tools) to modify syntax is fundamental to breaking this tribal idiocy around "languages" as atomic sets of programming system design decisions.
I can't understand you. Can you rephrase?
@Allowing users (and creating tools) to modify syntax is fundamental to breaking this tribal idiocy around "languages" as atomic sets of programming system design decisions.
It's far from tribal idiocy, and has more to do with modularity and substitutability. Most programmers do not realize what heights of engineering .NET|Mono, the JVM, Smalltalk, Symbolics Genera all are. .NET's MSIL, in particular, is the most modular stack-based assembly language I know of... especially version 2. It's pretty impressive once you realize you can specify modules at the assembly level. Granted, even in the '90s people were designing typed assembly languages, like TAL/T, which supported run-time code generation and could be certified for safety at compile time, link time and run time.
Really, I think talking about "not an acceptable Lisp" in broad strokes undermines attention to good engineering and striving to be great engineers.
> I can't understand you. Can you rephrase?
Oops, think-o, that should have said "it makes as little sense to split computational abstractions into these language silos".
To clarify: there are "languages" like Haskell/Ruby/Python/Java/C that make an enormous number of decisions about the kinds of abstractions we use to describe what it is a computer is to do. A "language" is a set of decisions about syntax, type systems, functional or object-oriented abstractions, memory allocation and garbage collection, call stack traversal. If one decides to compile to a VM instead of machine code, that introduces another set of variables - from JIT compilation techniques to the kind of VM specifications you publish (do you publish a reference VM like Squeak, or a bunch of papers and tests like the JVM). "The Java language" is a set of hundreds of those kinds of decisions. So is "the Python language". Comparing the two is really difficult, because for almost any problem, some subset of Java's decisions will be better, while another subset of Python's decisions will be better. But these sets of decisions aren't atomic (they can be split!).
Every time you late-bind a decisions to the user, it removes a dimension from the decision space that the "language" lives in. It expands the power of the language to span that entire dimension. Hence, the most powerful language is one that makes the most irreversible decisions.
talking [...] in broad strokes undermines attention to good engineering and striving to be great engineers
That sounds right, and I should avoid it if it's needless. However, it's also important to always try to unify concepts to simplify our models of things. The important thing is to make sure the simpler model is just as accurate.
"Accessible Language-Based Environments of Recursive Theories (a white paper advocating widespread unreasonable behavior)". http://www.vpri.org/pdf/rn2006001a_colaswp.pdf.
The hallmark of COLA is recursive design, which sort of makes it more like Smalltalk than Lisp (but this is a fruitless point, and the key is to encourage you to dig deeper into what COLA is rather than what "it's like...").
The hallmark of COLA is recursive design, which sort of makes it more like Smalltalk than Lisp
Late binding is certainly a big part of the smalltalk philosophy as well, and recursive design is a great way of implementing late binding. A system uses the same "user interface" internally, it certainly makes it easier to let people modify the internals.
The big difference I see between Smalltalk and Lisp (correct me if I'm wrong) is that Lisp systems tend to use compile-time abstractions Lisp->machine code->execution, while Smalltalkers from early on started thinking about interpretation and VM's - using abstractions which require run-time data to make decisions.
Model-driven architecture weenies like me will tell you the biggest difference is methodological, and that real-time object-oriented systems engineering has its roots in most of Kay's ideas.
I'm not sure what you mean by the following:
@Lisp systems tend to use compile-time abstractions Lisp->machine code->execution
I don't think so. The whole reason I enjoy Lisp is for how it has inspired me to do streaming models of compilation. When you're updating things dynamically, the most important thing is to have a logical object model - starting with getting your "(UML) package diagram" correct. Otherwise you end up with a large system that requires the entire system to be locked up for minutes while you do the upgrade, because different parts of the system depend too much on physical model details. That's why you separate message from method. So you can do this in any language, some easier than others.
As other mentioned, maybe Clojure was a big thing that happened to the Lisp world, but still, Clojure is a very small part of the Lisp world.
I did append the (2006) just for the sake of information. I hope no one will decide to not read it based on that.
You have to distinguish between the real Lisp world, in which people work on Lisp programs and systems to solve real problems, and the Lisp hype-world, which is a parallel universe consisting of blog posts, arguments, and pronouncements generated by people who are (mostly) doing no such thing, including the OP. Clojure is now a big part of this parallel universe.
The biggest plus: the event was completely free of random whining.
It might be worth mentioning that mailing lists for specific Lisp implementations and libraries tend to have high-quality and largely respectful discussions.