Lisp's reputation is so bad that many people don't even take a look at Lisp

I've been a Lisp hacker for years -- Symbolics Lisp Machine in my bedroom, coded in at least a half dozen different Lisp dialects -- but I ended up using Smalltalk for my startup. Treating Lisp's "rehabilitation" as a mere branding exercise shows just how screwed the Lisp community still is, and is symptomatic of the underlying issue. (By "Lisp", I mostly refer to Common Lisp.)

Lisp's problem is a meta-problem. The community is actively resistant to advancement, change, and evolution in the language. Attempts to modify (or even clarify) aspects of the Common Lisp standard are tedious exercises in futility, but the standardisation straitjacket is still seen as a good thing. A bit strangely, this resistance to change existing standards is accompanied by a slowness to standardise (even informally) anything that isn't already. The end result is that there's still no reliably portable way to write networking code in CL.

And this is why Lisp is unpopular.

The stasis is sad and ironic, given Lisp's hackerdom beginnings. Rather than wonder why, in 2009, so few are using Lisp, we should be amazed that the language is still pretty cool two decades after the innovation stopped.

I hope the Clojure guys change things. They seem to have the right approach.

Footnote: I was going to be a member of the CLRFI group (roughly equivalent to PEP or SRFI) that got some interest at ILC 2005, and which was going to try to create some sort of forward momentum in the community. It frizzled out, of course, and now http://clrfi.alu.org/ appears to redirect to the ALU homepage.

The code snippet from CLSQL looks horrendous:

  ;; select the user with ID 1 and set his username column to joe
  (let ((user (first (select 'user :where [= [id] 1] :flatp t :refresh t))))
    (setf (username user) "joe")
    (update-records-from-instance user))

  User.find(1).update_attribute(:username, "joe")

  (update-attribute (find *User* 1) 'username "joe")

Thus the situation is inescapable. You need tons of libraries to be productive. You need tons of programmers to write these libraries. You need an easy learning curve and appealing features to get these programmers. The bottom line is that you need the bad coders as well.

As a beginner, I can safely say that it took a while before I finally got reasonably productive in Lisp. I still consider it 'the best' programming language. But the learning curve is much steeper than Python.

Lisp will get nowhere without all the bad programmers. This may sound cynical, but the reason is that academically minded programmers won't spend time writing things like a download library for youtube videos, which many people find useful. These programmers are now all over Python.

So why is Lisp difficult to learn? The most severe in my opinion is assignment. People find it intuitive to make temporary variables and store intermediate results in these. This is done by 'let' in CL, but if the previous style of programming is assumed, 'let' is clumsy and inconvenient to use.

Another achilles heal I would like to mention is the lack of an editor with propper auto-completion. Yes, slime does this, but using emacs is too geeky for many programmers (remember you need these people) and this creates an additional stumbling block.

Yet another problem is the CL websites. Remember that many people are in a hurry. The Python website is ripe with snippets that lets you do stuff fast without understanding much. In contrst, CL gives you some of the most monotone autogenerated documentation sites I've ever seen. Again, fine for people who already know Lisp, tedious for people that want to try stuff out as beginners.

Faced with these realities I don't blame anyone who just want to get things done and run with Python. Unfortunately.

I didn't care too much about the syntax, a good editor fixes this without a problem and it was a good opportunity to look into emacs a bit (usually, I use vim). Surfing through the chapters of practical common lisp when pretty well and understanding was not exactly hard.

However after that, I was faced with confusion. There is a boatload of lisp dialects, each with a boatload of implementations and each implementation as another boatload of libraries that do something looking similar to what other implementations libraries do without being the same or compatible. How am I supposed to select any of these without having a clue? After a few days of back and forth, I decided to close the book lisp for me, because I could not find a way through this wall of selections -- at least not without investing a lot of time. Too much time for too little benefit over python.

HTH, Tetha

For example, Nu is one of the coolest languages out there that allows one to do quick and dirty "scripting language" hacks in Objective-C. Due to Lisp's simple structure, Nu was able to directly map Objective-C method call signatures into something that clearly looks like basic Lisp. It allows one to code in 2 languages, but the Objective-C part of the language actually looks very similar to normal ObjC. One of the biggest benefits of Nu is that writing unit tests in ObjC is no longer painful... the whole testing structure is as simple as typical ruby unit test cases.

Clojure is another one of those wonderful libraries that is a must-have for anyone doing normal java. Even basic string manipulation is extremely verbose and annoying in Java, but is probably too heavy to load a JRuby/Jython interpreter... I think that's where clojure helps fill the gap. I recently implemented an "equivalent" of a google-analytics tracker in Clojure and the final result was only around 200 lines of code (with a little bit of java here and there). It's quite nice to use clojure as a glue-language to make all your java libraries work together with little clutter as possible.

How's about conceptualizing the problem less as "People don't come to the light about how Lisp is not as problematic as they think it is" and more as "We need to explain better how people's lives are going to improve if they learn and use Lisp". (I know, I know, I may have just committed the sin of marketing. Please forgive me.)

I do not have a Lisp-sized hole in my life right now.

I won't even say that libraries are a real problem. Sure there is no CPAN, but still there are a lot of very good CL libraries around, and they are not really hard to install.

But:

1. The language is sophisticated. This is a "problem", in the sense that such a language takes time to study. It will never be popular. The Condition system and CLOS, for example, are way more complex and versatile than, say, the equivalent areas of Java. And then there are macros. Common Lisp has not been designed as a mainstream language for mainstream programmers. It is, I fear, an elite tool.

2. Certain kind of operations are inconvenient. Maybe when CL was designed, sockets, multithreading, text processing etc were not so popular. But now, it feels like CL has not really adapted well to the modern world. Having to install a compatibility library for using sockets is reminiscent of things like "trumpet winsock". Does not feel very cutting edge.

3. The literature is lacking. "Practical Common Lisp" is very good, but one book is not enough. For example, what are the best practices to organize large CL programs? No book tells it (maybe some obscure out-of-print text that I have not yet tracked?). What about programming idioms, or patterns as you prefer? Apart from an ancient presentation by peter norvig, nothing exist. And so on.

4. The ANSI standard is a big blessing and a big curse. It is a well-written and smart opus. But nowadays no one has the resources for a new standard; both in terms of money and focus. This means that ANSI CL seems an ancient pinnacle of beauty which will never be equalled and certainly can not be improved. (the perfection of ancient architecture, as seen by renaissance architects, comes to mind). This makes CL a splendid but dying civilization.

5. I have a feeling that the community is slowly dying as well. While in recent years many CL sites and blogs have appeared, comp.lang.lisp has lost many important contributors. Nor I can really blame them: while still a wonderful language, CL is nonetheless petrified, and I suppose one may became slightly bored after a decade or two of discussing the finer points of the Standard.

p.s. I've never written anything longer than a textbook exercise in scheme, so i don't know if you can actually write real stuff in this(or how hard it is) but playing with it is really fun, you should to it just for that.