Is Lisp still useful in today's world? (2011) | Hacker News Reader

Is Lisp still useful in today's world? (2011)

programmers.stackexchange.com

50 pointscpleppert59 comments

Learning to program in lisp (specifically reading Practical Common Lisp [http://www.gigamonkeys.com/book/]) has made me a better programmer in all other languages even if I don't really use Lisp. Even my friends/coworkers have noticed it when coding together. My style is forever improved.

Learning Lisp just seems to promote better ways of tackling programming problems that is transferable to other languages where it's not as encouraged naturally even if superior.

Completely agree that learning functional programming has improved my thinking and code. The one difficulty I continue to encounter is switching back and forth between recursive and iterative thinking. From a computational perspective they may be equivalent, but from a cognitive perspective they aren't (at least in my case). I don't know how many times, for example, I've been bitten by a stack overflow when jumping into Python from Haskell (or equivalently, jumping directly into the ST Monad because I want to iterate through some lists).

I've been using Common Lisp for two years now, I've written exactly ONE recursive function that wasn't an exercise or an example. Functional interfaces to functions with local state in loops? What exactly is wrong with that?

For example it is my opinion that this function is in no ways inferior to a recursive equivalent:

  (defun range (end &key (start 0) (step 1))
    (loop for i from start to end by step
      collect i))

As far as I know (which isn't too much), there's nothing wrong with that. But since monkeyfacebag mentioned the psychology of code writing, there definitely seem to be algorithms that lend themselves to recursion. At the very least, some algorithms are more intuitively grasped as recursive - and the same obviously holds for iterative.

yes, usually anything having to do with recursive structures, like threes, is a good fit for recursive algorithms. In fact, the function I wrote had to generate a directory structure out of a template: https://gist.github.com/pvlpenev/4760658 (and i still used loop :)

But keep in mind that some things, although recursively defined, are better computed in other ways, the most obvious example is a Fibonacci sequence, which if I really needed, I would precompute and stick in an array for constant time access or something like that(maybe memoize a recursive function).

Personally, I find that traversing data structures recursively is more straightforward than iterative approaches, at least for most of the data structures I deal with. This sort of thing, for example:

  (defun search (tree val)
     (cond
        ((null tree) nil)
        ((= (car tree) val) tree)
        ((< (car tree) val) (search (cadr tree) val))
        ((> (car tree) val) (search (cddr tree) val))
        )
     )

I write this sort of code all the time for more complicated structures; iterative solutions would involve keeping track of a bunch of local variables, which only makes the code more difficult to deal with.

The Little Schemer is the book that got me thinking about data structures recursively (starting with linked lists), and that was later furthered by SICP.

http://www.amazon.com/Little-Schemer-Daniel-P-Friedman/dp/02...

http://mitpress.mit.edu/sicp/full-text/book/book.html

I noticed my usage of recursive functions going up and becoming more natural to me after reading The Little Schemer. Check it out, it is a really nice book.

I suppose that most of the data structures which are naturally understandable in their recursive forms (like merge sort) would already be accessible via libraries.

Functional interfaces to functions with local state in loops

That's how I write Lisp too. It seems to me the nicest way to minimize overall complexity.

I remember a correspondence between Knuth and Dijkstra about a deep problem involving four (!) stacks in the recursive solution. One or the other of them stepped back and found an iterative solution that was much easier to understand.

However, whether or not your example shows something that is not inferior is likely a matter of opinion, not fact.

After reading up on lisp and associated topics (purity mainly) I rewrote a component in our C# code base and made it magnitudes of order more stable and atomic.

I hadn't written a line of lisp (although counted many brackets when trying to read along) but just studying it helped me think about programming problems a bit differently.

Yep! My joke is that "Practical Common Lisp" is the best Javascript book ever written ;)

"Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot."

-- Eric S. Raymond

If we're quoting that then we should also quote PG's objection to it: if Lisp makes you so much better a programmer, why wouldn't you want to keep using it?

Practical reasons: The language is great for enhancing your style but in practice it's been one of the hardest languages I've come across to make do real things. It interacts with the outside environment very poorly and more awkwardly. It's got multiple slightly incompatible implementations and it's library set is relatively light, green, wastefully overlapping and not very well maintained.

There's a reason Java won't go away and that's that it runs everywhere releatively easily and interfaces with stuff acceptably well.

Python is now doing a good job of this too.

Lisp has a looong way to go on this front and I do speak from some limited experience.

It's very pleasant if all you're doing is staying inside lisp and interacting only with your own code but as soon as it's asked to play nice with others it gets a lot less fun.

And staying inwardly focused, never looking out and never talking to anyone else ultimately means get very little real productive work done.

Please take a look at clojure. There are a lot of videos on youtube... there is a specific clojure channel. I don't recall the specific name, and work blocks youtube unfortunately.

Clojure runs on the JVM and has great Java interop, so you get the fun of lisp, and the convenience of all the Java libs.

True, and I recently did the coursera class on clojure. I'm definitely keeping it in the back of my mind for future projects.

I still personally find JVM languages slow memory hogs during development but that's another issue :P

It's definitely a good offering

ABCL, a common lisp running on the JVM is getting better and better by the release. It's java interop isn't quite as polished as clojures, but it is usable, and you get Common Lisp, which I like slightly better than clojure.

THe only thing that really distinguishes lisp these days (IMHO) is its macros. However, I think there are better languages out there to learn any particular paradigm-Python/Java for OO, Haskell/Erlang for functional, C for procedural, Ruby/Python for dynamic & duck typing, etc.

Lisp is great if you need a compiler tomorrow. If you have the time, though, I think that other languages are better when there's more time to invest.

EDIT: To actually answer the question, of course it's still relevant and useful, but perhaps other languages have caught up. It's still used in industry (Google uses it via ITA, IIRC).

Multi-methods are still scarce among languages with OO features, and it's a big hassle working around that shortcoming. There's probably more gems in CLOS that haven't made it out into the mainstream yet.

Multimethods and the condition system are 2 of my favorite things.

Genuine question here: when are multimethods useful in practice?

One of the less controversial uses of OO dispatch is to replace switch statements or chained ifs with a polymorphic function. Try to do this more than a few times to the same codebase, and you'll either wind up having to merge classes into weird hierarchies or use hacks like "the visitor pattern" to proceed.

The other big use of multimethods is when you have static-type function overloading (as in Java/C++) and decide you need to move the overload selection into runtime via the dynamic type of the arguments. This isn't possible in the general case without doing all the dispatch work yourself, but it's a trivial with multimethods.

It's not clear to me what Python the language offers over the Lisp family, other than improved readability for some class of users and translation to and from pseudocode (see Peter Norvig's comment in 2010: http://news.ycombinator.com/item?id=1803351 ). These aren't necessarily small advantages, but neither do they mitigate Lisp's power and flexibility. I do, however, think Python the community offers a lot of practical advantages to the new user over, say, any particular Lisp community.

A high quality reference implementation. This is, in fact, a trait shared with pretty much every other language to gain traction in the last 20 years. In Lisp, there is substantial fragmentation on how do things like GUI and Networking, and often no clear winner. SBCL seems to be the de-facto FOSS implantation, but Allegro is the leading commercial vendor.

"In Lisp, there is substantial fragmentation on how do things like GUI and Networking, and often no clear winner"

Is there a clear winner for GUI in Python? GTK, Qt, etc. Python also has quite a few networking libraries; networking in today's world is more than mere sockets.

"SBCL seems to be the de-facto FOSS implantation"

Meh, GNU Clisp is pretty relevant, and a lot of people are using Clozure. CMUCL also remains relevant, if only because of its useful extensions (SBCL is a fork, and dropped many extensions like Motif support).

I think what CL really needs is an update, a CLtL3 that adds support for modern features. Modern Lisps like Clojure and Racket do not have many of the constructs that make CL great (which is not to say that Clojure or Racket are not great languages, but they are missing some of the nicer CL features).

There are alternative's in the Python world, but they are not mutually exclusive.

Where-as in the lisp world stuff like the socket interface is DIFFERENT depending on the implementation. So you don't get to pick and choose.

Compatibility libraries exist for nearly every implementation detail like the one you describe. Threads and networking being incompatible was a valid concert 15 years ago.

It's still a layer of complication and headaches most languages just don't have.

> (which is not to say that Clojure or Racket are not great languages, but they are missing some of the nicer CL features).

Any examples? Just curious.

Well I suppose it really boils down to taste, but two things come to mind:

1. LOOP -- (loop for i in some-list maximizing i) is just the beginning of nice iterative constructs in CL.

2. Conditions -- continuations are a building block for this, but it's nice to not have to build it up yourself. Agents in Clojure are pretty good also, but not applicable for any exception (correct me if I am wrong, I am not as much of an expert in Clojure as in CL).

Again, just a matter of taste. Clojure and Racket certainly have advantages over CL, just by virtue of supporting more modern features (especially Clojure, which inherits all the features of the Java standard library).

CLOS and MOP, read-write consistency, restartable exceptions, reader macros, optional typing, many high-quality implementations(including several commercial ones that come with excellent support), high-quality standard document, probably still the fastest dynamic language around, superb introspection support, one of the best interactive work-flows, surpassed only by smalltalk, DECADES of being tried and used in every conceivable industry and application, from super computers, space probes and AI, to web apps, games and prototyping(you'd be surprised how much software was prototyped first in lisp).

AAAAAND macros too, I guess. They are nice.

</smuglispweeniecomment>

> commercial ones that come with excellent support

Maybe I'm just too small-time for it to matter but what, exactly, does this commercial support actually buy you? In the last 10 years of programming, I've only run into a handful of bugs in my tools. When dealing with OSS projects, they were already known and fixed in newer versions. When dealing with $BIG_DATABASE_COMPANY they told me to use a workaround and didn't fix the bug until the next major release of the software.

I don't have a problem paying for commercial software, I understand it supports development and improvement of the tools, but what exactly are you getting when you spend (tens or hundreds) of thousands of dollars on licencing costs for support?

Ideally, being able to call up the company and have the expectation that they will ship you a bugfix as soon as they can roll one together.

But that's pretty rare, IME/O.

I'm not a Lisp guy, but the amazing thing I see in Lisp that is missing from other languages is that due to the extremely simple syntax, code is data and data is code. That lets you do some amazing things (Cf. SICP).

On the other hand, isn't one lesson from modern computing that treating data as code is really dangerous? I have yet to see this addressed by Lisp advocates.

The way in which code is modified in Lisp is generally in the source code itself, and overtly in the run-time environment or binaries. The latter might be what you're alluding to.

Allowing code to be treated as data is a very powerful thing when it comes to programming. But of course, with more power there is more danger.

Define dangerous? Are you concerned with security? In which case, don't eval untrusted code, and you're fine.

Maintainability? If done right "code is data" could improve the maintainability, or decrease it drastically, just like any other feature. If you have 80 lines of comments for every 10 lines of code, you've decreased your maintainability, comments are dangerous by that logic.

Yes, by "dangerous" I mean prone to security exploits. I've got nothing against Lisp--I think it's a beautiful language, and I've dabbled in CL, Scheme, and Clojure--but whenever I hear about using data as code I get a warning tingle in the back of my head. One of the lessons of the recent Ruby YAML debacle is that an executable data format will get used for untrusted input eventually somewhere or other. (Edit: For example I can totally imagine the digital circuits simulator from SICP 3.3.4 saving/reading Lisp code as an input document.)

Maybe the power of Lisp's data-as-code is worth the risk. We don't stop using string concatenation because of SQL injection. It's just that I've never even read so much as a warning about this feature, and I'm curious what Lisp users have to say about it.

  eval("dangerous python code here")

As I pointed out, it is a problem in any language that has eval. The way you deal with it, is you never eval anything untrusted, which is actually extremely easy to do. I don't believe I've ever used eval ever.

In CL specifically, because the reader can evaluate code as well, every time you use the read function, you should know these two things: 1) never use it for user input. 2) bind read-eval to nil. You'll be fine, or more accurately, no worse than anybody else :)

On the other hand macros, and having a json style serialization format that is much more powerful and actually extensible is a big plus :)

Is it actually dangerous to use read for user input?

    (defun read-data ()
      (let ((data (let ((*read-eval* 'nil)) (read))))
        (mapcar
         (lambda (x) (format 't "~a: ~a~%" (car x) (cdr x)))
         data)))

Then I can

    (read-data)
    (("a" . 1) ("b" . 2) ("c" . (funcall (lambda () (princ "I ran code!")))))

And the following will print.

    a: 1
    b: 2
    c: (FUNCALL (LAMBDA () (PRINC I ran code!)))

But of course unless I eval the code, arbitrary code will not run (assuming I haven't forgotten some aspect of the reader).

"It's just that I've never even read so much as a warning about this feature"

...have you really looked? Here is a warning, in the section titled, "Reader Security:"

http://letoverlambda.com/index.cl/guest/chap4.html

> have you really looked?

To be fair, no. I'm not a practicing Lisp programmer; I just admire the language. But I've read a fair number of Lisp tutorials, as well as SICP, and I don't recall it being mentioned. I'm glad it's out there somewhere!

I don't buy your example from chapter 3 of SICP. You could just as easily serialize Python code and eval it. That would be bad practice in Python and it would be in Lisp.

The power of "code is/as data" means that Lisp macros can manipulate code as easily as other Lisp code manipulates data. It's not really about eval.

> You could just as easily serialize Python code and eval it.

That seems to be the consensus among all the people replying to me. :-) Fair enough. The difference I see is that lots of Lisp authors praise "no syntax" and code-as-data, so it seems to be encouraged, whereas I don't see Python programmers encouraging use of eval. Lisp seems designed for this kind of meta-programming. And it's really cool! It just makes me nervous.

As an outsider, I really appreciate the responses from you all. Basically you seem to be saying, "It's not an issue. We use it for macros, not for untrusted inputs. Don't call eval and you're fine." Obviously I wouldn't want you to do anything else. :-)

Anyway, thanks for the replies. It's great to have a community where I can ask Lisp users about something like this and learn what they think! It's something I've wondered about for a long time.

Code as data is dangerous, sure, but:

1. Python, Perl, Ruby, and even Java have support for code as data. If this makes Lisp a risky language, then you should be very worried about all of the above.

2. It is easier to disable code-as-data primitives (eval for example) than it is to add them to a language that lacks such support. I use Lisp for code that is not public-facing; the ability to extend the program at runtime can be very useful when security is not an issue. Languages that lack support for such things are harder to work with for cases like mine.

3. Code-as-data in Lisp is primarily a compile time feature, not a run time feature. Most Lisp programmers are using this feature via the macro system, rather than having programs that compute new functionality based on their inputs. You can disable all runtime support for treating code as data, and still take advantage of it at compile time.

Is Plato still useful in today's world? I think if you answer that then you should be able to give a a minimum a qualified "yes" to the question about Lisp.

Not sure about Plato, I would rather say Buddha.)

I was introduced to both Lisp and PLATO in 1992 at UIUC. I was very confused by statement until I realized you meant Plato, not PLATO, which has very little use in today's world.

https://en.wikipedia.org/wiki/PLATO_(computer_system)

If you are a JavaScript, Ruby, Python programmer it will teach you where first, rest, map, reduce, partial, apply, and a slew of other methods came and what functional programming is. It also has the side effect of teaching you how machines work at a low level (if you follow the SICP) with "environments" and frames. If you are a Java or .NET developer you'll pick up functional techniques and understand the origins of the object model. That's not even getting into macros, which basically are runtime code that writes code. I'm not aware of another language that has lisp style macros.

Ultimately it will help you be a better programmer even if you don't at first see the benefit.

* Most raw code written

So there are 20,000+ red-black binary tree implementations in Scheme. Great?

Sorry, but my view of what is really "useful in today's world" is what the language has that makes it easy to get at, i.e., make use of, other software, e.g., TCP/IP, device drivers, operating system services, authentication services, APIs, something old written in Algol, Cobol, Folderol, Fortran, C/C++, etc.

Actually there is a theme that a language can be interpretive and, really, still fast enough if mostly it is used just as thin glue to connect other, efficient software where nearly all the time goes. E.g., IBM's in-house mainframes on their VNET were long run heavily with just their interpretive language Rexx: VNET was a little like the Internet except the network was less smart and the computers also played the role of routers. At one time IBM had 3600 such mainframes around the world. I've been using Rexx on Windows as a scripting language and want to convert to PowerShell due to the better access to Windows services.

Speaking as a Lisper, I have to admit that this is an open question for me. On the one hand, the needs of parallel systems and state management offered by functional languages stands to bring Lisps (Clojure, Common Lisp etc) and other languages like Erlang ever more attention as we programmers attempt to manage the complexities of multicore design and distributed systems.

As a systems programming language I think that lisp has utterly failed. The Lisp machines are dead for various reasons (which I'm researching right now as a side-project) and to the best of my knowledge compiled lisp has clearly not taken the place of C in building operating systems (although I hope to do exactly that as does the author of [http://www.loper-os.org/]).

All of this comes down to an individual's definition of utility. Lisp is by definition of Turing Completeness just as capable as any other language you may wish to name, but with the usual Turing Tarpit or small language warning that you may have to roll your own. My perception, and one which several posts here on HN has affirmed is that traditionally Lisp was used by lone AI researchers who needed the power to roll their own anything quickly and efficiently. As a result of this "roll my own" mentality and simple lack of the internet (it was early and mid 1980s or so) libraries and SDKs as we know them were never built for Lisp systems.

Another difficulty is that as several other comments note there is no "one true" standard for Lisp. There is Scheme (which has official specs), there is ANSI Common Lisp which is a spec upon which several implementations have been based, and then there are countless other DIY and nonstandard lisps which elect to use different function names and otherwise make code non-trivially portable between lisp implementations. Not having a clear standard didn't help the lack of a library ecosystem at all, but some dialects such as Common Lisp and Clojure seem to be developing workable community library ecosystems. Of late Common Lisp has grown a library structure via ASDF code loading tool and the Quicklisp package manager, but Clojure is the only lisp I've ever encountered that really made any effort at all in the direction of providing native support for packaged libraries. The Clojure language includes syntax designed for allowing one file to explicitly state and "require" code in other files or even other libraries: a feature which is lacking from the Common Lisp standard. Thanks to technomancy's Leiningen tool and the clojars repository Clojure has a user-created system similar to Quicklisp + ASDF but trading ASDF's search path idiosyncrasies for a search path structure which should be familiar (or at least unsurprising) to Java developers.

TL;DR / Conclusion

If your boss comes to you and asks you to write a webpage, Lisp is probably not what you turn to. Could you? Sure, <shamelessplug> my blog is built in Clojure [http://arrdem.com].</shamelessplug> [http://refheap.com] is built in Clojure. [http://www.chris-granger.com/] is (presumably) Clojure. I know I've seen blogs in Common Lisp and other dialects but URLs escape me. Your boss says "Access the database", there are Clojure (okay fine Java but there is no real difference) libraries for SQL, MongoDB, Cassandra and more. Common Lisp also has SQL and MongoDB libraries. In short, while there is value in Lisp it is (for the time) not practically greater than the value of any other language despite the elegance of the functional approach and the power of the macro system. Hence its failure thus far to take over the world. However that same value proposition is improving as the CL and Clojure communities create and publish ever more libraries leading me to hope that Lisp may indeed take over the world one day.

This. Very. Site.

I'm not so sure that's a good example. It seems to me that PG had a Lisp he'd created for his own gratification and Hacker News was a good chance to try it out in the real world. It's hard to imagine Hacker News would have suffered if Arc hadn't been ready yet and he'd been forced to use Django or something. I think HN helped Arc more than vice-versa.

The point is that he (with mr. Morris, I suppose) made this site quickly, cheap and are able to evolve it on the go, almost without downtime with help of very few people.

Another point is that this site is the proof of an approach to software development described in the On Lisp book. So, it is not a gratification, it is just a real-world project they have done for themselves to make money. It is not some showcase or a demo, it is real thing.

In case of Django, I think, it will require much more man-hours for building and maintaining and more resources (cost) to run.

This is an example of Less is More principle in action: less code (the core language concept) and, as a consequence, less everything - time, people, money.

Lisp isn't just a language with parenthesis. It is a whole philosophy (a multi-paradigm language). Well, better to read On Lisp.

PG says in several of the Arc essays that HN exists because he found that programming languages turned out better when their creators worked on the language and some problem being solved in that language together. HN exists because he needed an application to force Arc to be better.

I hate to break it to you but HN is not a great site because of it's implementation. HN's value lies in the content and community. If somebody other than PG launched an identical site today, nobody would even bother making an account.

Its greatness is in that it's good-enough

Reply on news.ycombinator.com