Common Lisp - Myths and Legends | Hacker News Reader

Common Lisp - Myths and Legends

lispworks.com

71 pointsfogus38 comments

I love CL. Absolutely love it. But these sorts of articles (this one is actually quite a few years old) just depress me. It's as if every one who defends CL has to make these sorts of "It's only a flesh wound!" sorts of arguments. With all deference to PG, one web application in over 10 years does not an argument make. Like I said, I love CL, I use CL, but it is not a popular language and I can't blame any company that chooses to go with Python or Ruby simply due to inertia of the languages.

And don't get me started on how you are supposed to decide how to choose an implementation of CL...

The annoying thing is, I can't use LispWorks with an RDBMS unless I pay for the Enterprise Edition. So I can't even do a pilot project without it being "official" within the company. So there is zero possibility of a grassroots adoption happening.

Contrast this with Oracle. One of the most money-obsessed companies in the world, I'm sure we'd all agree. But you can download Oracle Enterprise Edition for free with zero fuss and kick the tyres. They don't care, no time limit or size limit or anything, until you go into production with it.

The open-source CL world has been slowly improving over the last several years. SBCL with CLSQL might work just fine for you.

That's really simple. You can get time limited full versions of LispWorks Enterprise from them. Let them know how long you need to evaluate the product.

You can't also access an RDBMs from C without paying for Visual Studio Enterprise Edition, or using any of the myriad of Free compilers out there that are much better anyway, which everyone else uses.

That's stupid, but it shouldn't be a dealbreaker.

First, you can get a time limited version, which may work.

Or you can write a wrapper around the few parts of the C api you need to use for your project.

> chooses to go with Python or Ruby simply due to inertia of the languages

Not sure if that's inertia of the languages. My Python path started with a) having only heard it's good for scripting/prototyping and b) the need for a quick way to test a SOAP interface.

Nobody in the company used it at that time, so language inertia was heavily against Python. I had the first successful SOAP call within hours of downloading the implementation (yup, the is important).

Googling for Lisp soap client gave some links for an implementation on commercial Lisp (deal breaker at this stage) and one whose quickstart docs start with at least three required dependencies "Make sure X works for you", "get package Y", and so on.

My point is that "batteries included" really makes a difference.

It's ironic when PG says Lisp is a "bottom up" language that it's so hard to get it into a company in a bottom-up way. Whereas Python always starts off being used by one person for their personal work, then as others see how productive they are/it is it snowballs from there.

I agree with you, although I do a fair amount of personal scripting in lisp to make me more productive at work. But for things like that, I could just as easily use Python, and Python is more readily accepted by other developers at my company.

It's no use pretending otherwise, "what's with all the parenthesis?" is still a major roadblock to acceptance.

It's no use pretending otherwise, "what's with all the parenthesis?" is still a major roadblock to acceptance.

Sigh. One might imagine a similar objection such as "what's with all the significant whitespace", but apparently that shocks people less than a few parentheses.

I wish more developers would see radically unfamiliar syntax as an opportunity to discover ideas they might be missing out on.

At my office, it has nothing to do with parenthesis and everything to do with the fact that a new person can be pointed to http://docs.djangoproject.com/en/1.3/ and told to get started. Done. They can learn it all on their own.

In all these lamentations about LISP not being top-dog I never see anyone point to a tutorial to bootstrap a project for a beginner.

The first google result for lisp web framework points to http://common-lisp.net/project/cl-weblocks/ which points to a Trac site for documentation which points to http://weblocks.viridian-project.de/ and under documentation you can get a link to the early draft of the guide at http://viridian-project.de/~sky/user-guide.stx.html

And once you finally arrive at the destination, there isn't a complete set of documentation. The project actually looks pretty cool, but how could I rationalise trying to learn off a set of incomplete docs? Or recommending it in a professional environment?

EDIT: And looking over the comments here, the only links offsite is someone complaining about pricing and someone else offering up some git projects where someone is currently translating a project into english. Pretty fucking pompous to say non-Lispers are scared of parens.

Sure, common lisp is not the best language for "connect the dots" style web development, but people are using it to write webapps. If you don't know the language, and you're ignorant of the options and tools, you will not be spoon-fed. If you need a way to glue a bunch of ready made components without spending too much time learning your tools and environment, python|ruby IS the better choice.

But of course web development might not be a sufficiently hard problem for CL to yield a significant benefit over python. Also, combining them is also an option, write your hard stuff in CL, let your less experienced programmers write the web interface in Django. Its a matter of what works for you, i guess.

Why am I not surprised your response is unhelpful yet pompous. It's what Lisp Does™

Your comment didn't indicate to me that you needed help. I'll try to be helpful, but i can't do anything about the pompousness, sorry. :D

So, how can I help you?

You've already declared web frameworks below Lisps's station. I'm not sure you can help me. Obviously the problems I want a language to help me solve are no concern of yours.

The truth is that anyone smart enough to use C++ for real work is certainly smart enough to use Lisp, and they don't, and it would be worth figuring out why. It's definitely not the syntax.

Not to mention that ViaWeb is a website... Was that not sufficiently hard a problem?

"EDIT: And looking over the comments here, the only links offsite is someone complaining about pricing and someone else offering up some git projects where someone is currently translating a project into english. Pretty fucking pompous to say non-Lispers are scared of parens."

Please can the attitude.

Not everyone speaks English as their first language. The RESTAS documentation is well-written and the translation is high quality (I helped proof-read it).

If you have trouble using github, there are help pages for that. If you think "github isn't friendly to newbies," too bad, because you will need to use git if you want to write software.

If you wanted links to tutorials about Common Lisp, you can ask for them. But if you really wanted those (and not just to troll HN), you would have asked already. Like that guy who asked for documentation for RESTAS and received links to the documentation and several example projects.

Here's something that's not a myth or legend: if I live in Europe I pay ~€150 more for the professional version than I would if I lived in the US. (http://www.lispworks.com/buy/prices-1c.html vs http://www.lispworks.com/buy/prices-2c.html)

SBCL + Emacs probably has more features and runs faster anyway.

I've been taking some time to learn CL lately, and one of the things I've been doing is SPOJ (http://www.spoj.pl) problems. One one paticular problem (FCTRL) I attempted to solve it in CL and came up with code that I was pretty sure would work. After submitting a number of times using the SBCL compiler, I could not solve the problem in the time allocated.

So I went off and just manually translated my code to Python. Exact same algorithm, pretty much word for word translation. Submitted, and made the cut (barely).

This I found dissapointing; how could a byte code interpreted language be faster than a compiled language?

So I picked up 'ANSI Common Lisp' and started optimizing. Found some problems where I was using / instead of when I could just use floor (but then I found a problem where floor always does a cons; why is there no simple floor where I don't get multiple return values). Submitted, finally got accepted (and slightly faster than python). Further optimizations included giving the exact ranges for certain numbers and inlining the floor function.

After an hour or two of optimization work I got it down to about 66% of the python run time. Not even close to what I was expecting in runtime performance, especially considering the 'SBCL is just as fast as C' arguments that are all around.

Not that I'm not going to stop learning CL, but it seems the performance claims just don't add up to the performance reality (for this case). And the development time was doubled just to make the program run acceptably fast.

As Paul Graham said in ANSI Common Lisp: Lisp lets you write slow programs fast, and fast programs slow. But in many cases 'good enough' is just that and I can really see how python excells in this regard and how it's taken over the hearts and minds of a lot of Ex-Lispers.

Common Lisp is not simply fast. Common Lisp enables you to write a lot of code in Lisp that is fast. If you write Python and just call the in C written functionality of the implementation it can be easily as fast. With Common Lisp it is possible to stay a long time in Lisp, without the need to use C to improve the speed. Plus you can write code with optimizations that can then get rid of much of the overhead (type checking, bounds checking, ...). But that should only be done locally. Generally the performance model of Common Lisp is not that simple. It takes some practice and some understanding of the compiler and underlying runtime to get code 'really' fast.

If you are disappointed with the speed of your code, you should really ask the experts in the Lisp community for some help. There are often trivial ways for speed up and then there are sometimes very complex ways to get better performance.

It is hard to learn this from books. You really need to interact with some people who have experience.

Code please.

I'll try and post the Python and CL code tonite.

Here is python code that I first used to solved to problem in the time allowed:

    #!/usr/bin/env python

    import sys
    import math

    count = int(sys.stdin.readline())
    for i in xrange(0, count):
        n = int(sys.stdin.readline())
        ndiv5 = math.floor(n / 5.0)
        if n >= 25:
            ndiv5 += math.floor(n / 25.0)
        if n >= 125:
	    ndiv5 += math.floor(n / 125.0)
        if n >= 625:
    	    ndiv5 += math.floor(n / 625.0)
        if n >= 3125:
    	    ndiv5 += math.floor(n / 3125.0)
        if n >= 15625:
    	    ndiv5 += math.floor(n / 15625.0)
        if n >= 78125:
    	    ndiv5 += math.floor(n / 78125.0)
        if n >= 390625:
    	    ndiv5 += math.floor(n / 390625.0)
        if n >= 1953125:
    	    ndiv5 += math.floor(n / 1953125.0)
        if n >= 9765625:
    	    ndiv5 += math.floor(n / 9765625.0)
        if n >= 48828125:
    	    ndiv5 += math.floor(n / 48828125.0)
        if n >= 244140625:
    	    ndiv5 += math.floor(n / 244140625.0)
        print int(ndiv5)

And here is my optimized Common Lisp version:

    (declaim (optimize (speed 3) (space 3) (safety 0)))
    (declaim (inline floor))
    
    (defun calc-trailing-zeros (n)
      (declare (type (integer 0 1000000000) n))
      (let ((ndiv5 (floor n 5)))
        (declare (fixnum ndiv5))
        (if (>= n 25)
    	    (setq ndiv5 (+ ndiv5 (floor n 25)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 125)
    	    (setq ndiv5 (+ ndiv5 (floor n 125)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 625)
    	    (setq ndiv5 (+ ndiv5 (floor n 625)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 3125)
    	    (setq ndiv5 (+ ndiv5 (floor n 3125)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 15625)
    	    (setq ndiv5 (+ ndiv5 (floor n 15625)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 78125)
    	    (setq ndiv5 (+ ndiv5 (floor n 78125)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 390625)
    	    (setq ndiv5 (+ ndiv5 (floor n 390625)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 1953125)
    	    (setq ndiv5 (+ ndiv5 (floor n 1953125)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 9765625)
    	    (setq ndiv5 (+ ndiv5 (floor n 9765625)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 48828125)
    	    (setq ndiv5 (+ ndiv5 (floor n 48828125)))
    	    (return-from calc-trailing-zeros ndiv5))
        (if (>= n 244140625)
    	    (setq ndiv5 (+ ndiv5 (floor n 244140625)))
    	    (return-from calc-trailing-zeros ndiv5))))
    
    (let ((count (parse-integer (read-line))))
      (declare (fixnum count))
      (dotimes (n count)
        (declare (fixnum n))
        (let ((factn (parse-integer (read-line))))
          (princ (calc-trailing-zeros factn)) (terpri))))

Notice the Python code doesn't even have the early returns that the Lisp code does. The Python code comes in at 5.42 seconds; the optimized Lisp version's best time was 3.39 seconds. I know the algorithm to solve the problem is probably not the most elegant, but it does get accepted. Any ideas on how to optimize the Lisp code further would be appreciated.

Hey, I was curious about your claims, so I ran both versions on my computer, and SBCL came out 10X faster. For one million calls (count=1000000) and n fixed to 100000, SBCL took 0.6 seconds versus 8.6 seconds.

One thing I did, though, was to call both functions directly with arguments, and not rely on read-line/readline. That might be the difference, I wouldn't be surprised if Python was optimized for text processing, and SBCL was lazier about it (Lisp programs call read et al. infrequently). Your code is decently optimized, IMO. Cheers.

Thanks for checking. SPOJ uses SBCL 1.0.18 so that might be a bit of an issue, since it seems to be a couple of years old. I was thinking that the input routines might have been a bit of a problem and that CL isn't optmized as much for that type of processing. Maybe something to look into in SBCL code base for improvement.

I don't know what you're expecting. Your algorithm is basically:

1) Read a line from a stream, parse it into an integer. 2) Do a little math. 3) Convert an integer into text and print it to a stream.

In Python, (1) and (3) are implemented in C. For Common Lisp, (1) and (3) are mostly implemented in Lisp.

Beyond that, (1) and (3) will completely dominate the runtime. (2) is probably just a couple of dozen CPU cycles, but the print and terpri will involve flushing stdout, which will involve a system call which will be thousands to tens of thousands of CPU cycles. Depending on how the stream is buffered, the read-line will likely involve a system call as well.

Basically in this test is just measuring how quickly the program can sit and wait for read()/write() system calls to complete. Since they're both calling into the same kernel, it's unsurprising they're the same speed.

Mmm, this looks like "number of trailing zeroes in n! when written in base 10", or, equivalently, "number of factors of 5 in n!". If we let that be f(n), and u = floor(n/5), then f(n) = u + f(u). Then, instead of writing out all the powers of 5 up to whatever your integer limit is... if you think for a little bit, this will work:

  (defun calc-trailing-zeros-2 (n)
    (do ((n (floor n 5) (floor n 5))
         (total 0 (+ total n)))
        ((zerop n) total)))

Methinks this is much better; if there is a performance disadvantage, I wouldn't care about it (and you can add type-decs as you wish; if you do, note that you'd probably have to say (the fixnum (+ a b)) even if a and b are known fixnums). If you want to verify that it works...

  (dotimes (i 30) (print (= (calc-trailing-zeros i) (calc-trailing-zeros-2 i))))

It prints a bunch of T.

As for the performance comparison, I'd echo what others have said: I expect the I/O (read-line, princ, terpri) to dominate the run time. ... Now for some experimentation. "randomthing" is an SBCL image in my executable "mybin" directory that executes the "let ..." code at the bottom; achieved with (save-lisp-and-die "randomthing" :executable t :toplevel (lambda () (let ...))). Let's test it:

  ;Clipboard is "100000" followed by 100000 instances of "500000"
  $ time pbpaste | randomthing > /dev/null

  real	0m0.453s
  user	0m0.352s
  sys	0m0.096s

  ;Clipboard is "100000" followed by 100000 instances of "000000"
  $ time pbpaste | randomthing > /dev/null

  real	0m0.352s
  user	0m0.263s
  sys	0m0.091s

In the case of "000000", the "calc-trailing-zeros" part should take about zero time, and therefore the remaining time is all the I/O stuff. Depending on whether we go by the "real" or "user" thing, this tells us that the math part takes up either 2/7 or 2/9 of the time of the whole program.

And just to be sure that "pbpaste" isn't the main cause of slowness:

  $ time pbpaste > /dev/null

  real	0m0.024s
  user	0m0.012s
  sys	0m0.009s

Fairly insignificant. Likewise, startup time for this Lisp image can be experimentally determined by having my clipboard only contain "10" followed by 10 integers:

  $ time pbpaste | randomthing > /dev/null

  real	0m0.032s
  user	0m0.010s
  sys	0m0.027s

So, yeah, I/O dominates the run time, at least on my SBCL (SBCL 1.0.47).

Someday when I fix my blog software, I am going to write the following blog post:

"How should I determine whether or not to use Language X?"

1) Do you want to use Language X? If you don't want to use it, don't.

2) Do you communicate with people that use Language X? Programming doesn't happen in a vacuum. You probably don't want to read the language implementation source code to determine how to call a function. So it's good if someone is around to teach you the language or to write docs for you. It's also nice to have libraries, because when you want to write Awesome Web 2.0 App, you probably don't want to write a sockets library too.

If you answered yes to both questions, you should use Language X. If you start using it and don't like it, then use something else. Please do not write a blog post about your experience, because nobody cares.

http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=s...

I couldn't find a date on the article, but it seems it's only web example is no longer valid.

"its only web example". It's means and can only ever ever mean "it is".

Technically you are right, but your comment doesn't add anything to the discussion. Typos happen, and I wouldn't be so anal about that. There are more constructive ways to get upvotes.

He improved every future discussion in which parfe participates.

Not really. It was a typo. I hope you aren't so distracted by a spurious apostrophe that it would detract from a discussion.

I had honestly considered writing my website in common lisp, but discarded that idea because I couldn't find good documentation for the RESTAS web framework.

Do you mean the official documentation, or 3rd party examples and tutorials? Andrey has been working hard at translating the official docs to English the past few months: http://restas.lisper.ru/en/

As for tutorials, you are right that there could be more. Between https://github.com/archimag/rulisp and https://github.com/archimag/cliki2 there are some good example sites built in RESTAS.

[advertising]

Actually PR, but yes.

Just like everytime you hear the words "Zed Shaw", "Linus", "RMS", "Ruby on Rails", "Node.js".

It is all branding.

Reply on news.ycombinator.com