Lisp for the Web, Part III
vitovan.com
vitovan.com
Or one of the frameworks built on it:
The article does a pretty good job of starting with the basics and introducing more complexity step by step. I would argue that the author's approach is better than any attempt to introduce a full-stack framework, which does not help the beginner level lisp programmer (clearly the target audience) understand - one might take away some recipes from that but no real understanding.
Which, if any, "modern" frameworks one uses then finally depends on the project and taste. After all, gluing together the various routing-, persistence-, ORM-choices only takes a few lines of code anyway.
And, after having been consulting as a lisp programmer for some years now, I would even argue that the overhead of learning some full-stack frameworks is usually not worth the benefits they claim to provide.
I'm not an experienced Lisp developer, but this jives with my experiences with frameworks in other languages. It seems like frameworks cover 95% use cases: they make your life easier for 95% of what you're trying to do. But for the other 5%, you can't just work outside the framework, because frameworks couple you to their paradigm. So you end up trying to solve problems through the framework that the framework doesn't have good solutions for. You're stuck solving problems without the debugging tools or basic building blocks that writing your own in a programming language offers you; it's like trying to tighten a screw with a hammer in the dark. The result is that you end up spending a lot of time on that 5% of problems.
What makes this particularly dangerous is that you typically don't start solving the really unique problems of your project until you've already gotten pretty far into your project: you don't run into the 5% problems until you've already committed heavily to using the framework. A lot of less experienced devs get caught up in this: they start using a framework and everything seems easy. Drunk with their newfound power, they try to do everything in a framework, and each new project reinforces their belief that the frameworks are saving them time. It takes a long time to be on enough long-running projects that you see the pattern and recognize frameworks for the double-edged sword that they are.
To be clear, I'm not saying that frameworks are always bad. For simple or short-lived projects, a framework is probably the best way to go: you'll probably never hit those 5% cases where the framework will hurt you. But if you're doing something at scale, or for a long time, or doing something very complex: a framework might not be the best approach. And large-scale, long-running, and complex projects are where the money and job stability are at.
There is definitely a middle ground to explore here. For example, in the Python world, there are some great minimalist frameworks (Flask, Bottle) which can be tied together with a template engine (Cheetah, string.Template) and SQLAlchemy to give you most of what a big framework like Django or Pylons gives you, but in a much less coupled way that allows you to create your own extensions points. Libraries are much more flexible than frameworks. Sure, you'll feel more pain up front, but you won't spend your time trying to get an ORM to emit a specific SQL query, or waiting for your cache middleware library to accept your PR on a 10-year-old bug in its invalidation.
That way, if a new server comes out (e.g. Woo[0]), you don't have to learn all of its internals and rewrite the entire application using it -- you literally just change a keyword argument in a single function call, and now your application is running on a different, much faster server. If someone writes a plugin for Clack (e.g. clack-errors[1]), all applications built on any Clack framework using whatever server can use it. A Hunchentoot plugin works for Hunchentoot applications.
Using Clack is sustainable web development.
Sure, for a five minute demo, Hunchentoot is fine. But why teach bad practices?
Woo is many times faster than Hunchentoot. And projects can die.
>a person needs to understand the problem a tool solves before using the tool; otherwise you're training a code monkey, not a programmer.
There's a reason we use high-level languages: They hide irrelevant details. You shouldn't have to learn how TCP works or know every detail of HTTP to know what can be done with it.
"Hello World" on the local machine...
https://github.com/fukamachi/woo/blob/master/benchmark.md
is that what you mean by 'faster'?
Personally I'd prefer a more representative benchmark, which actually uses the network... Really measuring the qualities of a Web server is a bit more work.
Yes.
This is only true if you're writing Web 2.0 media websites that are only nominally not static websites. If you are solving an actual hard problem instead of just driving ad traffic, a framework is going to get in your way.
Simplest explanation of Lisp grammar I've read!
My impression was that the only thing wrong with XML was the syntax (hence the JSON revolution). After all, apart from syntax, possibly some of the ill-advised "external element" stuff ... I think XML is strictly better in every meaningful sense than JSON.
Already gave a nice comparison here: https://news.ycombinator.com/item?id=10044598
Firstly, it is rare to see the SBCL REPL used directly.
Most commonly, lisp is developed via an IDE that will include a full-featured REPL (there are several; for sbcl SLIME is what is most commonly used. ClozureCL has its own IDE on OS X, and commercial lisp implementations will ship with their own).
GNU clisp has a more feature-full REPL, so you may see that used occasionally.
Secondly, lisp has a build system called ASDF. There is a tiny amount of code needed to setup a simple project, and there is a tool named "quickproject" that will generate it all for you. With all the improvements to ASDF in the past decade, I never do a "write lisp file, then load it" anymore.
GNU clisp has a more feature-full REPL, so you may see that
used occasionally.
EDIT: It looks like clisp has experimental threading support if you compile it correctly: http://www.clisp.org/impnotes/mt.html
So I guess the better question is: does Hunchentoot work on clisp?
Why do people who write or talk about Lisp always have to start with something like this? To be honest I still can't see the appeal of Lisp. Yes everything is a function so you can add/remove 'language features' as you want, and the syntax is really simple, and code is data (a rare need of mine anyway). But Lisp sacrifices the most important thing of all; readability.
Everyone tries to read programming languages as close as possible to plain english in their head, but Lisp goes completely against this and makes you write things like (* 1 (+ 2 (/ 3 4))) where you have to glance back and forth to even understand the equation - is 1 * (2 + 3/4) not easier to read? Even in C# the phrase 'if (!(foo is bar))' annoys me because it reads 'if not foo is bar' when I'd like to say 'if foo is not bar'! So I think me and Lisp have no hope.
No. Read on macros.
> and code is data (a rare need of mine anyway)
Rare? Read on macros, then on "Blub paradox".
> But Lisp sacrifices the most important thing of all; readability.
> Everyone tries to read programming languages as close as possible to plain english in their head
No. That's too wrong to even seriously argue against. The best I can do is to paste this: http://blog.ppenev.com/parens.png without any comment.
The rest of what you wrote is completely subjective and I don't want to comment on it. But I think you should go and take a look at Smalltalk, which is another awesome langauge with 4 decades of history, which pretty much reads like plain English.
The parenthesis are actually the least bit I have a problem with (as the indentation generally makes flow clear). But the actual function within a function with a function syntax and so on just seems to me to hinder the reading of the code and thus the understanding, maintenance and modification of it.
Why can't you believe that this is only because you're not used to reading this syntax? It's very frustrating for me. I know the basics of ~80 programming languages (see my blog for a full list) and I know for sure that "readability" is an artifact of familiarity and nothing more. I experienced the transformation of a bit of code from "unreadable mess" to "elegant and concise" more times than I care to count. Why can't you believe is true? Why don't you want to check it for yourself? Why don't you just - as I did - try using some Lisp for half a year and try to judge its readability then?
And above all else, why do you insist on taking part in the discussion when you didn't invest the required time?
EDIT: this comment went from 1 to -2 to 2 to -1 again during the last hour. I wonder, why the downvotes? Am I being rude? I didn't intend to, sorry if it sounds like that. I'm not sure how to change the wording to make it better (obviously I'm not a native English speaker). Could someone please help me?
Or am I being wrong in my central proposition, which is that '"readability" is an artifact of familiarity and nothing more'? I admit that this "nothing more" part is an oversimplification on my part, but not by much. It's just that it's not worth discussing these relatively minor things which affect readability without first accepting that 99.9% of perceived readability comes from familiarity. Other than that, though, I'm honestly not aware of any serious papers or experiments which would claim otherwise - that some programming languages are inherently more readable than others.
I'm not much of a programmer, and don't usually comment on posts regarding on programming languages, but after looking at a lot of languages to learn; I am most intrigued by Lisp. I don't like wordy books. Lisp seems to be the least wordy? As yes, I know it looks complicated, but there is a simplicity about it too?
And as a novice, I agree that "readability is an artifact of familiarity". Engish is easy because it's the only language I was taught. I had a hell of a time learning to write, and speak it as a kid. I look back, and I think I had a learning disability?
I told a guy at work, "There's no way I could ever learn Chinese." He looked at me, and said, "I think you could learn the language. It's a lot of one syllable words. You're just used(familiarity) of English?
I use JavaScript because I can ask questions on Google, and get an immediate response. I hope more people put up tutorials on Lisp. I would love to see a walk through(step, by every little step) on say the construction of let's say --Hacker News? I've always been intruded by construction of this website. I looked at the source code, but need help--on a lot of levels. I looked at lobster.io, and it's not Hacker News. (I guess it's part "familiarity"?) Actually, what should do is learn Lisp, and then put up a tutorial, but that will probally never happen because I'm just not the guy for the job on a lot of levels.
A lot of people recommend learning the basics of a language before jumping into a complicated application, and that's good advice. I have found, I learn better when someone builds a program, and slowly walks me through the steps. I do take the side notes seriously. When someone recommends learning A before moving on the B; I follow that advice.
As usual went off base-
> (+ 1 2 3 4 5)
or
> 1 + 2 + 3 + 4 + 5
In my (subjective) opinion, the first one is. The extra +'s are redundant and noisy, and sacrifice a meaningful syntax for "trying to read as plain english". The syntax of any language takes practice and lots of reading - that's not unique to Lisp.
In your example, the 2nd one is actually what I'd prefer to read as I know that it will be compiled down to 5 ADD assembly instructions, whereas the first looks like it will be turned into a function call that will then initiate a loop. I'm guessing/hoping Lisp optimises the (+) function though so probably a null argument from me. See my reply to nnq for a better example.
(doof-boof "1 3" f1 3 "f1 3" 'f "f3 " f")
For me, a mostly visual thinker that ends up building and visualizing a code tree in his head anyhow, Lisp is very readable because I can more clearly see the tree without all the noise.
And when it comes to math, staying close to standard math notation is a clear winner, and this is why all lisps have ways of writing infix for formula-heavy code, stuff like http://data-sorcery.org/2010/05/14/infix-math/ and others. But lispers don't use them that much because they simply get used to the Lisp notation more.
...oh, and Common Lisp is NOT an example of readable Lisp. It's probably the most unreadable and obsfucation-friendly of them all :)
Take a look at some Clojure code for something that's just as readable as Python after you "get you eyes used to it" (jargon for learning to ignore the parentheses and learning the special meanings of the "funny symbols" like :, ~, # and a few more): https://github.com/weavejester/compojure/blob/master/src/com...
Anyway to take an example from your github:
> if (and form-method (= request-method :post))
am I right in saying this is pseudo-imperative:
if (form-method && (request-method == 'post'))
To me the ability to read it left to right seems like a huge win. With the lisp version I seem to be holding the program in my head until the very last (= ...) just so I can work it all out. Perhaps that is just in-experience though.
It's only a win if your mind actually needs the "left to right representation". For me, reading that "if" is more like:
1. oh, so this condition is and-ing 2 child conditions, so they must both be true
2. now the first sub-condition... ok, so it just sees if form-method is truth-y
3. now the 2nd one... ok, so we're comparing if two things are equal... let's see what these things are
4. so request-method must be equal to post
The advantages of people who think more like you would be that:
1. you probably find it *much easier* to explain your thoughts about code to others, since translating thought to words is a piece of cake for you (I envy you verbal thinkers for this a lot! :) )
2. you can easily map code to math formulas, and figure out differences between them (I can do this easily too, but I just need to use a more complicated algorithm in my head)
1. breaking complicated stuff into smaller pieces that can be understood off-context is easier (try breaking up the sentence I just wrote above at each comas and see that they kind of make sense separately too)
2. thinking about meta-code (code that generates code like Lisp and Scala macros) is much easier: because our mind's representation is closer to the machine's AST representation, writing code that writes code that writes code etc. feel just as "natural" as everything else to us. For example, when thinking about implementing an ORM, my first thought is something like "oh god, if the language I'm working in just had *real macros*, I could write all this in a few hours and couple hundreds line of code, and not need a mind-bending hierarchy of classes to do it" :)
The whole point I'm trying to make is simply that people think very very differently! Indeed, the Lisp advocates should stop repeating the "it's awesome" mantra and understand that it's only awesome for those with the brain wired a certain way, and it's probably painful for the others.
Nirvana would be to port Lisp's features to a more math-like and English-like language. But nobody has been successful at this. Both Scala macros and Template-Haskell are truly fucking hard if you try to actually use them for getting stuff done. So I count them just as failed experiments. But there's always hope :)
Actually typical Common Lisp is much more readable than Scheme or Clojure. 'obsfucation' is built-in in Clojure, where it has to be programmed in Common Lisp. Scheme has great potential for unreadable code, given it is a Lisp-1 and it makes heavy use of recursion and higher-order functions.
Common Lisp OTOH favors: long descriptive names, descriptive keyword arguments, CLOS based data structures (instead of untyped lists and vectors), explicit loops, explicit calls to variable bound functions, ...
I for one start with the shape of the code -- the pattern of color (from syntax highlighting) and negative space (from indentation) -- and only then focus on the different parts.
And Lisp is particularly helpful this way, because instead of paragraph-like blocks of code where you have to reconstruct the state of the code in your head by mentally modeling side effects, you can see the general outlines of what's happening from the levels of indentation.
I want to defend Lisp, of course, but I'm also genuinely curious about different ways people read code.
Example:
if (...)
{
...
}
else
{
...
}
loop
{
...
}
The only other obvious variation I can think of is I will read bottom up when tracing the path a variable took to get to where I am currently looking.
if (condition0
&& condition1
&& condition2)
But then, I also like Lisp. Prefix notation for "and" gives the same benefit directly, no tricky formatting required.
All of these things are _complications_ not just in the syntax but in the mental model. In order to effectively manipulate the code, you must hold all of those rules in your head while writing it.
In Lisp, the syntax requires the direct expression of context (parentheses). It also has dead simple semantics: do this to that. This means that Lisps are essentially optimized for authoring the code in the first place. This is what is meant when people say it is easy to reason about.
Of course, this is a trade-off, but arguably one that makes a developer more productive. Yes debugging is more difficult. Yet I'd venture it is a less common activity with Lisp as wrongness in the program is often obvious to the author.
But yes, not for everybody.
Because to them it is awesome. People are different, and just because it isn't awesome to you doesn't mean it can't be for others. For me, I was impressed because Lisp overcame so many limitations I found in other languages.
When it comes to readability, I think that a lot of it is what you are used to. I do not use Lisp all of the time, so when I switch back to it, some time is required for me to get back into the groove where it becomes readable, and then it's second nature. (I teach multiple computer science courses using different languages, and switching gears from class to class can be a challenge.)
All of that being said, the greater use of punctuation and syntactic forms in other languages can aid in readability, but I don't think it is significant enough to completely offset the power you gain from the Lisp syntax.
[1] I believe it's from birth, it was a thought experiment at first, IIRC it didn't even have numbers. Just symbols. It was about writing your own abstractions (see the symbolic differentiator in Lisp first paper). If you don't think about it that way then it surely will look absurdly convoluted.
Well, the people who write or talk about Lisp generally probably think it's true, and it's the sort of thing that, if true, makes for a fantastic selling point.
>you can add/remove 'language features' as you want, and the syntax is really simple, and code is data
I think this is the meat of it: that Lisp offers an exceptionally low barrier to entry to the world of, let's say, "experimental programming." It lets you do things you would never dream of if you've only ever mucked around in something like C or Java. Whether or not those things are actually useful in and of themselves, it's fun and rewarding to experiment not just with programs, but the way you write and even think about them.
Of course, there are people who will tell you that Lisp is also good for solid technical reasons, but I think this is the main appeal of Lisp to most people who describe it as "awesome."
>Everyone tries to read programming languages as close as possible to plain english in their head
That's simply not true. You may do this, and it's a perfectly valid approach, but not everyone thinks that way. Personally, I think of code in terms of what it "does," rather than what it "says," if that makes any sense— it's difficult to explain.
As for readability, I firmly believe it ultimately comes down to familiarity. Yes, infix syntax is easier to read for a limited subset of small, operator-dense arithmetic expressions, but I find that kind of code is relatively rare. Broadly speaking, you're going to get most of your structural information about Lisp code from the indentation; the parentheses eventually fade into noise. And honestly, the same is true of just about every other programming language.
Note that your sbcl must be built with threading enabled for the example to work. Otherwise, the initial Lisp landing page will show up at URI /, but the handler for /hello?name=Blah will never be executed because sbcl-without-threading silently fails to use threading and instead runs the server in a blocking way in which later code never gets a chance to run.
This was confusing for me at first, but once I built sbcl from source using "./make.sh --fancy" everything worked perfectly.
For example: implementing authentication and authorization.
So I agree with everything you said but I'm mostly talking about getting shit done.
I fail to see any problem with this. Writing non-idiomatic code does not mean that it's going to take more time. There's certainly a value in writing idiomatic code in a given language, but "practicality beats purity", at least in my mind.
It's still hot, I added the code for generating rss one minute ago.