Ask HN: Common Lisp vs Clojure for web apps? | Hacker News Reader

Ask HN: Common Lisp vs Clojure for web apps?

37 pointshackoder42 comments

I'm building a somewhat complex web app, but eventually what I need are 1) support for postgres, 2) libraries which make it easy to deal with data in json and xml, and 3) allow me to generate html and javascript.

Overall, I think both CL and clojure support this basic feature set. And I'm trying them both out on a trial app anyway to get a feel for them.

However, I'd appreciate if anyone can provide me with some tips/guidance/gotchas/"this might help" sort of advice to get me going. Thanks :)

View on HN

Either will allow you to do this. I imagine the only difference (in terms of existing features for your platform) would be 'generating JavaScript.'

CL has Parenscript http://common-lisp.net/project/parenscript/ which is probably more powerful than whatever JavaScript compiler/macro/generator tools you would have to hack together yourself in Clojure.

As for my own personal opinion, I would choose the Lisp that has tail call optimization.

If you are willing to look into a third option, I highly recommend HOP from Inria http://hop.inria.fr/ which is a Scheme web platform with a true JavaScript compiler.

Parenscript, a javascript generator, eh?

I don't understand, why would anyone want to generate javascript from S-exprs? I understand this is fashionable in the CL world, but I've never seen any good justification for this practice outside of saying, "Now everything is in lisp."

But this isn't a terribly compelling argument. Most web applications are multi-language and it doesn't seem terribly detrimental to them. Further, the state of the art in web technology moves so incredibly fast that it seems strange and almost obstructionist to insist on a secondary library that needs to be kept in synch.

Finally, it tends to lead to crap designs. Anyone who does cross-browser implementations designs will tell you that simple CSS over a generic HTML framework is almost never enough. Frequently you need to introduce new enclosing scopes or add seemingly senseless markup to help make IE play nice (a torturous exercise many web designers still need to go through). So while CL has some awesome web libraries, I'd say their extensive work into making HTML+JS+CSS generation from pure lisp seems like a foundation built on sand. Sometimes I wonder if the whole exercise was just to avoid working with HTML, which many (more prejudiced) developers find distasteful to work with.

I don't think Clojure is at a significant disadvantage to CL in this field. If anything, it's at an advantage in web applications, as it has access to a massive number of very good web-related and utility libraries. A paucity of modern, battle tested libraries that "work everywhere" has always been Common Lisp's achilles heel.

Well people would do it for the same reason they use the GWT. In that case they are more familiar with Java and GWT lets them write Java and have it compiled to javascript for client side use.

Good points, KD. Clearly, you've been there. Is there a good writeup somewhere of the deviations from "nice" web design that are needed?

I (OP) agree with a lot of what you've said. I have done a lot of work in javascript, and my reason for html/js generation wasn't because I believe that everything should be lisp. I enjoy writing html, js and css from scratch.

The reason I chose it as one of my criterion is because it allows you to get off the ground _fast_ if you're coding in a single language (so basically it helps with prototyping stuff). I really enjoy the html generation libraries- I can test stuff without doing a mental switch between languages, even build a half decent UI very quickly, and that is a huge advantage. My initial scripts will not need to be run in multiple browsers, they'll be simple scripts to get the app going. Since I intend to keep the scripts small and smiple, I can replace them later with javascript as the app grows.

Thank you for a serious response!

So you view it as a net win in rapid prototyping?

You say you "don't understand", apparently disingenuously, because you then go on to announce what the argument supposedly is, object to it, change the subject (what, exactly, does Parenscript have to do with HTML?) and end with a red herring.

As someone who works in Parenscript every day and hasn't had his coffee yet, I find that irritating.

> (what, exactly, does Parenscript have to do with HTML?)

Because we frequently see Parenscript used in isolation from sexp->html compilers? I sure don't.

I'm not ending with a red herring, I'm saying you're creating a huge integration hassle with the rest of the web application world. Heck, even pattern-substitution templating systems are falling out of style as it becomes feasible to do structure-substitution templating. Most designers and javascript experts do not know lisp, and further don't want the equivalent of a wet blanket separating them from their medium.

And I don't mean to upset you, but I also notice that you didn't exactly provide any justifications for parenscript. It's a cool project, and I've even used it for production work. However, I never felt it was anything more than a stopgap solution, and my impression from the CL community was that they didn't like HTML+JS because it wasn't lisp.

Parenscript excels at combining high-level abstraction with runtime efficiency in a web browser. It has to do with Javascript, not HTML, and if you've used it as much as you say, you know that perfectly well. Arguing that PS sucks for web apps because web designers don't know Lisp could hardly be more of a red herring.

"I'm not ending with a red herring, I'm saying you're creating a huge integration hassle with the rest of the web application world."

How so? You're still getting regular JavaScript out of Parenscript. That's the whole point.

"Heck, even pattern-substitution templating systems are falling out of style as it becomes feasible to do structure-substitution templating."

What is pattern vs structure substitution? I've never come across those terms.

When you can generate javascript from s-exprs, you can use macros with javascript.

So you can add first-order generated-code errors to your generated-code errors when your javascript generator has a problem or a new feature comes into play incorrectly.

I understand that's nice and all, but couldn't you just write javascript? While it lacks macros, it is certainly dynamically expressive enough that you can do pretty much anything you could reasonably require. A cursory examination of Mootools and JQuery could tell you that.

  > I understand this is fashionable in the CL world, but I've never seen any good justification for this practice outside of saying, "Now everything is in lisp."
  [...]
  > I understand that's nice and all, but couldn't you just write javascript?

I agree with you on this point (JQuery is quite nice), but this is orthogonal to the CL vs. Clojure vs. $Framework debate.

http://api.rubyonrails.org/classes/ActionView/Helpers/Protot...

http://clamato.net/

http://github.com/collin/jabs

http://en.wikipedia.org/wiki/Objective-J

If you are using all the higher-order features of javascript to make it terse and expressive, you are probably "doing it wrong" in terms of producing fast code in a selection of browsers. From this perspective, javascript is the compiler output language, like machine code. Parenscript would then be like assembly - a 1:1 mapping - but embedded in a powerful enough macro system that it can be wrapped into a readable DSL.

> you are probably "doing it wrong" in terms of producing fast code in a selection of browsers.

And yet JQuery and Mootools are both incredibly fast, as benchmarks show.

And yes, I'm aware of the model you're using, and reiterating explicitly what I implied isn't making your argument any stronger, in my opinion.

They are fast because they have been coded by experts. So maybe they look cute - would your code that looks as cute run as well? Probably not unless you can match them in knowing why.

Or by leveraging their work as their documentation suggests.

Seriously, performant javascript is not some magical thing only wizards can attain. Performant _generic_ javascript is tough stuff, but just using the libraries is hardly JIT compiling.

Which benchmarks for what?

I'm not being smug, I really want to see some benchmarks that make sense. The only thing I've seen is either "jQuery version x.y.z. is % faster than jQuery version a.b.c." or "jQuery is % faster doing something than Prototype." Yes some of these libraries may be getting less slow, and some of them are less slow at doing certain things than other slow libraries, but the only thing that would make sense is to compare them to hand-written JS and see the trade-off in performance vs code size.

Most jQuery application code I see is just adding layers over the DOM in a way that doesn't provide any extra expressive power or conciseness (frequently the opposite).

Yes, selectors are cool, but only if you need to manipulate a document that's been generated by someone else. If you have control over how you generate HTML, you can actually keep track of your DOM elements in a way that doesn't involve searching at run-time and expresses domain concepts - I mean, imagine this!, actually program normal code.

One reason to generate JavaScript is if you need a piece of code on the server and on the client. Stuff like validations comes to mind.

"I don't understand, why would anyone want to generate javascript from S-exprs? I understand this is fashionable in the CL world, but I've never seen any good justification for this practice outside of saying, "Now everything is in lisp.""

No, that is the justification behind things like GWT which are written by people who don't know web programming for people who don't want to learn web programming.

The point of having everything in Lisp is enabling the possibility of building your application in terms of domain concepts (I want to say "DSL" here, but the problem is that the term DSL has been coopted by Java morons who think it means something like scripting language).

The difference with other single-language systems like GWT is that this is done using macros - you're translating domain concepts directly to the JS+HTML+CSS target, instead of trying to shoe-horn them into some brain-damaged OO layer that may or may not need runtime support.

It's not about "abstracting the web," (abstracting to what? "Abstracting" JavaScript to Java is going from dumb to dumber); it's about the domain.

Clojure has scriptjure: http://github.com/arohner/scriptjure

which compiles s-expressions to javascript.

Very cool, thanks for showing me this.

On the Clojure side, definitely check out Compojure, a Sinatra-style framework.

http://github.com/weavejester/compojure

> I'm building a somewhat complex web app

I'm guessing that if you had a CL background the choice would be easy!

The answer also depends in part on what your app structure's going to be like.. some considerations:

A. Will you embed your UI logic in Lisp?

B. How interactive do you want your application to be?

C. How computationally intensive will it be?

D. Is horizontal scalability (ala swarmiji) important?

E. Will the application deal heavily with forms?

F. Do you want to use SQL, or go through an ORM? Or use objects directly? (eg (mapcar #'do-foo (all 'clients)))

> make it easy to deal with data in json and xml, and

cl-json has been very easy to use, with trivial mapping from lisp objects to json.

> allow me to generate html and javascript.

We use html-template. If you're using forms you'll have to decide where you want to put your validation logic. With html-template you'll probably end up rolling your own, or do it in javascript.

A big consideration may be that Clojure is much more strictly functional than CL. That may limit your coding flexibility (and be harder to adjust to if you come from an imperative-coding background).

IMHO going with CL will yield greater benefit with Slime and the debugger, related to dynamically responding to requests, debugger-tested programming etc.

Real-world Clojure programming seems to require a knowledge of java libraries. CL libraries have been available for everything we've wanted to do. Postmodern is probably faster, and more "native" than JDBC.

If you want to put your UI complexity in Lisp (as opposed to javascript) then check out Weblocks, which has many nice features, one of which is generating validations from the data-model. If you want to write a RESTy app then plain hunchentoot should serve just as well.

Have you checked out some of the Clojure/CL webcasts? They should give you a feel for the relative strengths and weaknesses of developing in both environments.

Clojure seems likely to be able to plug into whatever Java libraries you might want to use. CL (in particular, SBCL) is used to power a number of web sites and they use Lisp all the way down. One of my favorites is http://wigflip.com/ that has a number of fun toys. Weblocks is a good CL framework. and there is support for Posgtress (as well as mysql, etc etc) available for CL.

Check those out for that side of what you can do with CL.

I am less familiar with the Java world, but Clojure seems to be very capable in that area.

Last time I checked Weblocks it had an outdated documentation (as the original author left the project) and new developers were adding new super-cool stuff in such way that using it actually involved becoming a dev on the project.

Few years later I experienced the same with UCW (beign another continuation-based web framework). And the same with Lisp-on-Lines (mr author, do you have something working outside of your own lisp image yet, after 5 years or so?)

Then I came to conclusion that there's some reason in the CL world that many projects become write-only vacuums with NIH syndrome, and their dev communities don't care of creating a _user_ base as they favor having _dev_ base. Also the term 'change management' or 'release management' seems to be incomprehensible by CL people. That causes almost impossible to deliver a product to the customer based on such software and stay sane.

In the second sentence I meant: Few years before.

UCW, Weblocks, and LoL were all started by people who did not understand web programming. If you're using continuations to "abstract" away browser-server interaction, you pretty much don't get web programming, period. Instead of focusing on the browser, these people want to focus on the server and pretend everything synchronously flows from there. This is a brittle and expensive (esp if you don't have first-class continuations!) illusion.

That explains why the frameworks were not high-quality in the first place (note: I personally don't believe that any web framework is worth using), and more importantly why the people who started the projects left: they just don't care about web programming (or actually any programming that has to deal with asynchronous human input) and just wish it would go away.

Release management is a catch-22 in the Lisp community: until people who maintain Free Software Lisp projects stop believing they don't have any users, they won't invest in it, but until they start investing in it, they'll continue to believe they have no users because people will just give up instead of complaining about it.

Smart conclusions.

I'd add to the Lisp-vs-release-management problem: The "I live inside the image" approach discourages delivering stable tested code changes, as it's quite hard to isolate them. I agree using a REPL is great for prototyping, but if one never builds from scratch, he can create the unmaintable code which doesn't work outside of a few specially crafted and hacked Lisp images.

Luckily Clojure doesn't seem to hold this legacy. People usually seem to inject code into the repl when they code but then switch to builds using ant, maven or by some layer above them (see Leiningen). I do it as well, compile inside a repl as it's quicker, then compile from scratch before a commit. It seems to be a sane compromise. It's also good you can't save current "jvm image" so there's no temptation to never leave it. I have a sad observation that sticking to "live inside the image" actually produces lone-ego-superstar programmers not willing to cooperate.

I sadly agree that "lone-ego-superstar programmers not willing to cooperate" are more prominent in the CL world than in some others. (Edit: "semi-autistic pricks" is the term that comes to my mind. Still, many of us are not like that.) I'm also willing to believe that the Clojure community is refreshingly different in this respect. But I don't buy that "living inside the image" has anything to do with this, or at least anything to do with causing it, and I especially don't buy that a technical limitation of Clojure/JVM constitutes some sort of virtue.

OTOH you can measure the technical limitation and then go around it in hours. That's a lot easier than struggling against social limitation for years (as w/ CL).

I'm biased - I generally prefer the Clojure way of doing things to CL (data structures, unicode, portability, library compatibility) but then the last time I seriously used CL was ~2006-2007, so things may have changed since.

But, to address your requirements:

* JSON: clojure.contrib.json

* XML: clojure.contrib.prxml

I gather there are a JS generator and an SQL interfacing library in contrib as well, though I haven't used them. Worst case you can access any Java libs for those purposes.

You also get to plug into any webserver that supports Java servlets, if that's an issue.

The killer feature for server programming in Clojure is the STM though. It means you can multi-thread your server relatively worry-free without resorting to moving all your state out-of-process (e.g. all state lives in the DB). You still need to pay attention to synchronising I/O, but compared to manual locking of everything, STM is a piece of cake.

(context on my perspective: I've delivered a Clojure-powered back-end server for a multiplayer Flash game for a customer, as well as made some minor short-lived web sites; clojure was great for all of those use cases - I'd link to them if I was allowed to put my name against them)

I say Clojure, because I would be hard pressed to think of a platform better than the JVM. Garbage collection, speed, deployment are all top notch in java-land.

Postgres support will probably be easier w/ Clojure, because everything is writen in terms of JDBC. Postgres is also routinely integrated w/ clojure apps.

Clojure supports xml parsing out of the box, and there are json abilities in clojure-contrib.

It really depends on you (and I say this as someone who just chose to write a huge CL webapp).

If you have experience with Java - I'd probably go with Clojure. If you have a lot of background with Lisp (i.e., Slime/ASDF don't scare you) I'd go with CL. If you don't know either - just pick what's more interesting to you (but be aware there's a steep learning curve either way).

On the CL side of things:

-You want to use Hunchentoot as your webserver

-CL-SQL for persistence

-CL-WHO if you want sexp => HTML, or html-template if you want pure HTML templates

-Parenscript is good for Javascript since code you write in Parenscript can run on either the server or browser (it makes it really easy to Ajaxify things, for example). But, if you're doing lots of really fancy JS and you're great at the language already, I'd stick with pure Javascript.

-For JSON, I personally prefer YASON to cl-json, but take your pick (they're both fine).

I preferred Postmodern last year, for interacting with Postgresql. Around that time, I vaguely recall it was considered faster than CL-SQL as an Elephant backend; and I found it far easier to install than CL-SQL. (I recall during CL-SQL's installation, it kept signalling errors for each DB it supported which I hadn't installed. Someone at the office said I was supposed to just keep telling Slime to ignore the errors, prodding the install forward. That actually worked.)

Maybe things have changed in the interim?

If the OP chooses Clojure, then I'd note that the Java classpath really demands you respect its inflexibility. add-classpath is a second class citizen; I've had JDBC (the Java DB manager) swear up and down that it registered the MySQL driver I loaded via add-classpath. But come time to read from the DB, it only worked when I ensured that the MySQL driver was loaded at the beginning in the normal way. What that implies is that if I want to load a new DB driver, I'd probably have to restart the JVM. (Or do something absurdly tricky, or debug more deeply.)

I think your problem is more likely to be with class loaders than class paths. Clojure probably has to set up its own, to allow all the monkey patching it does, including dynamically setting the classpath. If JDBC was loaded already, it will be able to see Clojure objects passed as parameters, but not able to find the class by reflection on its name.

Experience with Java won't get you far with Clojure. They're about as different as two languages that share libraries and a VM can be.

I think it would be easier to go Java => CL because at least you still have mutable data, imperative style, and usually a low level interface to OS threads. The object system is different, but it's front-and-center.

Clojure treats objects as something you make for unavoidable interop. It has transactional references and snapshot values. It emphasizes functional style. It has a time semantic - something you can also say for Scala, Erlang, and Go, but not CL or Java.

My preference is for Clojure, but for a Java coder it will feel a lot more alien.

The object system [in CL] is different [than Java's], but it's front-and-center.

No, it's very much off to the side. I almost never use CLOS. While it's far better than static OO, it's still OO, which means you still get object spaghetti. The beauty of CL is that its object system is basically just a library and you don't ever have to go there.

Side note: it still surprises me how few people have figured this out about OO. Yesterday I was browsing through the source of an OO-style JS library and it didn't take 30 seconds before the old familiar frustration kicked in of having to wade through a morass of methods-type-declarations-and-constructors in order to find the simple line of code I was looking for.

People used to talk about learning to think in objects; unlearning to think in objects is way better.

Both Clojure and CL can do those basic things, now which one do you know best?

This feature-based language choice is meaningless if you're not taking your own strength into consideration.

Clojure is cleaner than CL, but SLIME does not appear to work as well with Clojure as it does with CL. For example, if you have a compiler error, you have to navigate to the location of the error yourself. If you have a runtime error, you get a Java backtrace, that SLIME does not help you with at all (good luck). This is something you should consider, because it will have an effect on your development speed.

I was in a similar situation: I love LISP and I wanted to use it for web development. I spent several days looking at Clojure and Compojure for this purpose but came to the conclusion that it was far too restrictive and cumbersome for rapid web development.

Then I discovered the Dragonfly framework, a web framework for the newLISP scripting language and became its co-author.

Advantages over Clojure/Compojure:

- No need to switch web servers to Jetty/TomCat. (runs as cgi).

- PHP-like development, no need to recompile/reload code.

- Extremely dynamic and flexible routes.

- Very fast and very low memory usage.

- newLISP is an awesome language.

http://www.rundragonfly.com/

http://www.newlisp.com

Questions to the Dragonfly forum:

http://www.newlispfanclub.alh.net/forum/viewforum.php?f=17

It's still young and is being actively developed, but I think if you give it a look over you'll find much to love about it.

Reply on news.ycombinator.com