Clojure: It’s About the Libraries
stuartsierra.com
stuartsierra.com
It is the most explicit to date abandonment of the age-old Lispers' Dream, which was "Lisp All The Way Down." Clojure is the antithesis of the Lisp Machine. Rather than a crystalline pyramid of comprehensible mutually-interlocking concepts, behind every Clojure primitive there lurks Black Magic. The Clojure user who is missing some routine or other will run crying to Java for help, rather than implementing the feature himself correctly - that is, as a natural part of the entire language, built on the same concepts. Clojure pisses on everything I've ever loved about Lisp. It rejects even what little consistency and intellectual rigor there is to be found in an abomination like Common Lisp.
Clojure is the False Lisp, which Reeketh of the Cube Farm.
I don't care if everybody really is more productive in Clojure than in Common Lisp. The latter is not my standard of comparison (Symbolics Genera, the state of the art, is. Or, if you want something that is distinctly inferior but is currently available on the market: Mathematica.) Clojure is the gutted corpse of a dream.
I am tired of this abomination being hailed as the future of Lisp. Users of real Lisps, such as myself, will keep hoping, dreaming, and working on systems which do not betray the original virtues of the language.
If you got all of the revolutionary Lisp ideas, you don't have to worry about the rest -- currently I'm more or less forced to use C++ instead of CL (which is still my favorite language) -- but this doesn't prevent my mind to still 'think' in Lisp.
If you have some really revolutionary idea, and you want to break it through the history grown walls, you always need to introduce it in homeopathic manner -- but since your new idea is really, really much more ingenious than all of the previous ones, it will finally and actually replace the more limited ones.
But this takes time.
If you limit yourself to think either black or white, you will never win...
And to answer your post, I'd say that Clojure does give you what you need - it gives you the axioms, it gives you defmacro - the rest is history. All those other library functions you can think of in a more abstract sense ("George's problem"), and in the end these should be machine code no matter which HLL they were originally written in.
The Lisp way has always been very dynamic - you can change things, even when they're orbiting Mars, but Clojure is functional, you generally don't change things. So in that sense, Clojure is not a true Lisp, but only because it chooses a different set of trade-offs, and if you don't like that then its fine, but it doesn't make Clojure evil, just different.
"Lisp All The Way Down" is when you can select any part of the system - and the full source (written in the same basic concepts as the rest of the system, including the program you are writing) of the element will pop up. Any changes you make will take effect immediately and in real time. We had this in 1985. Why can't we have it now?
> it doesn't make Clojure evil, just different
The language is being promoted as the long-awaited, Messiah-like successor to Common Lisp. That is the main source of my annoyance.
sigh
Rich Hickey explains what he thinks are the good parts of his language, but I would hardly say he has a messianic complex about it. A lot of people (myself included) like it because it combines interesting ideas with the pragmatism of fitting into the popular Java ecosystem. Perhaps a desire to have everything converted to a single language is more indicative of messianic thinking?
Common Lisp is a wonderful language, certainly worth learning. But it somehow seems to induce a bunker mentality in its users.
But why is this a desirable dream? It seems to suggest no other languages or technologies have any merit, except Lisp. Unix and C are brilliant pieces of work, and Lisp's hang up of refusing to work well with them really set Lisp back. Compared to, say, Perl, Python and Ruby which really embraced Unix concepts (pipes, scripting C programs, etc.).
I happen to think the JVM is a nice piece of work, too, and I'm happy to see a Lisp that works seamlessly with it and benefits from its strengths. Cooperation and interoperability are good things.
"which Reeketh of the Cube Farm."
Cube Farm coders deserve good programming languages, too.
I beg to differ:
http://www.art.net/~hopkins/Don/unix-haters/handbook.html
> pipes, scripting C programs, etc.
None of these things are necessary when every piece of information on your machine lives in a Lisp world image. As it should. And would, if the machines we are now using were the descendants of real computers, rather than of early-1980s children's toys.
http://www.jwz.org/doc/worse-is-better.html
"None of these things are necessary when every piece of information on your machine lives in a Lisp world image."
http://www.artima.com/weblogs/viewpost.jsp?thread=24807
And your last link would be applicable, save for the "ocean" actually having been "boiled" in this manner. The Lisp Machine existed. You cannot un-create it. It proves that all of the supposedly impossible or impractical pipe dreams of the Unix haters are in fact possible.
So, C is better than Lisp, depending on your metrics.
As for the ocean, it is no longer the machine on which you are typing your program. It is all of the information, data, and code available to you on Internet. That's a lot of ocean to boil. Any Lisp needs to work well with as much of it as possible to be considered a useful language today.
A camel is also a better vehicle than a truck, depending on your metrics. (Quite seriously.)
For the sake of your career as a programmer (assuming this is your trade), you'd better hope not.
What makes you believe Lisp is so superior to other classes of programming language? For instance, what makes a Lisp superior to a stack-based language like Forth or Factor?
Because I feel straightjacketed whenever I cannot redefine my language on a whim in every conceivable way, or am forced to write a whit of boilerplate. (Yes, none of the existing Lisps fully meet this ideal, but nothing else comes close.)
And for that matter, why do you think Clojure is so awful? It has no reader macros, true, but it does provide a more succinct way of writing hashes and vectors than most other lisps. Other than that, well, it's a Lisp.
Having to juggle the stack at every step (i.e. FORTH) is a mental burden. It forces you to organize your thoughts in a certain - not necessarily harmful - but very specific way.
> Other than that, well, it's a Lisp
A Lisp unworthy of the name; one which encourages users to pepper their code with opaque routines which have no underlying Lispiness. A Lisp which barfs Java stack traces.
I gauge programming languages by their effectiveness at solving practical problems. Do you believe Clojure's design makes it less effective at solving problems than other Lisps? Or are your objections to it purely ideological?
Bu I will try to explain my real objections.
"Beauty is the first test: there is no permanent place in the world for ugly mathematics." (G. H. Hardy)
I was drawn into programming at an early age because it seemed beautiful and allowed me to run wild with creative ideas. Thus I insist on systems with a purely minimal amount of accidental complexity - so as to leave the most room in my head for the ideas I am trying to put into practice. I have zero interest in "programming as construction work" - implementing the ideas of others, or for that matter doing anything which isn't in some important sense original. Thus I want my programming environment to resemble pure mathematics, where absolutely nothing is present for which there is not an ironclad and inevitable logical reason, where everything is open to question, and the answer is never "because the drunken intern wrote it that way five years ago."
Most of the "practical problems" programmers supposedly "solve" shouldn't even exist. They are created through the idiocy of other programmers. In particular, I am talking about any situation where the output of one program requires "glue" to match the input requirements of another; or any situation where formats are converted; and many other cases. If the moronic mistakes of the past four decades were undone, most of the people who are calling themselves programmers would have to find honest work.
Screw "practicality." Programmers are the proverbial firemen who start fires when they aren't at the station-house, and gloriously put them out, hailed as heroes every time.
Clojure will never be beautiful because it promotes the use of undigestable foreign matter in Lisp code: primitives on which you cannot pop the hood to reveal more Lisp, with turtles all the way down. Its claim to the name Lisp is in an important sense fraudulent. I rest my case.
I think I disagree with you on two points. I place a higher emphasis on efficiency than I do aesthetics; I want a language that allows me to do things quickly. I also have a slightly different idea of programming language beauty than you do. I like languages that have an small but expressive syntax and a concise core library. Clojure fits that definition better than any other Lisp I've found, so from my point of view, it's the most beautiful Lisp as well as the most practical.
But aesthetics are subjective, which is why I dislike arguments about them. I don't particularly care what language a library function is written in, just so long as it integrates elegantly with the language I'm working in. You might think that using external libraries is aesthetically displeasing, but that's a subjective opinion.
What I am interested in discussing are technical details about a language, as they are entirely objective. We can discuss whether a particular language produces more concise code than another, and come up with examples to prove it. I like those sorts of discussions, because even if I don't agree, I'll always come away with a better understanding of the favoured language of the other guy.
By contrast, arguments over aesthetics just seem pointless, as neither participant will learn anything or change their view.
From Wiktionary:
1. An outcast
2. A demographic group, species, or community that is generally despised.
3. Someone in exile
4. A member of one of the oppressed social castes in India.Thanks for correcting me :)
I think this is the core of our differences. Lisp has traditionally been used mostly as a language -- as Dijkstra put it, it "has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts". Most other programming languages have been used as tools.
Clojure looks like a perfectly competent Lisp-on-JVM (with immutability!), but I don't see it changing how I think about anything. It's thoroughly on the "tool" side of things, which understandably leads to people in the "language" camp being miffed that anybody thinks it's comparable to classic Lisps from the "language" camp.
The Lisp engine in Abuse was a tool, too, and highly productive for its uses, but at least nobody claimed it was a successor to the Lisp line.
It's true that Clojure is a programming language designed to be practical, but I fail to see why this would make it any less useful as a language for communicating ideas, or any less able to change how people think about things.
Could you provide an example of an idea that is more easily communicated in another Lisp, rather than in Clojure?
The immutability is a non-trivial change in thinking about how to write a program. You can write a program without mutation in Common Lisp, and you can mutate things in Clojure, but the greater cost of implementing mutation in Clojure means you are less likely to do so. This leads to different programs.
No, that is not as big of a change in thinking as Lisp was compared to the languages of the day when McCarthy created it. But I find it a fine addition to the family of Lisps, nonetheless.
Attached is my official Smug Lisp Weenie card. I have no further use for it. Please pass it on to someone who is worthier than I am.
Regards.
If only. If I could get something like Python's Numpy on the JVM, I'd be using Clojure on my current project(s).
Colt: http://acs.lbl.gov/~hoschek/colt/ Parallel Colt: http://sites.google.com/site/piotrwendykier/software/paralle...
There's been talk about integrating this functionality into Clojure. Incanter is an example of a project by someone who wanted to get primitive support for matrix math and has had quite a bit of success.
How do you know I don't do my research by making provocative statements on public fora?
There is something to be said for that. I would not have thought to Google for "R-like interactive environment, built with Clojure and Java charting and numerics libraries." But now that I know it exists, I'm pretty excited about the possibilities.
You don’t need primitive-static-method in Kawa as of several years ago. It’s just there for backward compatibility. Newer syntax for calling Java methods is much easier:
Tail call optimization? Scheme is a cruel parody of itself without it.
However that being said, the real reason why Clojure is going to become the most important Lisp is the same reason why Java has been displacing the theoretically-faster C++: multicore.
Threading the locks have been integrated into Java from the beginning, with every object having a monitor and functions like wait() and notify(), and a synchronized keyword. These are nice, and do help, but they don't solve the problem. I've heard that Java fixed up some problems with these in version 1.5 or 1.6, but haven't looked at the details.
My understanding is that none of the imperative languages are very good at multicore. Erlang is said to be effective at it (because of pure message passing); and some people claim FP will help because it overcomes the problem of shared state by making it immutable (while this is true, I'm not convinced it's particularly helpful in large projects, in practice).
So I'm interested in your reasoning.
Without a memory model you're going to be writing more code that is less correct at a substantially higher cost. If you're interested in this google for a Google Tech Talk by Josh Bloch on 'Java Memory Model'. This basically comes down to my ability to get at architecture specific stuff (CAS, fencing) in a platform independent way and have it work. Cliff Click has a really neat tech talk on writing a lock-free hash table in Java, don't bother trying to write it in C++.
Ultimately we will need a better model for concurrency, which will be some combination of functional programming and lightweight processes (actors). Erlang/Scala/Clojure/Haskell are in the right neighborhood (maybe Gambit Scheme, don't know much about it), but Erlang and Haskell are really hard to teach to mere mortals like myself. I don't know enough category theory to understand 'hello world' in Haskell, but I can write Scala and Clojure programs that work (and leverage my existing investment in the JVM).
edit: i left off something imporant. if anything saves imperative programming it will be transactional memory, which is exremely hard. hardware TM and code written for TM are in a chicken-and-egg situation. the answer is hybrid hardware/software TM and the sun guys are way ahead on this too.
What's interesting is STM is only hard in imperative languages. It's relatively straightforward in Clojure and Haskell. See http://enfranchisedmind.com/blog/posts/the-problem-with-stm-...
<brag>Now seems like a reasonable chance to brag that Doug Crockford signed my copy of BC the other day! God I'm such a geek.</brag>
http://delivery.acm.org/10.1145/1410000/1400228/p40-cascaval...
Edit: Prettier version http://mags.acm.org/communications/200811/?folio=40&CFID...
Incidentally, I have an entertaining alternative theory, that silicon will becomes specialized in the way that organs in the body are (and firms), perhaps mimicking the architecture of large programs which have intercommunication patterns that are established and predictable. A games examples: a Lua CPU, a physics CPU, an AI CPU. The Enterprise is even more ripe for this, as they have been trying to separate architectural components with SOA (which is a form of concurrency). It doesn't seem to have gone well, but something may have been learned there. Another example of large-scale concurrency is the internet: perhaps replicate a google server farm on a chip?
These ideas are a bit horrific, because they are horribly inefficient through standardizing on large scale components, so we lose the opportunity to optimize between them. But they work.
For any long-running process the JIT is going to turn your Java (Clojure, Scala, ...) into extremely well-optimized machine code. In fact, the amount of knowledge that the JIT has about the actual run time behavior of the software allows it to do better optimizations than you ever could by hand in certain situations (an ever-growing number of situations).
GC is another point, the HotSpot GC is arguably faster at allocating small, short-lived objects than malloc is. It is reeeaaaaallllyyy good these days, probably the best GC that you'll find in production just about anywhere these days. The thing is a monster. And the new one, the G1 collector, was sent from the future by SkyNet to kill John Conner.
Add to this the fact that you can easily exploit common multiprocessing patterns (thread pool, consumer/producer) with almost no code, and that advanced multiprocessing code can be written in a platform independent way, and it's a pretty small subset of applications that still make sense to write in C++.
Add to all of that the fact that even Java (let alone all the other amazing JVM languages) are orders of magnitude more productive than C++ for your programming staff, and the set of situations in which it makes sense to use C++ rather than the JVM is looking pretty tiny.
Oh and then there's portability. C++ isn't even portable across compilers (versions of the same compiler!), and for all the early trouble with Java portability, it really is write-once-run-anywhere now. Really.
Java has many benefits as you mention, especially ease of development, and I think these are what drive its adoption rather than any multicore benefit. So, I accept your (a); and (b) certainly has some truth, but I doubt the causality of (c).
The stats (terribly unreliable) show that C and C++ combined are ahead of Java, and the proportions are fairly static (C and C++ are steady, but Java is itself being javaed by even easier languages like Python (which incidentally supports my reasoning above): http://www.tiobe.com/index.php/content/paperinfo/tpci/index.... (about as reliable as a slashdot poll, but it's probably basically right).
I would respectfully contend that outside of micro-benchmarks (which kill Java because of long startup time and no chance to JIT) that Java is as fast as C++, and in the presence of lots of cores probably faster. This is again based in large part on my direct anecdotal experience, I would be very interested to hear about everyone else's experience in the matter.
But the real point is that with Moore's law now working on cores instead of mhz, single-threaded performance just doesn't matter that much going forward. Exponential increase and all that.
For a look at where this shit is going take a look a look at Azul.
You're making an interesting case. I haven't heard any claim that Java actually is faster than C++ in practice, just that it theoretically could be, because of JIT optimizations (then again, I don't think C/C++ is used that much in the server, anyway.) I also haven't heard of Java getting a comparative advantage due to multicore, just that FP would. But that's not to say it isn't true. You could be right. Interesting.
(The link seems to be commodity computing, like Sun blade servers).
+1 thoughtful and respectful debate.
On further investigation of a 150 million element dataset, Java was performing 1/3 again as many instructions, but missing cache half as much. I've been spending some time looking at all the JVM performance optimisations, but I still haven't worked out why this is - a random binary search isn't exactly a test case that's run time optimisation-specific. It's fascinating stuff, anyway.
edit: on iPhone so I can't post the link, but the comments section of cliff's blog has a discussion about java vs. C for this data structure. summary: maybe you could do it but it would be tricky and extremely non-portable, I.e. almost definitely not worth it. hope I didn't overstate, wasn't trying to dismissive (need to watch my language online).
Click was probably talking about the fact that Java has a consistent and defined memory model. C and C++ have no such memory model, but there always is one. It's just the memory model of the underlying architecture - which is sometimes extremely subtle.
No, just Haskell. Learning Erlang is roughly on par with learning Javascript well; it may involve a couple of new paradigms (like Javascript imposes event-oriented programming and prototypes, which may be new to you), but the language itself is straightforward. Different, but straightforward. The concurrency may blow your mind, but the syntax and basic mode of operation is unlikely to do so, in contrast to Haskell.
Don't be scared away from Erlang.
Huh? What do names have to do with it?
If I recall correctly, first and rest were introduced 6 months after car and cdr were. The difference between 50 years and 49.5 years is hardly great enough to have affected the subsequent evolution of Lisp.
The difference is that first, rest et al. are used when the data being handled are proper lists. A list of items terminated by nil.
Car, cdr and others can be used with lists as well, but they're more primitive operations that mean cons cells; a pair data structure that has two parts. You can view a proper list as "a primitive pair data structure that has the first item of the list as its first cell, and has its second items a pointer to rest of the list" .. and construct the rest inductively.
You can't use first and rest to operate on associative lists, graph structures cyclical or otherwise and a host of other data structures.
Some programmers decide to make their intent clear when operating on proper lists, while others prefer to stick to car and cdr out of habit, nostalgia, not-knowing better, street cred, or as a cheap way to advance pointers with one compact operation.
CAR and CDR are less relevant now that people use CLOS, CL's Object System, and to a lesser extent structures. Defstruct gives you the accessor FREE, it constructs the accessor functions as classname-slotname for the reader and (setf classname-slotname) for the update/write function. With CLOS you can choose the name for it with an initialization argument (:initarg :accessor, :reader, or :writer)
Please update your Common Lisp prejudice check-list.