Clojure or: How I Learned to Stop Worrying and Love the Parentheses
nathanmarz.com
nathanmarz.com
The irony here is that lisp's syntax will probably keep it from ever falling into obsolescence because it can adapt to any new general task but it will also prevent it from ever really breaking into the mainstream because for any particular task it's a non-optimal syntax and a steady procession of more specialized alternatives suit most people's needs better.
Now that I'm fluent, I strongly prefer the s-expression style for most problems. The uniformity of Clojure means that I can use the sequence library for an unbelievable variety of tasks. And I love that operations like "+" are just like any other function and can be passed around.
Another important point is that when you make a DSL in Lisp, that DSL can interoperate with all your other DSL's.
Syntax matters but it seems to have become a bit of an obsession lately.
If you find that your language still requires boilerplate code to implement features in whatever domain you're working in, then perhaps it's time to get a new language - or write a DSL.
I would have appreciated a DSL which could obviously be used as a way to query an SQL database more than the examples in the article.
I share his interest in Clojure, but some little things like poor stack traces still keep the experience of using Clojure from being totally fun. I use Clojure for work, but I am still mostly using Ruby for my own projects (with some Clojure).
Being a lisp doesn't hurt it, especially in the context of DSLs mentioned, but I don't think it defines it either.
Incidentally, one could argue that SQL is the most widespread DSL there is.
Edit: The author mentions this is basically a DSL in his post, which I missed the first time around. Not sure why he didn't just stay with that nomenclature though.
The "integrated languages" that are discussed do not need to be stored in strings, or have some sort of special pre-processing step to coexist with the primary language. In Clojure, a Cascalog query is a first-class structure simply by virtue of being defined in terms of Clojure syntax. Integrating LINQ into C#, on the other hand, required modifying both the compiler and IDE. And as pointed out in the OP, SQL is a second-class citizen pretty much everywhere you look, which leads to issues like injection.
(1 + 2) (x = 1 + 2)
It needs to be something like
(!! 1 + 2)
And you need to have spaces.
(1+4/5)
Won't work.
And operator precedence is do-able, but a bitch.
Etc... Etc...
As for the issue of spaces, 1+4/5 is just a symbol, which you can interpret as a string, which you can then parse back to symbols.
All that said, if I really wanted infix math, I would probably just make something that interprets this:
{1 + {2 / 3}{4+5}}
(!! (1 + (2 / 3)(4+5)))You can write a parser in any language; nothing magical about Lisp there.
Your statement is correct; it is just misleading. First, it is irrelevant in that we are talking about parsing the syntax of your language and producing code; not "any language" can do that. You would also have to write an eval for the many languages that do not have an eval. See SICP for an introduction to eval-apply logic.
Second, it disregards ease of parsing. It is easier to write a parser in Lisp to translate Lisp-like syntax into Lisp code than to write a parser in another language to translate that language's syntax into that language's code (and exceptions to this are because the language was based on Lisp). This has to do with basic Lisp syntax being a syntax tree and is aided by the CL standard providing many tools for parsing.
So yes, you can complicate things a bit by mashing your symbols together, but that's just one extra parsing pass (insert spaces around operators). One READ-FROM-STRING later, and you have a symbol tree. In another language, it might be a series of complicated lexx statements and functions where the language relearns how to do simple addition.
Yeah, that's a neat thing about lisp, but it wasn't what the OP really said - he was talking about parsing a random string.
Incidentally, I would be curious to get a design guy - one who really knows little about code - to look at blocks of code in different languages and give us his opinion. I have suspicions about what he might say, but it'd be a fun experiment.
(def && #(and % %2))
(def || #(or % %2))
(def *rank* (zipmap [- + * / < > && || =]
(iterate inc 1)))
(defn- infix*
[[a b & [c d e & m]]]
(cond
(vector? a) (recur (list* (infix* a) b c d e m))
(vector? c) (recur (list* a b (infix* c) d e m))
(ifn? b) (if (and d (< (*rank* b 0) (*rank* d 0)))
(recur (list a b (infix* (list* c d e m))))
(recur (list* (b a c) d e m)))
:else a))
(defn infix [& args]
(infix* args))
And of course, this isn't an attack on lisp. If it works for you, cool. But I just don't see how that distinguishes itself from a ruby DSL like say rspec.
I think the nicest feature of lisp DSLs is that lisp doesn't have the concept of an operator, and most langaguages make if difficult to define a function called or <- or something like that. But you can do that in Haskell or or ML, it's not really exclusive to S-exps.
In most other languages, if that sort of transformation is even possible, it is much less direct.
I'm not sure why people repeat that, but Lisp has syntax ...
In the same way Xml, Yaml, Json have syntax ... plus it has operators that do stuff, like: quote / eval / for defining methods / for defining macros.
Sure, a serialization format for what are basically syntax-trees is lighter than for a language that requires a LALR parser ... but you have to limit yourself to that serialization format in your DSLs.
Ultimately, you get the thing to behave how you want with the syntax you want, but under the hood you're not getting code as data that you can manipulate as freely and naturally as you can in lisp. Instead you hack on the language's object models and method dispatch to get the syntax to line up with the one you have in your head.
In this sense, beauty is only skin deep in Ruby when it comes to DSLs. (It's nice and all, but it's apples to oranges vs lisps.)
Here's the LINQ syntax built as a Nemerle macro ... http://code.google.com/p/nemerle/source/browse/#svn/nemerle/...
Where a query looks like this ...
def res = linq <# from c in customers where c.City == "London" #>;
Here's MooseX-Declare, which introduces completely new syntax in Perl ... http://search.cpan.org/dist/MooseX-Declare/lib/MooseX/Declar...
So the argument that you need Lisp-like syntax for adding syntax to a language without storing it in strings or through pre-processing ... is bullshit.
The only difference between a language like Nemerle and Lisp is that in Lisp a macro is easier to write, while in Nemerle you need some knowledge about the compiler's parser / AST and pipeline.
But having easy-to-write macros ... I'm not sure it's such a good thing ... and in Nemerle you're not limited in the kind of syntax you can add to the language. In fact Nemerle is a simple core language where many of the useful things in it are built as macros.
Really? I think someone even wrote a whole book that's mostly about macros, embedded languages and so on.
What I mean by "articulate the benefits" is a short explanation for the unique benefits of Lisp that a non-Lisper can digest. That's what I tried to do in this article by trying to show a tangible example of an embedded DSL.
PG's post "Beating the Averages" is a better example, but its argument is more based on authority than clearly communicating the tangible reason why macros are so useful. That said, PG's posts are what got me interested in Lisp in the first place, but I didn't fully understand the benefits of Lisp until I started using it heavily.
http://www.serpentine.com/blog/2009/12/17/making-ghcs-io-man...
The one of the fundamental part of the Lisp philosophy is to keep a syntactic sugar away, while Clojure is a collection of various syntactic sugar on top of something which looks like Lisp's syntax.
The statement that "Clojure is a dynamic programming language that targets the Java Virtual Machine" is true, while "Clojure is a dialect of Lisp" is just a very-very clever marketing statement.
^&~@%#{}[] - what all this shit is supposed to be? One must learn it. With all that crap Clojure is just another language, which locks like Lisp, especially for those who never saw emacs-lisp or Arc before.
http://groups.csail.mit.edu/mac/classes/6.001/abelson-sussma... - Lecture 1b: Procedures and Processes; Substitution Mode.
By far, the cleanest Lisp dialect out there is Dylan, and it uses Algol notation, not s-expressions:
http://en.wikipedia.org/wiki/Dylan_%28programming_language%2...
The reader of the code should not be forced to stumble over a semantic identity because it is expressed by a syntactic distinction. The reader's focus should not be directed toward the lexical tokens; it should be directed toward the structure, but using square brackets draws the reader's attention unnecessarily to the lexical tokens.
But it is not only about a style. When [x y] means a different thing than (x y) it is a different language construction, with different behavior.
So, the code snippets in this article aren't written in Clojure, but instead in some other custom language? I dont know Clojure, but those examples look very much like that crazy foreign Lisp 'nested parenthesis talk' to me.
Seriously - I don't see what's unusual about the syntax of these examples that makes them a DSL that doesn't just look like clojure code.
If my Linq implementation is allowed to resemble the language for which it is implemented, then I can easily write a 'Linq' implementation in plain C# that will look just like plain C#. (Although I grant that LINQ refers to the syntax additions to C# and not just the library)
Not hugely worked up about this, just feeling a little 'Emperors New Clothes' about it.
http://www.unixuser.org/~euske/doc/cl/loop.html
(It could be implemented identically in Clojure, if you cared to.)
Here's another example that might look "DSLy": http://www.brool.com/index.php/pattern-matching-in-clojure
The pattern matching macro "match" is written entirely in Clojure, without any compiler tricks. I wish I could do the same in C#! (I'd need to have a bunch of cruft defining a predicate object with options for each match criterion.)
All of these techniques boil down to how easy it is to do two things in a language:
- To write DSL code in your structure of choice without having to cover it in a huge amount of syntax goop, e.g. the loop macro or the magic LINQ syntax.
- To analyze and transform that DSL code programmatically, in a way that's more structured than simple string manipulation, so that you can do the right thing with it.
C# 2.0 was very bad at both of these. C# 3.0 improved a lot, by adding lambda expressions for the first, and allowing you to analyze C# expressions as a syntax tree. But Clojure (& Lisp in general) is really good at both.
The point he's trying to make is that the default way of doing things in a lisp is to adjust the language constructs to fit the domain of the problem. This (generally) results in a simpler mental model of the problem domain and less code. It's not a about being ABLE to do it. You can do something similar in most languages, but there it's just not as easy, not the default, and not as flexible.
Guys, honestly, downvote if it makes you feel better.
Straight off the bat the most glaring problem with that statement is that Linq is not part of C# in any way shape or form since all C# code is compiled to bytecode, it would be literally impossible for C# to have Linq and for it to not be available for the rest of the languages supported by dotNet. The funny part is the expressive form of Linq that reads like a sentence is not even fully supported by C#! Only VB.Net has fully implemented Linq expressiveness.
There are many more reasons that argument is wrong, but I feel just pointing out that one above shows how far off it is.
In my experience its almost always best to shy away from hyping your idea of better by knocking the competition, your article stands well on its own and C# has so many obvious flaws that theres no need to add to it. Stick to whats good about X, not whats crappy about alternative Y.
Again all in all an interesting read since im not familiar with Clojure, but I just had to nitpick.
In Clojure, you can create embedded languages without modifying the compiler. That's all I'm saying.
Can you name the compiler change that enabled Linq without looking it up? Can you define Linq, what it basically is? It feels like if you could you wouldnt make such a statement since all Linq is at its core is an iteration engine. Its literally just methods you dump your collection into plus an anonymous method on top and vrooom goes the engine applying the method to each item. The pretty syntax form you usually see is not actually Linq the framework and leads to significantly more problems than it solves, its basically just for PR purposes.
Edit: This feels like it might get out of hand and spin into a good old nitpicking programmers war. Reading my post again it came out way more confrontational than I meant, and I didnt mean any insult by it. My main contention is that the Linq syntax is often confused for the Linq framework - they are not in the same state let alone ballpark. Plus the argument can be made (and i would agree to a large extent) that it wasnt the compiler that was modified to allow Linq, it was Linq that was waiting in the wings for the compiler team to implement features they had planned quite some time before. However I am also being a stickler and stubborn, I nitpicked when even from my point of view it wasnt such a large error - it was the way it disrupted the flow of a pretty good article I was getting into, thats what made it stick out for me.
None of these required modification to the runtime, they were purely compiler features. And, as pointed out, they were features only Microsoft had the ability to implement.
(A better comparison would be to look at LINQ-to-SQL specifically; that would not be possible in any sense without the expression tree libraries and compiler support introduced in C# 3.0, since there was no way to "quote" a C# expression and look into it. That's much closer to the mark here.)
However, I agree with you in spirit. A big enough quantitative difference becomes a qualitative one; nobody would actually want to use LINQ+C#2. And likewise, few people want to write small-scale DSLs in C#.
But I would argue that since DSLs are all about affordances, it's not sufficient to say that similar functionality would be "possible". If the new approach doesn't represent significant semantic compression (which your hypothetical LINQ+C#2 would not), no one will use it. By that measure, the syntactic sugar added to C# 3.0 was absolutely a necessary precondition for LINQ.
I'm not even criticizing Linq/C#, I'm just using it as a point of comparison to help the reader understand Clojure concepts. How that comes across as combative I don't understand.
However, expression-based LINQ is implemented in C#. That's possible because of several language features added to C# 3.0, notably expression trees (code as data) and anonymous functions (lambda). Those features make C# expressive enough to do things like LINQ in C#, though in a somewhat clumsier way than Clojure does it.
For example, if you write cities.Where(s => s.StartsWith("L")), that "s => " is a lambda expression, but because the Where method takes an expression tree, the expression is turned into a data structure rather than executable code. This is similar (not identical!) to Clojure recognizing that you're calling a macro rather than a function and letting you see code as data.
To start splitting hairs about exact definition of what was written in the article seems to miss the point to me, the general flow and feel was dismissive of its ability. Is it clumsier? From what I can tell so far yes its clumsier, but its not powerless and immovable which is how it came off. Maybe im just sensitive though, or maybe im insensitive in how i portrayed my argument, but I really did just meant my original comment as constructive criticism.
There's no such thing as "LINQ bytecode". LINQ is just syntactic sugar that can coexist with vanilla C# (or VB.NET, or whatever) because Microsoft decided to modify its compiler and IDE to allow it. It is fundamentally impossible for you or me to make a similar change, unless we're willing to eschew the Microsoft toolchain.
In Clojure, you do not have the same limitation. That's the only point the article was making, and it's completely correct in that respect.
Parts of it are. Query expression keywords[1] are defined in the C# 3.0 Language Specification (see 7.15 Query expressions)[2].
[1] http://msdn.microsoft.com/en-us/library/bb310804.aspx
[2] http://www.microsoft.com/downloads/details.aspx?FamilyID=dfb...