Lisp in 3 words
vagrant-coder.com
vagrant-coder.com
It's hard to explain this to other people, however. It's doubly hard trying to explain it to people who have spent their careers building up high levels of skill and knowledge in other languages.
I'm not a professional coder-- I'm a hobbyist. I can program badly in many different languages. But I find that with LISP, I am much more productive and happier. I feel like I can "punch above my weight" using LISP, achieving things that would take a lot more skill and effort to do in other languages.
I sometimes wish I could explain it better to other people, but other times I'm just happy to work on my own in my "obscure" language with the all the silly parentheses.
The lower bound of code quality is limited not by choice of language, but only by what a given compiler or interpreter will flatly refuse to put up with — there’s always a better fool. The upper bound, on the other hand, is highly susceptible to such limitation.
Specific code quality is not what I'm talking about. I think you've misunderstood me. Let me try and be more specific and concrete.
The degree to which code written by different authors at different times can gel together depends on the idioms they have in common. In languages without common collection libraries, code needs to communicate at the level of arrays and custom iterators. Languages without a single common memory allocator need have libraries negotiate who owns what memory, and have custom free functions specific to routines and libraries. Languages without lambdas typically reinvent the concept a half-dozen different, incompatible ways.
The amount of friction when trying to compose code or integrate with existing code is proportional to its divergence from the lowest common denominator of all code written in that language. The higher-level a language is, the larger a standard library it has, the more concepts that are shared between libraries and codebases and don't need to be reinvented in each individual code silo.
Go back to C in the dark days of Windows 95. Merely getting a string out of the OS was an ordeal; you could choose to allocate a buffer, send it to the OS, but always prepared to react to a failure and reallocate the buffer when the string turns out bigger than expected. C has no universal string type nor a common memory allocator (you might rely on glibc's malloc on Linux, but you're relying on non-portable convention). This directly resulted in a substantial amount of friction simply interoperating between two binary libraries.
The same thing follows through with higher-level concepts. When higher-level concepts are concrete in the language or available in standard libraries rather than composed out of simpler primitives a half-dozen times in slightly different ways, you lose a bunch of composability. You spend more time writing shims and adapters to smooth away the differences between interfaces.
And code isn't the only interface; your mind is too, especially in a larger legacy codebase. You may understand iterators, lambdas, functors, monads and so on, but in a language without first-class support and standard definitions for these things, you'll find them reinvented over and over again, to varying degrees of fidelity, sometimes under obscure names, which you'll need to discover and research as you dig into code nobody still in the organization has ever had to dig through themselves.
None of this has anything to do with code quality possible in a language. I've had to maintain some extremely high quality assembly. But it wasn't mentally cheap to get into, and a lot of concepts had to be reverse engineered from the code before they could be modified and transformed back down.
The risk with something like Lisp is that you use it as a language construction toolkit. With a closely knit team that grows a codebase amongst themselves, this can be a huge productivity win. But I don't think it scales. And I think history agrees. Excessively malleable languages (including Smalltalk) have not outcompeted less expressive languages, for one reason or another.
Coding standards are needed whether you're using C++, Java, python or lisp.
If you have coding standards and peer review processes, then lisp's flexibility is not going to be a liability.
When I was developing on a quite malleable and a bit unpredictable language, I wished it would not be like that.
When I was developing on a less malleable, more predictable one, I was thankful it was like that!
I think it makes sense when you view a program as a cultural artifact. If a programmer has much freedom, so much that he is able to define his own conventions down to how to code the tiniest thing, he should really work hard to pass this "culture".
Unfortunately, as we all know, we seldom have the time and the means to share views about the code. So, indeed, the more universal the idioms, the less we jeopardize the understanding of the code by our peers.
(let ((...)) (f (g (h u v)))
or
let u = ... in f g (h u v)
I'll take any of these two any day.
What's special about the former one is that it composes perfectly with any other tree structure using the same textural representation of trees, where the latter one is only useful in isolation - one cannot use this representation inside some other arbitrary character sequences which is to be converted to trees, because the means by which it is parsed into a tree does not compose well (unambiguously) with other means of parsing sequences of characters into trees.
Perhaps the cleverest thing about S-expressions is that they were invented before we even had a theory of parsing unambiguously followed by a proof that we cannot compose multiple of these unambiguous parsers in a way which produces another unambigous parser.
But I'll add one delightful trait: sexps split usually complex parsing stage in two simple separate ones:
- chars -> sexp
- sexp -> meaning (special forms / anticipated mental evaluation).
At least not with plain text. There's some terrfic research in composing languages - currently the best is Tratt and Diekmann's Language Boxes, which enable arbitrary syntax composition by means of the programmer specifying the syntax boundaries explicitly - which requires a more sophisticated editor than out plain text notepad variants.
With language boxes, and their prototype editor Eco, in the end, they're ultimately storing a tree of data in their own custom format - it doesn't matter what encoding is used, just as it doesn't matter whether lisp uses parens, braces or whitespace - what really matters is that you encode things deterministically into trees which can be easily reasoned about without requiring some "magic" (AI) which attempts to create a tree from a sequence of characters unambiguously, when the programmer knows and can specify exactly what he means.
Ah, and I just realized, it wasn't Parr, but Tratt that wrote the paper "Parsing: The Solved Problem That Isn't".
Is there something as easy and small as Processing[2] in LISP form? Because the biggest criticism of Processing is also it's strength: download it, run it, and you have notepad with a compile-and-run button. You need something as un-intimidating as that for beginners of non-technical backgrounds. That's part of why we're using it as the first language in our design programme in Malmö University (apart from the fact that it's also a gateway language into Java proper, which is a useful language to become familiar with because of its widespread adoption).
EDIT: Here's a short blog post comparing the two languages and IDEs: http://www.hive76.org/processing-and-racket
That post led to a big discussion on the Racket mailing list. And the suggestion for this came up: http://www.pawfal.org/fluxus/
A 3D game engine for livecoding worlds into existence.
Fluxus is a rapid prototyping, playing and learning
environment for 3D graphics, sound and games. Extends the
Racket language with graphical commands and can be used
within it’s own livecoding environment or from within the
DrRacket IDE.
Thank you for the Fluxus suggestion! I'll give it a try - although development seems to have stagnated, lacking the critical mass needed to keep the project going (for now).
(also, they should have followed your advice on websites. So many artsy open source initiatives seem hell-bent on being inaccessible to outsiders with their presentation)
As for Racket's website, it's significantly better now then what it was when I wrote that blog post. I don't know if it was based off of my recommendations, but it definitely improved things quite a bit.
Ruby's site has also dramatically changed and I'm not sure it's been for the better. It looks like a generic WordPress theme now.
[^1]: Various extensions to Scheme are proposed as numbered SRFIs; the extension in question is SRFI 49, as defined by http://srfi.schemers.org/srfi-49/srfi-49.html
I can't find any direct connection to Racket, though, and it's not part of the Racket package repository.
The beginner track is very nice, taking you through the weird syntax, introducing pattern-matching with examples, and is relatively light on falsehoods.
In many respects, yes.
http://www.lispworks.com/documentation/HyperSpec/Body/03_aba...
There are differences of course which should be pretty easy to spot if you know lambda-calculus well.
There is Wisp (Whitespace to Lisp):
https://bitbucket.org/ArneBab/wisp
http://draketo.de/light/english/wisp-lisp-indentation-prepro...
(I have never tried it.)
I don't doubt Lisp is beautiful, but I don't think the author explained it well.
As a result of Lisp being in prefix notation, which is just a fancy way of saying you always put the action first (for instance you add numbers like (+ x y z)), that design choice results in EVERY part of the language having the same syntax. That is, everything is an action that eventually returns a value, and it all is used the same way. There are no special syntaxes -- loops, conditionals, function definitions, everything. Since it's all the same, everything all the way up to the top level is one action composed of other actions chained together.
Parentheses in Lisp languages are used differently than other languages: parentheses are used for grouping and ordering in other languages, but in Lisps they're used to denote scope. The first word after an open paren is always an action, and every word thereafter is always an argument. Then arguments can be a lower level of scope which performs another action, and so on.
So really, Lisp has almost no syntax at all, and a Lisp program is nothing but a big tree of actions that start at the top level and work down (sort of).
I've heard of it described as, "programming in the AST." You don't have to think hard about how an expression will be interpreted. In fact you can get very far with just the substitution model of eval and apply as they started off with in the SICP lectures.
Basically, rthomas6 explained it better. But may you allow me to rephrase myself, as a complement as much as an excuse?
There are 3 points.
The first says that Lisp is simple because once you know how to write function calls in classic language (like C, Java, Javascript...), you have all you need to write Lisp code. And really any Lisp code, not just function calls in Lisp.
The second point was that more classic languages go not just one but many different syntax routes while one is good enough to code.
Concretely, for a whole month I was thinking first as writing Java code and then translating it mentally into Lisp. Until I reached enough confidence to let that go and just code in Lisp. When I did that, everything fell apart. Except for one rule: write chainable action perimeters. It is enough, no matter how astonishing it may seem.
The last point was not much about saying that Lisp is beautiful. If anything, I find it elegant but not quite beautiful. No, the last point was about saying it is enough to look at Lisp for 5 minutes and then you are good to go.
Both ways really.
Use it right away, totally possible. Go away from it, entirely reasonable too. Just know that not much other language can maintain such a degree of decency because they generally claim more of your time right at the beginning while also having the nerve to give less.
In short, I'm saying Lisp is nothing special. In particular, it hasn't the ability to illuminate your own programming practice. I'm just saying that from my own particular experience, it does have illuminated my practice because of the other languages.
Lisp has not made me step forward because of what it is in itself. It made me step forward because every other programming languages I practiced have made me walk backwards.
But there is something to be said for using syntax as a way of representing meaning.
Conditions, loops, function definitions, types, and so on, can be expressions (depending on the language, they may be fundamentally so), but having them look different, that reduces the interpretive burden, I think.
Perhaps the bigger lesson of the OP is how few programmers understand how their language works. A case for people to write their own languages as part of a CS course, maybe?
I can't remember who said that learning Lisp makes you a better programmer, even if you don't use it day to day. I strongly agree. It did that for me.
I think this is a very important point. Since everything is the same in Lisp, it's nearly impossible to have an idea of what the program is doing "at a glance". In (sane) syntax heavy languages, you can glean a significant amount of information in a short period of time just by scanning the syntax without reading the code.
This is less of an issue when writing a program, and more of an issue when maintaining one.
It is very true that in syntax heavy languages, scanning is made very efficient from the syntax of the code. And sure enough, with big chunks of Lisp code, scanning is a chore. But it is important to understand that scanning is an enabler or, in a more mathematical expression, it is a sufficient condition to make scanning a palatable option. But it is absolutely not a necessary condition.
How?
Lisp code is generally terse and split into several tiny functions. There are some exceptions of course and, based on my experience, there is no exception for such exceptions: they are all tedious to read.
How tiny are the functions you may asked? Here's some stats about my project:
It has 8368 lines of Lisp code for a total of 676 definitions (variables, functions, macros and some other things). The average ratio of lines of code required per definitions is thus 12.37! That is completely, absolutely way way less than code in C, Java or Javascript. You may say that I just have a lot of very tiny utilities in the range of, say, 3 to 5 lines of codes. While it's true in a way, the overall average holds well to about 12 lines of code even if you look at the level of code systems (rather than the whole project) or even at the level of files (rather than whole libraries).
The point that I am trying to get across is that, while I agree there is a connection between having an idea of what the program is doing "at a glance" and gleaning information by scanning thanks to the syntax, this connection is certainly not forwarding any qualitative information about what a program is doing. I mean, from gleaning information from syntax to understand what a code does, nothing is passing that is remotely like 'this does that'. Syntax only guides your eyes. Scanning only allows to locate pieces of interest. But to know what a code does, you have to read it and there is no other way around.
And that's the very purpose of my essay. I am like you. I think I know better what a program is doing when I have some syntax but it's only an illusion. Syntax has absolutely no meaning in itself about what is being done. I have to read the name of the actions and what is passed to them. And if I want to really know what is being done, I have to read it all.
True, it's easier to read code with syntax guiding your eyes but when your code is only 12 lines, you can just read the whole very quickly and you know everything about it, not just the pieces you have gleaned. Syntax heavy languages make it easy to have a lot of information in a short amount of time but they make it more difficult to get the whole picture because they just spread code on so many lines with things less connected than in Lisp (two problems about that: statements do not return results and ask the programmer to do variable plumbing).
At the present time, I think it's all a matter of taste because getting to know what is happening still requires quite some work regardless of the language involved and the more complex what a program should perform, the more code there will be to read.
But Lisp has the edge here I think. Note that some of the various code systems I gave some stats about earlier are not trivial. One has two Lisp code parsers (one of them just gives the stats I exposed, the other one identifies dependencies): 1194 lines for 86 definitions thus a ratio of 13.88. Another one "full view debugs" code: 659 lines for 39 definitions thus a ratio of 16.9.
Despite the goals these code systems tackle, they do not sky-rocket in their stats; every one of them stays very reasonable. And both projects bring much insights about the structure of the code and what is being done. Especially the latter. Full view debugging is about computing all intermediate results inside a given code and layout them all in the browser. It's like when one debugs by hand except that it is all done automatically and everything is here for you to see.
In other words, you can scan and you do not get an idea of what the program is doing. You get what it is really doing. And because Lisp syntax is simple, that full view debugger works on any Lisp code (although of some of the most exotic actions, it does not go into them).
You could say "paragraphs and punctuation only guide your eyes, but to know what the text says you have to read it, there's no other way around" - ancient greek was written without word spaces, it barely introduces any ambiguity.
> 1194 lines for 86 definitions thus a ratio of 13.88
Just checked my current python codebase. It has 1540 loc, and 95 definitions , for 16.25 on your metric.
And bear in mind you can't put things all one one line in Python (i.e. it would be trivial to write 1 loc per unit in a Lisp, but not good practice). What would it do to your numbers if your program followed SRFI 49? Newline is just a form of punctuation, after all.
Ultimately "no, this language is more awesomer" gets nowhere, I think. You're clearly passionate about Lisp and think it is the best. Good for you. After writing hundreds of thousands of loc in several languages (incl. Lisps), I'm not so sure. Maybe I am just plain wrong. Maybe I just need to do more until I really 'get' it. Who knows!
I'm not sure to understand. But I fully endorsed your paragraph statement up to but not beyond the remark that a parenthesis is a punctuation. I was not suggesting we don't need some kind of punctuation or blank lines in code. I was pointing out Lisp dialects are proof you can basically code with just three signs of punctuations: newlines, left parenthesis and right parenthesis.
Are we also clear on the fact every programmer knows how to program in Lisp, even if they didn't even program in it? It's just like function calls, except for the whole language. Really. There's no need to write Lisp code a lot. It is enough to take a few steps back, consider the whole picture and see it for what it is.
Lisp is chainable action perimeters. And that's very easy because it is just like function calls in C, Java, Javascript or many other, more mainstream languages. If you know one of them - and I bet that's the case - then congrats, you can program in Lisp! Whether you choose to do so or not, you're the judge of course and either way is fine with me. But don't say reading or understanding Lisp is hard because for that you only have to learn the vocabulary, as is the case for every programming language.
Anyway, thank you very much for the pointer to SRFI 49! I have searched a bit about indenting Lisp code but never found something that detailed.
I'm sorry for the absurdly fine-grain of my stats. It's just that I have my own Lisp code parser for that and it includes inner comments and blank lines. So it's really that precise but I was silly to recopy these numbers without thinking about the issue.
My point was to say they are consistently that low. And for some languages, it's clearly not the case. But from the few codes of Python I have read and given your own stats, it feels like Python is very consistent. And in my eyes that's definitely a very good thing.
> "Ultimately "no, this language is more awesomer" gets nowhere, I think. You're clearly passionate about Lisp and think it is the best."
I just read again my comment and it can so be read like that... I should have been more careful. I'm truly sorry to have almost versed into a flame debate.
But to read my comment in the direction that Lisp is more awesome or simply the best is to amplify too much my claim which was relative to a precise issue: the needed amount of lines of code against the complexity of the problem. I really said nothing more than Lisp has an edge there and I think the examples give an idea of complex problems: parsing code to get a precise line count or to identify dependencies or to even execute automatically step-by-step the code, these are surely complex tasks. At least it is for me and I can't imagine coding it in other languages than Lisp. Because of their very syntax.
> I was pointing out Lisp dialects are proof you can basically code with just three signs of punctuations: newlines [I think you meant whitespace generally], left parenthesis and right parenthesis.
I agree. In fact, with SRFI49, you can do it with just newlines and spaces. My point is that you can write prose with just spaces, periods and capital letters too (or less). But the use of parentheses, dashes, colons, semicolons, question and exclamation marks, quotation marks, and ellipses, is not a bug. See the metaphor I'm getting at?
> Are we also clear on the fact every programmer knows how to program in Lisp
Yes, if by 'program in Lisp' you mean 'program in a lisp-like syntax'.
I tend to think that syntax is a rather superficial thing, compared to learning the model of computation and the runtime environment. Then Lisp is quite different from C, say. A C programmer doesn't already know Lisp. One problem I had with the OP was that: it seemed to suggest the syntax 'was' Lisp.
I think Python in Lisp syntax would be more Python than Lisp, personally.
But what is OP? Opening paragraph?
> compared to learning the model of computation and the runtime environment
There is indeed more important things to learn than syntax. The thing is, for Lisp, chainable action perimeters not only covers the syntax, it also covers the model of computation itself. The basic one at least, without macros and any object system. Still, that basic one is quite broad.
On that question you actually anticipate another article I wanted to share: shortcomings of instructions. It's only logical. If in Lisp dialects the atom is the chainable action perimeter, what other languages have? The answer is the instruction and well, I will keep that for next week but you can read it if you want. The URL is http://www.vagrant-coder.com/articles/english/Shortcomings-o...
Right, they provide additional structure that can in turn help someone express themselves.
But you're right, the complexity can obscure some underlying simplicity.
I suspect that the solution is to disentangle the two: to help programmers understand the computational model independent of the language.
> it also covers the model of computation itself
eval/apply is what I'd consider the model of Lisps, rather than chainable action perimeters. The two are linked, of course, former depending on the latter. I guess you probably mean roughly the same thing by it, so I'm not disagreeing, just saying what I was thinking of.
I agree, the von Neumann model of computation is not necessarily useful if programmers believe it is the only model. But I think it is crucial for programmers to understand.
OP = original post (i.e. the article on your site).
Once we got to see just how powerful macro programming could be, the S-Expression syntax was here to stay.
I'm curious because I've used functional languages in the past and even written some fairly substantial projects in one of them (a toy compiler written in Ocaml), however, I've never left with the impression that they are superior to other languages. In my experience functional languages give you some convenience with doing functional things while making lots of other things slightly more difficult.
Jane Street seems to like OCaml - https://www.janestreet.com/technology/
Much of the incidental complexity of real software comes from misunderstanding this, just as much of the real complexity comes from the problem of maintaining a big, distributed, mutable state.
Also, both "code is data" and "homoiconicity" are notions that ends up showing concretely in macros. And the conflict with the notion of "chainable action perimeters" is twofold but always harmful to properly operate as a programmer.
First, we don't deal directly with the true results of macros, which are codes. We let the evaluator compute theses codes internally and it then automatically chains with a call to... itself!
So yes, it can be said one can teach the evaluator new tricks but the infrastructure enabling that and justifying the expressions "code is data" and "homoiconicity" feels a bit too much like a three-card trick to me. The fact what seems the best advice on how to use macros is to not use them when functions would do appears to be, I think, another thumbs-up for the expression "chainable action perimeters".
However, how many different roles do they have apart from writing action perimeters?
The answer is 0!
And 0 times 10, or 100, or 1000 or well, just go for any number, it doesn't matter, that's still 0.
Also, note that Lisp has only parentheses. While in the end, it is a matter of taste, I don't think one can disagree on the fact Lisp only has parentheses and they serve only one purpose.
Other languages have in fact many roles for parentheses while also having brackets and many roles for them while also having curly braces and many roles for them.
An action perimeter returns a value. To perform its work, it needs parameters. That is: values.
Some of these values can be written by hand, others need to be computed. In that latter case, you ask for a computation by writing an action perimeter.
And there's no need for a special syntax to use the value that results in executing that perimeter. You just write it in the appropriate place, as a parameter.
So in practice, you can chain perimeters with one another. Until you have expressed all the computations you wanted the computer to do.
After looking at the French version I see that "action perimeter" is from the french perimeter d'action (accents removed) which translates better in english (once again according to the interwebs) as: sphere of operations. http://dictionary.reverso.net/french-english/p%C3%A9rim%C3%A...
Once again, I'd go so far as saying the concept of functions works just as well here, but at least it gets over the near homonym of parameter/perimeter. The chaining of scoped actions is pretty vital to lisp, and I suppose functional programming in general.
An action perimeter is different from a function because, for instance, you can write (if <test> <then-clause> <else-clause>).
For a function, all its arguments are evaluated first and then it is applied. That's certainly not the case for if since the whole point of a condition is to refrain from evaluating some code unless a condition is met.
Hence, in Lisp, the simple notation encompasses and is fully operative on every action, including the ones that do not behave like functions. While in many language chaining perimeters is only about function calls. And in addition it is rarely idiomatic to chain function calls in such language.
Thank you for pointing out the similarity between parameter and perimeter which borders on confusion. Thanks to everyone who brought it up to!
It's clear that's an issue I overlooked. The reason is I use the term argument in English and in French, never parameter itself. But that's probably because once I realized the notion of a perimeter is important in Lisp, I stopped using the word 'parameter'.
Sorry for that! I'll try to find a better term.
Remember that DNA structure - an aminoacid in the CAR and the next CONS in CDR?)
Lets say that it is not a coincidence, given that the DNA structure was a host topic these days.)
Guess where the Actor Model came from?)