An Intuition for Lisp Syntax
stopa.io
stopa.io
I could not help but thinking at the end, this is really awesome but s-expressions are kind of hard to read and reason about for my brain, it would be cool if we could generate them from more readable syntax, maybe something like javascript :D . Unfortunately the standard js AST seems to be not a great fit but i found a S-Expression encoder for js AST https://github.com/anko/eslisp
This way instead of having to write s-expressions manually we took a whole circle and are back at js but with macros, not relying on eval and having a code is data representation...
The classical one is "The Nature of Lisp"[0], which introduces data-as-code through XML and Java build tools. Same idea, just with examples more relevant at the time of writing. Still worth a read for non-webdev programmers.
Having learned Lisp, it's half funny, half disheartening to watch the industry repeatedly tries to rediscover "code as data", as people's configuration or data files in markup language du jour grow in complexity and eventually start directly encoding executable code... and then stop shy of embracing the code/data duality.
--
"Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp."
Also:
> Hacker Robert Morris later declared a corollary, which clarifies the set of "sufficiently complicated" programs to which the rule applies "...including Common Lisp."
[0] - https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
The place where JavaScript contains "an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp" is Babel.
This is referring to CL implementations that depend heavily on C.
Many CL implementations now are almost 100% Lisp code.
One thing that XML could have done that s-expressions can't is a way to serialize a dag if proper nesting of tags wasn't required. I'm playing around with a language based on that, the mental load of having no clear way to delineate blocks visually makes it hard to reason about, unfortunately.
But seriously speaking, for me Lisp would be much more appealing if it had been introduced to me as a specialized niche language, though for an important niche nonetheless, instead of as the be-all-and-all language for your superpowered startup [1].
(For those that don't already have a goto list of counterarguments to "Lisp all the things": My main contra-point to Lisp is that with all the meta programming powers you get, you write yourself into your own little corner where no-one but you and your friends live. You want advanced syntax highlighting, linting, automatic refactoring for your special features? Write it yourself! You want outsiders to participate (think: new employees)? Write all documentation yourself, too!)
> You want advanced syntax highlighting, linting, automatic refactoring for your special features? Write it yourself!
That's true to an extent. Simple things are simpler in Lisps, because the syntax is trivial. So highlighting and structured editing are easy. The rest, is near impossible, at least for a full-featured Lisp like Common Lisp. That's a consequence of being an extremely dynamic language, that ships a compiler and intertwines parsing, compiling and execution. The flip side of being able to run arbitrary computation at compile time is that, in general, you can't know what the program will do until you run it. The dominant free CL IDE, SLIME, does just that: it queries your running Lisp image for its current state, to provide you with formatting and autocomplete and other hints.
That said, "near impossible" is a function of community size. Had CL anywhere near as much popularity as Java or Python does, I'm sure folks at IDEA would make automatic refactoring work for CL as well :).
> You want outsiders to participate (think: new employees)? Write all documentation yourself, too!
That I strongly object to. Code generation and compile-time execution aren't magic, or even particularly hard concept. They're just another flavor of code. You manage it the same way as regular code - you package it into modules with well-defined interfaces, and document them. I've worked with CL in a team settings, and I've worked on legacy CL code that's as old as I am; it's not harder than "regular" legacy code. And you always have to write your documentation yourself, there's no escape from that, no matter what language you use.
It needs to be added that in Common Lisp, once the program is running you can do everything with it while it's running: Inspect the stack frames, change variable values, rewrite/recomiple/update functions, update class definitions, update objects to said new class definitions, save the current program state to disk,etc.
So it's programming that is geared to RUNNING the program and modifying it while its running.
I never inherited a Clojure-codebase that has been written during long duration, but I have digged around in a fair amount of Clojure-codebases that are open source, both user focused and libraries. Same with JavaScript. And I can say, as someone with more JS experience than Clojure, that the Clojure projects tend to be a lot easier to understand than the JS ones, probably not because of the language, but because of the habits that the language "forces" you into.
This trade-off is not restricted to Lisp, but also applies to e.g. C++ template meta programming. Compilers have gotten betters with templates, but still debugging the more advanced usages of templates can become hellish. Codifying the features the meta programming supplies in the language itself or in well supported libraries means that error messages get better and many usage scenarios are documented on Stackoverflow.
I don't doubt your experience regarding Clojure vs JavaScript code bases, but this probably has to do with other reasons than the meta programming the original blog post is about?
Which is weird, because in most Lisps meta programming happens at compile time. It should be possible for tooling to just show you the expansion of any given macro invocation. In fact, when I used Clojure a decade ago, there was an Emacs command that would expand the macro invocation under your cursor, so you could see what code was actually being generated.
So I don't really think that's a fundamental limitation. I suspect it's more related to the fact that Lisps aren't especially popular, and don't get the attention from tooling that other languages do.
> Codifying the features the meta programming supplies in the language itself or in well supported libraries
But without the meta programming, those libraries might not actually be possible to write. You will either end up with a more dynamic interface (doing meta-stuff at runtime), or a clunkier and more verbose interface. I think being able to expand a macro invocation to see what it turns into is enough for all but the hairiest of macros.
Exactly. On common lisp, for example, it's just a keypress, and it has a "macro stepper" so it shows the first expansion possible, then the second expansion, and so on and so on... until you end up with compiler primitives!
Shouldn't you be doing this anyway? Not trying to be snarky: it's that every job I've worked at has a classic underdocumentation problem. Tribal knowledge dominates, and the practicalities of shipping product overrun the need for teaching employees, new and old, about the idiosyncrasies of the system. [edit] This seems hardly an issue with LISPs or DSLs in particular.
This is contrary to my experience (though I've mostly used Racket rather than Common Lisp). Tracking what's a function vs macro, where an identifier is introduced/used, etc., is all baked into the underlying language and not something I have to write myself for every language extension I make.
> You want outsiders to participate (think: new employees)? Write all documentation yourself, too!
Were you planning to not document your internal-use utility code?
So many people start out thinking like this when they learn Clojure or some other Lisp and then after like 2-4 weeks of reading Lisp code, any C-like code starts to look comparatively irregular and Lisp feels much easier to read.
I recommend Clojure/ClojureScript. It's quite practical for web development and also an excellent gateway into functional programming.
Here is one anecdata. When my daughter was 14 I taught her a Lisp and then a traditional programming language. She found the random syntax of the trad formatted language bizarre.
Two things were important to make it "click" for me:
1) I realized that "Lisp has many parentheses" actually has it backwards. A Lisp code has about as many parentheses as an equivalent Java code; there is almost a 1:1 relation. It just has less of... unnecessary things. Which results in more "parentheses per square meter", but from this perspective that is a good thing!
2) Some people complain that dense code is more difficult to read. But you don't have to turn everything into one-liners! You can format it into just as many lines as the Java code would have, except the lines will be much shorter now, and therefore easier to read. In Java, you don't have much freedom with formatting, because a typical statement takes more that half of the screen width, so you are stuck with one long column. Lisp with shorter statements gives you more freedom. You can abuse it to write hard-to-read code. You can also use it to design beautiful code that is easy to read. Also, there is a huge difference in legibility of a code that fits in one screen (so you can see it whole at once) and one that does not; and the Lisp functions are shorter on average.
function doSomething(a, b) {/* something! */}
const doSomething = (a, b) => {/* whatever */}
const doSomething = (a, b) => b => “some closure”;
(defn do-something [a b] (fn [b] “some clojure”))
To me at least, (fn-name args) was both more readable and makes more sense than fnName(args) after a few days, and I never learned a lisp until I was 37. Maybe I just found a style preference later in life, marking me as a lisp survivor. Maybe we’ve just been doing it wrong for decades and have grown accustomed to it. I can’t say. But the power of the language itself coupled with the code editing features it enables makes me think that the whole field has been on the wrong track, or maybe even off the rails, for decades.
The JS code I write now is more flexible, more resilient, more testable, and more maintainable as a result of learning Clojure. I doubt anyone can say the opposite: that a C-style language improved their understanding of lisp. (Not addressing you with this part, dear poster, since your gripe is rooted in the syntax.)
[NaN] “kids these days“ is just friendly ribbing. I’ve got the grey beard now, so I‘m free to play the part. Language maintainers tend to be older, and we cause more problems.
I don't see how you can argue otherwise. It is as if you have no clue at all about how UX works. You can't get used to everything, there is a hierarchy of syntaxes, and lisp syntax is just plain bad for humans.
Yet many people will still adamantly say that they find salad tastier. And still go on and eat cake and get fat. They are just lying to themselves, I'm not sure why. Lispers are the same way, its just a reality distortion field.
Have you ever programmed a decently sized lisp program using an editor that allowed you to work at the AST level?
Until then, there's just a large probability, regardless of how tasty cake is, that your displaying classic ignorance.
I don't really get this argument. It's not any harder to read `(if foo` than it is to read `if (foo)`. It's purely a matter of what you've gotten used. I don't think it's an objective measure, it's purely about what you've already trained your brain to pattern-match on.
> and has much more focus on text over symbols than other languages
Funny how you get the exact opposite complaint about languages like Perl.
Return the maximum value of the array “numbers”. The languages are listed in descending order of text > symbol.
Clojure:
(apply max numbers)
Math.max(...numbers);
⌈/ 4 3 2 7 5 1 3
⌈ numbers
Furthermore, how is this
(if a
true-cond
false-cond)
if (a) {
trueCond
} else {
falseCond
}
There are a few options out there; maybe take a look at Sweet Expressions? Some Lisp implementations have native support for different syntax, but the nice thing about these formats (which are essentially just serialisations of concrete syntax trees) is that we can convert them into the expected format automatically, e.g. using a pre-processor.
I have a go-to page on my blog for discussions involving s-expressions, including links to various alternative syntaxen: http://chriswarbo.net/blog/2017-08-29-s_expressions.html
very similar, build out of python []
I think it's likely this is just a matter of familiarity.
Like the way that people think the Windows (/ whatever) GUI is intuitive - but when you test this by putting someone who has never used it in front of a screen, it actually isn't. "What do you mean if I write something and don't also 'save' it, whatever that is, it will disappear? This is supposed to be better than paper...".
The text editor handles indenting and matching. After a while it is completely transparent.
Obviously opinions on the language shouldn't be given too much weight if a person hasn't spent too much time investigating/ using it.
As a non-JavaScript person, all the JavaScript examples looks like gobbledygook unless I sit down and consciously think about what the examples are showing, and it kinda hurts my head a bit, but I can slowly force myself through it.
But as someone with a few years of lisp under my belt, even the closure examples (and I've never used clojure) parse pretty close the speed of thought.
It really is hard to explain to someone who hasn't used lisp for 6-12 months, but there really is a lisp "enlightenment" experience, which is totally unlike anything I've experienced with any of my other languages. I'm not saying this makes it better or worse than other languages, but I am saying it's real, and that lisp code is not only parsable, but exceptionally so to an experienced lisp programmer. One day you're sitting there thinking there's a whole lot of brackets, balancing and indentation in this godforsaken language, and then something just clicks over in your brain and you don't even see the brackets any more and it's all just data structure, and you see the shape of it, and you say to yourself "well that was pretty fucking cool" and from that day on you don't really see the brackets any more, it's all just names, indentation and data structures.
edit: and it should also be recognised that an editor implementing parentheses balancing, indentation options, and keyword lookup/ completion also becomes a relatively trivial exercise when your code consists of variable names and structured data.
In my experience, the Lisp enlightenment comes in stages, two big ones are 1) s-expression parsing you describe here, and 2) grokking "code is data".
WRT. s-expressions enlightenment, I vividly remember when it happened to me. Few months into learning Lisp, I bought a hardcover SICP and, for some reason, decided to do exercises in it on paper. Few exercises in, as I was writing Lisp code with a pen in hand, suddenly something clicked in my head - like my brain JIT-compiled a new processing module - and from then on, I no longer needed to mentally count or track the parens. I just knew how many and where they were, it dropped to semiconscious level. Ever since, I'm very comfortable with s-expressions; in fact, I prefer them as a notation to Algol/C-like code.
I think no matter the language the syntax will disappear over time and your brain will learn to look at the character matrix and directly see logical constructs, moreover this will always apply cross-language to other languages with similar syntax. Fit example, I primarily work in TS, I have decent amount of experience in other C-style, and to me Rust is easily readable with very little experience actually using it.
I don't think anyone who write Lisp-like languages professionally doesn't use tools like parinfer/paraedit, where balancing parenthesis is not something you have to do.
Not with him.
But I might be wrong here, it's simply hard for me to imagine someone writing with lisps professionally going the route of textual editing when structural editing is right there.
http://www.lispworks.com/documentation/lw71/EDUG-M/html/edus...
http://www.lispworks.com/documentation/lw71/EDUG-M/html/edus...
Gerald Jay Sussman
You wrote a really nice comment ages ago, which partly inspired me to learn lisp and also it’s formatting.
I still want clean it up a fair bit, but here’s the draft article:
https://ashokkhanna-530.medium.com/formatting-lisp-5e28020b8...
Hope it’s okay to use your quote! And if not, I’ll take it down
Ex french & english vs french & indonesian. English and french can recognize words from each other and kind of see some similar grammar rules (the medicine vs. la médecine) while with indonesian those moments are not as frequent.
It then feels more like tree manipulation than like writing code. Braces are always matched, you literally cannot delete the closing brace using normal text editing. Each tree depth as its own brace color so it's easy to distinguish between them.
Restructuring tree to join two blocks of code or to extract block to higher level is exposed as command mapped to keyboard shortcut. It's not semantic editing in sense that extension is aware what is function and what is data, you can still make nonsense code. Still, reading and manipulating code is quite effortless.
In Emacs, you may find Lispy and Prism even better.
is
(map foo '(1 2 3))
[1, 2, 3].map(foo)
Or:
(let ((array #(1 2 3 4 5)))
(vector-set! array 0 3)
(vector-ref array 0))
let array = [1, 2, 3, 4, 5];
array[0]=3
array[0]And when it comes to macros and (pseudo)-quoting, the LISP is hands down more obnoxious (again, for me) than, say, Python's f"{}".
I wonder how much of that has to do with your familiarity with "traditional" language syntax. For example, when I started writing Lisp, I had a similar opinion. But I write Clojure professionally for a while, and that disappeared. Now I haven't written Lisp in probably 7 years, but I still have no problem reading it.
> Parens, parens, everywhere, nor a drop of structure.
On the other hand, in some sense it's all structure. There are certainly advantages to having everything be delimited.
Slightly off-topic but interesting none-the-less. After starting to program with Clojure both as an hobby and professionally, how do you go back something that is not lisp/repl driven?
I've tried time and time again to go back to JavaScript, as I used to be OK with it, but I just cannot justify the hassles I have to fight with everyday, compared to if my co-workers just picked up Clojure instead.
Even Java has advantages in terms of tooling, and from Java 8 on, you can write code using some Clojurish idioms with streams. Though for immutable data, you need something like Lombok + pcollections.
JavaScript, on the other hand, really doesn't offer anything over Clojure, so I can see why you'd struggle going back to it. It's just a downgrade.
And yes, human ability to pattern-match things is astonishing, I am sure if I were to program exclusively in Scheme for half a year, I too would one day grow accustomed and used to it. But do I want to? I am really not convinced about that. A human can get used to pretty much anything, even to almost constantly being in mild pain, but... no. I'd rather just not.
In a good JS editor, how many key combinations and mouse clicks are required to jumping into, jumping out of, and cutting a block of code (eg. a function definition or a conditional expression), or transposing, merging, splitting, annexing, and de-annexing 2 blocks of codes? It usually takes me at most 2 key combinations with a Lisp editor (including navigating the cursor to the right place), thanks to Lisp's uniform structure.
> Say, naming a thing, a function definition, and a function invocation are all very visually different in, say, JS
Aren't these also visually highlighted in a Lisp editor as well?
Besides, a Lisp editor can optionally blur the parentheses so users don't mentally have to.
(edit: formatting, recounting the key presses required)
> Besides, a Lisp editor can blur the parentheses so users don't mentally have to.
Besides, another languages can throw away the parentheses entirely so neither users nor editors don't mentally or visually have to.
I have not seen transposing, merging, splitting, annexing (moving a block into the inside another block), and de-annexing (the opposite of annexing) 2 blocks of codes in JS without using the mouse yet, so I just want to check.
> Besides, another languages can throw away the parentheses entirely
JS uses parentheses for grouping complex arithmetics and for function's argument lists (eg. func(arg) in JS vs. (func arg) in lisp). JS also uses curly braces to mark code blocks, which is similar to Lisp parentheses but at the cost of more complex parsing for the compiler and the mental distinguishing between functions' argument lists and code blocks on the coder (they are just lists).
JS statements use semi-colons, which makes editing them feel like editing lines of codes while editing Lisp statements is editing nodes of a tree, which is a very different experience.
Which mainstream languages have no brackets?
Is that in emacs/slime or another editor?
How do I do that with spacemacs?
===
(package-install 'paren-face)
(global-paren-face-mode 1)
(custom-set-faces '(parenthesis ((t (:foreground "gray50"))))))
===
(edit: formatting)
(* (f (+ 2 x)) 5)
5 * f(x+2)
And yet we learn all kinds of syntax that we haven't been familiar at all (zero indexing, x=x+1, etc which we used to the exact inverse: math assignments being immutable or denoting an equation not increment, etc),
-- not to mention advanced stuff like generics, futures, closures, etc --
and we seem to manage just fine...
(math 5 * f (x + 2))
Also, I would probably rewrite the expression as:
(* 5 (f (+ x 2)))
(* (f (+ x 2))
(g (+ x 3)))
It has nice, readable, ruby-like syntax, but it's a Lisp at the core. Really powerful macros (see Ecto), and a syntax that isn't off-putting to the majority of programmers.
Most things in Elixir, including constructions like if and def, operators and even module accesses (`Module.function`) are compiled to a simple AST based on function calls.
Because that AST isn't usually written by humans, besides the function and the arguments, each node might contain additional context information, like the file and line number the call appeared in. That lets you do some really cool stuff, like distinguishing x (a variable you define) from x (defined by a macro), which is a frequent source of bugs in other languages that support macros.
99% of Elixir is syntax sugar over function/macro calls, and its true AST is smaller than Clojures.
You can read more here: https://hexdocs.pm/elixir/syntax-reference.html#the-elixir-a...
(operator arg1 arg2 arg3 ...)
(+ 1 2 3)
=> 6
(+ 5 (- 10 6))
=> 9
Notice how the operational form is just a list with some symbols and data literals. In Lisp, the syntax for writing data structures and the syntax for writing code is the same syntax. And since a function operates on data and produces new data, what if a function could operate on code and produce new code, since code is just data?
We call such a function a macro (and I don't mean Excel macros). Macros run at "compile-time" (technically 'read time') and the code output is executed at "run-time" (or during 'evaluation'). The benefit of macros is that if your language is missing a feature, you can add it.
You can see this in practice by looking at the source code for the `and` and `or` logic functions in Clojure, which are typically built-ins, but in Clojure they are just macros bootstrapped on top of the special forms `if` and `let`: https://github.com/clojure/clojure/blob/38bafca9e76cd6625d8d...
Clojure only has 13 special forms:
[fn let let loop do while . if def recur
try catch throw quote var
monitor-enter monitor-exit]
When I started learning Clojure, I found the ClojureScript Koans to be very helpful in getting a feel for the semantics and to become familiar with the argument placement: http://clojurescriptkoans.com/
If you come from a traditional OO-background, my condolensces and I recommend starting with Rich Hickey's 2-hour talk, "Clojure for Java Programmers": https://www.youtube.com/watch?v=P76Vbsk_3J0
Is this condescending attitude really necessary or useful?
https://srfi.schemers.org/srfi-105/
Or, you could use T-Expressions if Python style whitespace indentations to represent code nesting is easier on your brain.
https://srfi.schemers.org/srfi-110/srfi-110.html
Scheme and Lisp are VERRRRY Flexible.
This is VERY elegant!
Little known fact: The people who came up with lisp wanted to do this, too.
As a non-LISP developer I get by without macros and all the fancy things that are possible with LISP. I would like to see a strong case of daily tasks that are much easier with LISP macros. Anybody?
Super expressive and readable. I went from not being able to read clojure to now having written a few apps in it and feeling comfortable with the syntax in a few weeks.
I definitely write less code to achieve the same ends but it’s still early days for my lisp adventure.
Which I don't believe it does, IMO the primary reason LISP isn't mainstream is because it results in less readable code for humans (i.e. the primary objective of programming languages), it's semantically the perfect minimalist language for a machine but I don't believe it's optimal readability for humans. Other disadvantages include lack of typing & poor tooling support. There's certainly domains it excels at due to its intrinsic qualities but I don't see it ever becoming a popular mainstream general purpose programming language.
Now I learned C-like language before I learned Lisp-like languages, so that might be the reason I felt I understood Lisp way faster than I felt I understand C-like languages. There is simply less to learn about the language and more to learn about conventions, while C-like languages always have bunch of extra syntax you have to learn, otherwise the compiler cries.
> Other disadvantages include lack of typing & poor tooling support
Not sure these are inherent to Lisps. There are plenty of typed lisps, and even the ones that don't have types, can have types added to them after the fact, just because they are Lisps.
Regarding tooling, I'd say that the entire C-like ecosystem is far behind any Lisp language regarding tooling. I mainly see people using "println"'s for debugging, and sometimes using a "line debugger", while Lisp developers modify their code at runtime and have a far more advanced debugger, something C-like languages will probably never do as well as Lisp-like languages does it, again because of s-expressions and the code they make you write.
So your argument is that all code is as easy to read as long as you have done the work to get familiar with it? I hope you understand that it is bullshit. Brainfuck isn't easier to understand than python no matter how much time you spend with brainfuck.
My point is that "readability" is composed by many factors, not just one. What is "readable" to some will not be "readable" by others. It depends.
For someone who knows Fortran, learning a Fortran-like language is easy (like C or JS). For someone who knows Common Lisp, Clojure is easy. But for someone who knows Fortran, Clojure is less familiar, hence the code will, on a glance, look less "readable".
The other factor is if something is "simple" by itself.
Most of this view comes from Rich Hickey, who wrote Clojure. He discusses "readability" or "simplicity" rather, in his talk "Simple Made Easy". If you haven't seen it before, do yourself a favor and watch it: https://www.infoq.com/presentations/Simple-Made-Easy/
He'll explain it much better, and with further points, much better than I ever can.
I don't disagree with your overall point, but I actually wonder how true that is. If you spent years working in Brainfuck, and had never touched Python, I suspect it would take a bit of effort before Python was as readable to you as Brainfuck.
After one year of clojure, it's annoying for me to read classic C-like languages. There's also people who prefer reading code without syntax color, so clearly this is a subjective matter.
I find clojure tooling really good, and it's hard to go back to a non-repl oriented workflow (this is not at all the same as something like python or javascript repl). But this is admittedly a gray area, there's areas where it's immature compared to other languages, and others where it shines because of it's repl-oriented nature.
The lack of typing is certainly a sound argument, although my personal opinion is that for many use cases it isn't required. This is going to be a contentious opinion, I'm sure.
Note that clojure has an advantage on this over other dynamic languages: idiomatic clojure code means that pretty much everything is pure data, in basic structures: maps and vectors. So the functions that you use to operate on your data are always the same. Over time you'll learn them, and they won't change between domains nearly as much as, say, if you were coding in OOP oriented python.
So when it came time to implementing a .NET LISP [2], I adopted much of Clojure's additional syntax for improved readability & interop with .NET APIs [3]. But you can only improve LISP's syntax so far, e.g. its Template libraries for HTML generation [4] make for horrible HTML DSL's which looks nothing like the HTML it's supposed to generate. The solution to overcome this was basically to not to use LISP for templates, instead create a multi-language scripting language [5] that embeds lisp into it allowing it to Combine strength's of all languages [6], e.g. use LISP for algorithms and Handlebars / JS Expressions for templating.
The REPL is definitely one of its super powers which is one areas where it shines & basically the primary use-case where I still use it. I've created a live "watch" mode & deep integration with .NET libs that I use for discovery, e.g. run DB queries, call HTTP APIs, execute shell scripts, etc. [7]. It especially shines for being able to open a REPL session with a remote production .NET instance letting me inspect its live running state & invoke system functionality like querying its configured RDBMS, executing redis commands, send tweets, emails, etc [8], I've even got it to Live Script Unity objects in-game :) [9], which speaks to the power & elegance of LISP that's able to achieve so much with so little code.
At the same time I don't think REPL-based programming is all that useful during normal development, you can execute encapsulated code fragments fine, but most of the time I'll need my whole environment constructed before being able to inspect it as I would when debugging, so I find it useful for opening a REPL session into a live running instance, but not using the REPL to construct the live instance. So for my dev workflow, static analysis & typing, great IDE, tooling + debugging is a lot more useful.
[1] https://github.com/mythz/clojure-linq-examples
[2] https://sharpscript.net/lisp/
[3] https://sharpscript.net/lisp/#net-interop
[4] https://www.cliki.net/HTML%20template
[5] https://sharpscript.net/docs/syntax#language-block-modifiers
[6] https://sharpscript.net/docs/syntax#combine-strengths-of-all...
[7] https://sharpscript.net/lisp/#run-and-watch-lisp-scripts
[8] https://sharpscript.net/lisp/#techstacks-tcp-lisp-repl-demo
I disagree. Being a mathematician LISP just feels natural and the ugliest, most horrible and hard to read language by far is java (IMO, of course). But that's just probably because all my life I've been thinking about functions and not objects.
If your background is OOP and look at some LISP there's no surprise if it looks unreadable, even more so if you have no real motivation to understand it.
It’s fresh in my head just how unreadable i found Clojure when browsing source of random projects on github a couple of weeks ago. Its not that penetrable to begin with.
However, that was only a couple of weeks ago, since then i’ve come to really appreciate the simple magic of lisp syntax.
I’ve also been able to put it to practical use rather than just appreciating it - i’ve been spoiled by Calva in VS Code, structural editing that doesn’t just work, it actually flows - i’m no stranger to structural find and replace in Intellij but this is different, this smoothes an impedance mismatch between our thinking and expressing code.
None of that is strictly impossible in languages without macros but best case you'd have to wrap everything in a copious amount of lambdas making for very nasty looking code. Worst case some compile time optimizations wouldn't be possible.
Any single macro isn't THE killer macro, but giving all of them up is a big deal. Similarly replicating semantics of a single macro in some other language may be doable but getting many of them to compose would be tough.
Most popular languages tend to get new keywords, operators and other forms of syntax over time implicitly acknowledging limitations of the "user space" syntax. You don't see that as much in LISPs since most such things can be implemented as a macro.
Smaller stuff like hand rolled parsers and serializers are much easier to get working fast (both time to write and time to run) using macros. Since a good chunk of my work is data flow in one form or another, I miss LISP macros a lot.
For example parametrization and modularization are used to keep code DRY and separated at a functional level.
However this can only get you as far as you still write ceremonial repetition around using those functions/modules/objects. Macros can get your code to that extra level of brevity and clarity.
I managed to speed this up a lot by using a (TemplateHaskell) macro to (a) read in the dataset from an external file at compile time, (b) transform it into a structure that has fast lookups and (c) serialise that structure to a Haskell syntax tree. Not only did this make lookups faster at runtime, it also eliminated all the error checking for locating/reading/parsing/etc. of the dataset: any problem would cause a compile error; if compilation succeeded, the resulting program didn't have to know or care about any of that.
Actually, we can but it's not safe: Scala's tuples implement a 'Product' interface which we can use to loop through the values, but this loses a lot of type safety (looking up values via Product will upcast them to type 'Any'; our function's signature will ask for a 'Product' rather than specifically for tuples of numbers of at least a certain length, etc.)
Macros generate code at each of their call sites, which lets us automatically translate our 'higher level language' (e.g. 'Tuples of different lengths') into the 'real language' (e.g. Scala). For example, we can have a macro that translates 'tupleSum(5, 10, foo)' into a call like 'tupleSum5_10(foo)', where the function 'tupleSum5_10' is hard-coded to sum the first 5 elements of a Tuple10 (and is therefore acceptable to Scala, since it doesn't try to abstract over tuple lengths), e.g.
def tupleSum5_10[T1, T2, T3, T4, T5](t: Tuple10[Int, Int, Int, Int, Int, T1, T2, T3, T4, T5]): Int =
t match {
case (x1, x2, x3, x4, x5, _, _, _, _, _) => x1 + x2 + x3 + x4 + x5
}
If we provide this in a library, users can treat tuples in a more high-level way. Since each macro call just generates some boilerplate Scala code, the result will be type-checked, etc. to make sure we've not done anything dodgy, e.g. since a call like 'tupleSum(5, 10, foo)' expands to 'tupleSum5_10(foo)', Scala will check whether 'foo' matches the type 'Tuple10[Int, Int, Int, Int, Int, T1, T2, T3, T4, T5]'.
I think the most famous use of macros in Scala is Shapeless https://github.com/milessabin/shapeless/wiki/Feature-overvie... (although I've not personally used it)
Note that the exact same problem with tuples happens in Haskell too, which can also be worked around using Haskell's macro system (TemplateHaskell).
But if you want to really appreciate macros, you'll need to read some books. There's a lot to learn.
My introduction to lisp was ANSI Common Lisp and On Lisp by Graham. On Lisp is all about macros. You need ACL to understand On Lisp. What you learn is pretty impressive -- I seem to recall one of the later chapters of On Lisp features a compiler for Prolog in two and a half pages of code -- but it requires a certain amount of supporting material. Still, if you want the most direct route to understanding (some of) lisp's greatness, these two would be it.
Another possibility is Paradigms of Artificial Intelligence Programming by Norvig. This one teaches you Common Lisp in the introduction, but I'm not sure it's enough by itself for you to really understand some of the later chapters. If you're prepared though (read ANSI Common Lisp first), this book is a gem. It's less about AI than about transforming and optimizing programs. So, code as data.
You might also look at Practical Common Lisp by Siebel. I haven't read this, but a lot of people liked it, and the code is very real-world (a little dated now, though). It's available online here: http://www.gigamonkeys.com/book/
In general, you wind up learning a couple of different lisps. Common Lisp and Scheme have the best literature, but the lisp that's most in use today is Clojure. Clojure's macro system is a refinement of Common Lisp's. For learning clojure there are a lot of teach-yourself-X-in-21-days type books. The best of them is the O'Reilly book: https://www.oreilly.com/library/view/clojure-programming/978...
Scheme doesn't (always) have macros, but I'd be remiss if I didn't suggest something. The Schemer books are some of the most effective pedagogy I have seen on any subject: The Little Schemer, The Seasoned Schemer and The Reasoned Schemer. They are very cute, but don't let that fool you. They get hard (in TLS's case maybe too hard) at the end.
Finally, there is one of the most important CS books of all time: The Structure and Interpretation of Computer Programs (SICP). It will change the way you think about programming forever. It also explains some important details about how lisp works, and it's so definitive that a lot of them aren't covered elsewhere. ("SICP already did that...") I reread my copy every five years or so, and I always come away knowing something new.
You don't need to read all this stuff to be a good lisp programmer. One or two of these would probably be enough. But I think it's important to have choices. If you want to talk in more depth, my email's in my profile.
Python has decorators. They have a specialized, limited interface. Sometimes they are limiting you. A macro would impose no limit.
And in the same way you should think twice about introducing a new code of annotation to your program, you should think twice about adding a macro to a Lisp program.
But boy, when you really need it, it can really be a life saver.
(https://hn.algolia.com/?q=http%3A%2F%2Fwww.defmacro.org%2Fra... 0)
The other less known Lisp feature (in the sense of Common Lisp), is the image-based development, which makes for a great REPL. Really interactive, tiny feedback loops, where you compile your program one function at a time. You never wait for a process to restart, even working on a web, GUI or game project. You keep your test objects around. You restart to a clean state when you want. You get compilation and type warnings immediately. You have an interactive debugger that points you to the erroneous line, you fix it, you tell the debugger to try again from the previous stackframe, and you see the function complete.
The reason for that is because introducing special syntax disrupts useful patterns that could be used for writing completely new control structures. Threading macros, for example, are pretty much impossible to do properly in Python because of syntax irregularities.
foo = bar
baz = 2 + (foo = bar)
baz = 2 + (let foo = bar in foo)
One language which handles definitions nicely as expressions is Nix. Nix has 'let' but I never use it. Instead we can use 'with':
with {
foo = bar;
};
"hello " + foo
with {
myAttrs = {
foo = bar;
};
};
with myAttrs;
"hello " + foo
with {
bar = 42; // To prevent nonRecursive complaining about a missing variable
recursive = rec {
foo = bar + 1; // The number 6, since 'bar' is taken from this
bar = 5; // The order of definitions doesn't matter
}; // The attrset { foo = 6; bar = 5; }
nonRecursive = {
foo = bar + 1; // The number 43
bar = 5;
}; // The attrset { foo = 43; bar = 5; }
};
// [] is syntax for a list
[
(with recursive; foo) // The number 6
(with nonRecursive; foo) // The number 43
] // The list [6 43]
(rec { foo = bar + 1; bar = 5; }).foo // The number 6The example amounts to:
function drawTriangle(left, top, right, color) {
drawLine(left, top, color);
drawLine(left, right, color);
drawLine(top, right, color);
}
drawTriangle({ x: 0, y: 0 }, { x: 3, y: 3 }, { x: 6, y: 0 }, "blue" );
drawTriangle({ x: 6, y: 6 }, { x: 10, y: 10 }, { x: 6, y: 16 }, "purple");
I think some people will argue that macros shouldn't be rare, that you should define a custom DSL for each application so you can work at a higher level of abstraction. However, macros themselves operate at a fairly low level of abstraction (operating on the AST, rather than closer to the problem domain). I would argue that if you find yourself using macros frequently, that's a sign your language is lacking in higher-level abstraction capabilities.
And humans are even better at it, and actually seem to cope with typographically diverse syntax better than the uniform ones. So it absolutely makes sense to have a nice, heterogeneous, human-friendly syntax.
As for writing custom DSLs... I always feel vaguely uneasy when I find myself writing, essentially, an interpreter/VM for a simplistic programming language in a form of a set of library routines/components that I then process to use to build my application logic out of. After all, I am already writing code in a rich programming language, why don't I just use it for my application logic in the first place?
I'd assume it's because that rich language isn't as well-suited to the problem domain as the DSL. I think one solution might be to come up with a more principled way of writing DSLs, that don't require you to revert to writing an interpreter or compiler "from scratch".
Having everything be uniform has other advantages, but I think they end up being more subjective. Some people like the regularity, because they don't have to remember so many different kinds of syntax. Other people prefer different kinds of syntax, because they find it easier to read. I don't think there's an object measure here, because it all depends on your past experience and your preferences.
I’m with you, generally. I have used CL off and on for the last 20 years, and when I come back to it it takes a solid week before I start “reading in Lisp”. It definitely doesn’t come naturally when you spend most of your time reading “C-type” syntax all day. But for some tasks it is an absolutely fantastic tool and doesn’t take that long to get back into.
To do the same thing with Javascript code involves parsing into an AST and walking the that tree; the hard part is that the rules for walking that tree are dramatically more complicated. In the Lisp case, the AST and the original code look very similar; in the JS case, they diverge quite a bit (there's a ton of crazy things in the JS spec if you dig through it)
In Feb 2020, during the first Clojure meetup organized in Chennai, I gave a talk titled "I'm LisP, I'm inevitable" in which I spoke about the journey towards lisp/scheme in muvee's flagship automatic video editing product.
http://sriku.org/posts/inevitable-lisp/
It may be helpful to some to understand the recurrent pattern here so they can detect it early when it happens to them.
It was a case where we'd raised people's expectations so high (what'd take an editor perhaps a week to do happens in about 5 seconds with muvee) that unreasonable performance expectations became the norm :) You can, in seconds, try variations that would cause an editor to pull her hair. To top this all, the "constructor" which is the heart of the product had to be tested against many variations in media and styles.
The details of that journey is probably worth another talk on its own.
muvee released muSE as open source btw and you can find it here - https://github.com/srikumarks/muse . .. and it is (I believe) still usable in the "Reveal Encore" product.
edit: ... and along the way, muvee also built the first mobile automatic video editor which shipped on the ROM of early Nokia camera phones such as the 6630 (https://mobile-review.com/review/nokia-6630-camera-en.shtml .. scroll down to the "Movie Director" section)
The unless example is a good one. So while you can trivially implement 'unless' in lisp, most probably in a large code base, you'll en up with 'unless', 'if-not' and depending on how creative others are probably also 'negat-if'.
Certainly the design of a language should facilitate and even actively encourage writing good code. Attempting to solve systemic organizational or educational issues with it is probably counterproductive though.
imo this is not only a problem in a singular large code base, but also libraries.
I know not everyone likes Go, but I've noticed that i find it super easy to dig into some open source dependency and navigate it, because in some ways Go is the exact opposite of lisp. In terms of the language not allowing a lot of flexibility at least.
I never found navigating foreign java code bases that easy because this project uses spring for dependency injection, that one does some other class loader magic etc, streams, no streams etc. pp.
That said, I do think LISP is cool, but I think it ends up being a right tool for the job kind of thing. I could imagine LISP being nice for things like game engine scripting etc.
The benefit is being able to extend the language. Context managers in Lisp are just macros. Clojure's spec and async are macros.
For sure, the joke about blowing your whole leg off (as opposed to just your foot) applies, but it's the same for concurrency, distributed systems, cryptography, etc. The answer isn't to ban it, it's to exercise more care in the construction of these abstractions so that mere mortals can use them safely.
This makes macro wizardry easier to spot, but restricts how "plastic" the language is somewhat.
In Clojure, "if-not" is part of clojure.core (the equivalent of java.lang in Java), so a decent programmer would almost certainly be familiar with it; and if not, it would be pointed out at a code review.
People use Lisp for a while, so the obvious simple ideas were probably already noticed and implemented.
With less frequent things, implementing the same thing twice under two different names is a problem that happens regardless of the language.
That being said, I wonder how data types would work as opposed to lists. What would a language based around maps look like?
[sendMessage with:arg1 to:arg2]
The relational calculus is an incredibly powerful formalism though, and it's definitely one that has an important place in serious software development.
But if you start with maps, then you can represent a list as a special case of a map in which the keys are integers that get incremented, starting at 0.
I keep thinking code blocks and the struct/map definition could possibly both use `{ ... }`, which might appear a bit like some other languages? What about function calling and definitions?
The S-Expr thing seems to indicate you’d still need a “head” then a list of key-value pairs?
They call "maps" tables. Even arrays in Lua are tables, as are everything else.
Pretty cool concept... And there's even a low level, statically typed, system language that uses Lua as a composer language, Terra: http://terralang.org
In Terra, you manipulate your program using Lua at compile time. Really cool concept.
I'll be sending this to friends who still haven't quite gotten their heads around 'why sexps'. :)
In every single non-LISP language/environment - whether it be C, C++, Rust, Go, C#, Java - I can sit down comfortably in VIM or Vscode+VIM and just start hacking away.
With LISPs, I always feel like I'm fighting my editor - and I am. I know about Emacs+SLIME, I know about paredit, I've seen it in action and it's impressive, I just can't force myself to expand the energy to dive into the Emacs eco-system.
And yes I've tried vim fireplace and Cursive and VSCode's latest ...whatever.....but they all just feel "wrong" in a way, like a square peg in a round hole.
I feel if I could just spend enough time to really understand Emacs and all of it's LISP-yness goodness, I'd enjoy LISPs a lot more and use them more in daily work
Some people program Lisp with very primitive editors, but they better use the Read Eval Print Loop for interactive programming, too. That would be different from the usual C, C++, ... Java programming. You would need to understand interactive programming. That's a level one needs to master - especially in many situations (and where one does not use SBCL) the dynamic typing of much of Lisp requires interactive exploration/debugging.
So, f(a, b, c) becomes (f a b c).
Why move that bracket? The answer is simple: we want the syntax to also denote a serialized data structure: a tree of atomic elements.
So (f a b c) represents a list of symbols F, A, B, C, and the interpretation of this list is a call of the function F on arguments A, B, C.
I tried to write an explanation but found it surprisingly difficult to articulate. In short, there's almost certainly other ways to accomplish the same thing but sexprs are dead simple and they work _really_ well in practice. Just go use them and it will make sense!
https://readable.sourceforge.io
https://www.dwheeler.com/readable/sweet-expressions.html
I would _strenuously_ disagree with that claim, but as previously stated I find the explanation of why I disagree to be surprisingly difficult to articulate. Nonetheless, I find that in practice sexprs are an incredibly effective tool for the task of programming as a whole.
In other words, to evaluate "f(a, b, c)", what you really evaluate is "a", "b", "c", and "f(a, b, c)". You never evaluate "a, b, c". If you move the bracket, then "what is in bracket" matches "what is evaluated".
Edit: Is this being downvoted because I'm hijacking the thread?
I have an interest in functional (Lisp) and decided that for me Clojure was a good choice. It was designed to be used in industry so there's people out there who are using it for real applications. Although I wasn't a fan of the JVM I've learnt my bias was wrong and it's given me access to an ecosystem of libraries.
As I'm more informed now I think that learning Racket, F# or one of the others would also have been fine. But, I don't regret Clojure.
They're all small(ish) communities so you have to be willing to invest in the community, be willing to put together solutions from bits of information and enjoy experimenting.
Whether that's the best use of your time is up to you ;-)
The end of the second last code block should be:
`data.instructions.forEach(...args.map(parseInstruction));`
And the full definition of `data.instructions` that demonstrates the use of this modification would be something like:
`[["rotate", ["rotate", ["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }], 90], 45]]`
Aren't we then back at the problem with using eval?
If we really cared, it would be trivial to extend this to a typed variant or change the evaluation semantics. Here's an example in another language [1].
The author presents it as such, but their structural editing example shows that it takes three operations to negate a condition (create array, write “not”, slurp forwards).
That would be one operation in most line-based languages (write “!”), and it also introduces less line noise.
If you have to modify the behaviour of a specific block of code you only need to edit that structure and your block won't get messy neither requiere special syntax.
As you say, If your "!" transaltes to an expression in your language and you are already writting pure functions that compose niceley then I agree that the benefit would be at a syntax level only, the homogeneity of treating all code (macros or runtime) equally.
A full-featured editor should support both. Line editing is convenient for some editing operations/workflows where you don't necessarily have a well-formed AST at all times. But for many other workflows, structural editing is preferable.
I use Emacs to write Haskell, Scala, Python, Bash, etc. and use at least some structural editing operations all the time, e.g. ctrl-right turns '(a, b), c' into '(a, b, c)' and ctrl-left goes the other way.
I love the idea of data is code is data.
---
In the 1960's the KGB was very interested in learning everything possible about the American space program, sending all sorts of spies to find every possible piece of information.
One afternoon, a breathless spy returned to headquarters with a page of paper in his hand, excitedly shouting to his superior, "Comrade! Comrade! The Americans are using Lisp to write their rocket launching software!"
The commander was skeptical. "How do you know?"
"I broke into their research lab and stole a page from the teletype machine! It's not the whole program, but it's the final page and contains the concluding logic of the program! See for yourself!!!!"
The commander looked at the page and smiled:
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) ))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) ))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))))))))))))))))))) ))))))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))))) )))))))))))))))))))))))))))))))))))) ))))))))))))))))))))))))))) ))))))))))))))))))))))))) )))))))))))))))))))))) )))))))))))))))))))) )))))))))))))))) ))))))))))))))) ))))))))))) )))) ))) )) )) )
---
Jokes aside, I think s-exp is actually a very good way to describe a tree because each nested parenthesis is a nested level in a tree with respect to some, e.g. (a (b (c d)) (e f)) means (b (c d)) and (e f) being a node in a and (c d) in b -> d in c -> f in e
It seems like large-scale composition relies on a well-structured way to define and enforce APIs, and oh wow, typing facilities.
Performance has historically been an issue for Lisp because its model is tightly tied to interpretation, but perhaps modern JIT-style compilation can address this.
Others?
What type of issues do large Lisp projects typically run into?
Of course, it helps to use the right data structures if you want fast Lisp code (using lists for everything is not the right choice).
No, not at all. It is the same level as Java, a bit slower but I guess that is because there is more work done on Java. Just because you compile a language doesn't mean it is as fast as other compiled languages. You can compile python down to a binary but it will still be super slow.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://drmeister.wordpress.com/2015/07/30/timing-data-compa...
https://groups.google.com/forum/?_escaped_fragment_=topic/co...
That could mean 9x slower?
990 seconds instead of 110 seconds.
That hasn't been true since 1962.
I don't see the issue here? What do you think is stopping you from doing this in, for example, Common Lisp?
> and oh wow, typing facilities
Common Lisp isn't typed, but there's no reason a Lisp dialect can't be. In fact, Typed Racket is just such a language.
(+ "hello" 3)
I very much enjoy writing Common Lisp, but use SBCL almost exclusively specifically because of the amount of (static) type checking it does. Even then, there are significant limitations that often leave me frustrated in comparison.
It is heavily typed. Very strongly typed for the most part (except numbers). So typed that it won't accept a "character vector" in the place of a string.
But type checking is mostly at runtime, not at compile-time.
The good part is that you can edit your program at runtime and restart it at the exact point the error happened, very easily.
That being said, I think you'll find truly untyped languages hard to come by. Other than assembly, Forth (similar to assembly in many ways), and esolangs such as Befunge (also quite similar to assembly), languages that see widespread use always check types eventually. Checking types eventually is table stakes. Checking types at compile time, before what you wrote has a chance to explode on you in an unpredictable manner when that one branch for an edge case finally gets taken after a few days of uptime, that's the truly desirable feature that not all languages manage to provide.
Also compare https://people.csail.mit.edu/jaffer/r5rs_9.html and e.g. https://docs.python.org/3/reference/grammar.html which I would consider a relatively syntactically rich language.
The R7RS-small standard [0] has already been ratified. It is a similar size to R6RS as a document size, but the semantics are about 8 pages, and include a bunch of macros that you don't need to implement, but just copy and paste into your implementation.
I'd say it's a similar sized grammar to Python's.