Lisp in Small Pieces of Clojure, part 3 | Hacker News Reader

Lisp in Small Pieces of Clojure, part 3

gmorpheme.net

183 pointsghosthamlet48 comments

Slightly unrelated, but to those who have never tried Lisp because of the parenthesis (like me until recently), here's my take:

I've recently started hacking with Elisp, and I really like it. Parenthesis are obviously pretty awkward at first. But now I wouldn't leave them for anything else. As it turns out, the brackets make it easy for your editor to understand the semantics of your code, to a whole new level. So your editor is now actively helping you navigate and edit your code. You go from expressions to expressions, add one after an other, transpose them and their parameters, absorb other expressions, insert them etc. It's true magic. I thinks that's why some don't like the fact that CL-loop is not "lispy" enough: you lose your editor's help inside a CL loop.

So the parents are actually really cool: you can basically tell your editor to move code around and insert / delete it instead of doing it yourself. With these tools (smart parents and paredit on emacs), your parenthesis are always matched ! And because it's lisp, you're actually forced to use these tools because it's a nightmare otherwise (which makes it kinda hard to learn at first), so you actually do learn them. I haven't used smartparents with other languages, but I hear it works pretty well.

Finally, it seems that lisp dialects compile to almost anything. Which means that you can use a lisp editor and get all its benefits even when working in clojure on the JVM or clojurescript on the browser etc. And once you're good at lisp, you can pretty much make Emacs do anything.

Conclusion: Lisp Is Pretty, give it a try when you get a chance ;)

[0] https://github.com/Fuco1/smartparens

EDIT: fixed some sentences so they make sense...

I also never really liked the loop macro. I vastly prefer the racket-style for loops over the loop macro. There are some odd corners where the loop macro is cleaner (destructuring bind),but that is achievable in racket as well.

I still haven't actually had an occurrence when for/fold hasn't been enough for my special case needs.

If you need it, there is also an implementation of CL:LOOP for Racket.

Which is rather slow and doesn't add much. Other than awe that someone actually managed to do it :)

Maybe it's not so slow with a better compiler, like the one from Chez Scheme - or is the implementation the problem?

Most scheme implementations are better at optimizing non-mutating code since it can draw conclusions about types and such easier. Chez probably does a better job than racket because Chez is pretty spectacular :)

Fortunately, if you want that sort of thing in Lisp there's always the iterate library: https://common-lisp.net/project/iterate/

It looks pretty good to me, and I hear good things about it, although frankly I've always just used LOOP.

Still very much a language on its own. Rackets loops look very much like a regular lisp construct.

It sounds like you're the kind of person who'd prefer DOTIMES[0], DOLIST[1], and so on. Lisp has a lot of different ways to handle iteration, it's the curse of flexibility when people have such varying tastes. Another great option that often gets overlooked is the Series package[2], which lets you write in a functional style and then mostly optimises that away.

0: http://www.lispworks.com/documentation/HyperSpec/Body/m_doti...

1: http://www.lispworks.com/documentation/HyperSpec/Body/m_doli...

2: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node348.html#S...

DOTIMES is not my cup of tea. I don't like to have to use mutation to get things done. I prefer to have an explicit but more verbose style (named let) or to use a for loop.

A matter of preference of course. The benefits of for loops or named lets is that they are functional and thus continuation safe. I did a bunch of continuation-based nondeterminism things last year, and knowing things weren't mutating was nice.

First of all, thanks for clearing things up about Lisp! I have a question which I think is the concern of many people in Data Science sector as well: I love the idea of Lisp and have tried it before. I also found the idea of macros super intelligent. But when it comes to data science, I face a lot of problems with Lisp. Currently I do my job with Python (mostly) which benefits from a plethora of libraries. But Lisp falls short in this area. The number of available libraries and active communities is not comparable to that of R/Python/Java... And the language itself is fragmented: There are just too many "Lisp"s out there: CL, Scheme, Racket, ... I even tried Hy once, but that's not quite the experience I want.

Could any one suggest how one might do data science and analytics on Lisp?

P.S.: Hy Lang: https://github.com/hylang/hy

wow, Hy Lang is sweeet !!! Will check it out, thanks ;)

Considering your question, I'm not a data scientist but this book seems to come up often when search for 'lisp ai' on google. It's pretty old, but maybe it's a good place to start: https://github.com/norvig/paip-lisp

My biggest complaint about lisp (which I love dearly) is that the overwhelming majority of the lisp community (or lisp communities) have already settled on Emacs as the obvious choice in editors. And for good reason, the emacs tooling is unparalleled. And because everybody is using Emacs, non-emacs tools are neglected at best. It seems quite common for stand-alone REPL's (let alone extensions for other editors) to be neglected and shoddy because everybody influential in the language's development are already using emacs and have no use for a stand alone REPL.

It's great for people who already use Emacs. But a lot of people have deep personal investment in other editors because they got started years ago programming in some non-lisp language for which Emacs was virtually a footnote. And sure, learning Emacs is possible for established programmers who are new to lisp, but learning a new editor in addition to a new language is a hurdle most languages don't have (or if they do, it's often a reasonably simplistic GUI IDE.) It's amazing how many books, manuals, etc start off on the first page saying something to the effect of "You don't have to use Emacs... but seriously go learn Emacs."

This all said, there are a few ways to get psuedo-SLIME behavior out of Vim but they all seem to suffer from lack of TLC and established lisp users usually look at you strange and question why you're not just using Emacs. (Also the s-exp structure of lisp code still makes it easier to manipulate in Vim (sans plugins) than other languages. The % movement really gets a workout when editing lisp code)

If all you want is Vim-but-with-SLIME, there's always evil-mode.

And it's not that you have to use Emacs to become a proficient lisper... it's just that once you're used to doing things the lisp way, you'll want a hackable environment that also does things the lisp way. This includes the editor.

That said, I prefer lisp for my smartphone programming. Not because I have emacs on my phone, but because lisp is such a great choice precisely when you don't have a good editor and keyboard.

But you can have Emacs, even with SLIME, on your phone! At least if it's Android. Termux is amazing!

I have had some problems with QuickLisp in the CL REPL app (which uses ECL) so I might actually give this a shot. Do you happen to know if it supports libssl? Android apps no longer have automatic access to native shared libraries.

I should say that I have no doubt my issues are of my own making. I still haven't figured out a way to run a custom non-manufacturer based AOSP ROM on my phone without weird issues.

… but seriously, go learn emacs!

After using it long enough, you'll realise that no editing environment even comes close. emacs really is superior to everything else out there: org-mode, magit, notmuch, gnus, slack-mode, all the different programming language modes, eshell &c. combine to form the best editing & development environment out there. From that first moment you write a little bit of elisp to make your life better and for the rest of your life (or at least as long as your eyes & hands permit you to use a computer), emacs really is worth it.

Not using emacs is a bit like trying to walk from Mexico to Canada, with one leg in a splint: possible, but why hobble yourself?

I hear you, and it's not like I haven't dipped my toes in to test the waters. But after 20+ years of vim usage it's not so easy to turn on a dime. (Particularly when Vim is more than sufficient for the work I'm actually getting paid for.)

FWIW spacemacs is a pretty decent vim-alike. Not perfect, but still pretty awesome. It's tougher, though, if you've been using a lot of extensions to vanilla vim.

Because Emacs kind of related to Lisp Machines editors.

The best Lisp editing experience is only available on commercial Lisps like Allegro.

As someone who likes Lisp but who also likes my editors to stay focused primarily on just editing text and letting me type, I think the minimal editor support for a happy enough Lisp editing time is visually matched parens. (rlwrap'ing a terminal REPL counts.) Everything on top of that is gravy, up to the point where you've had enough, and I'll readily admit such a minimalistic setup is probably not the maximal happiness for anyone but at least puts you about at the same level as other language users when those users are outside of a do-everything IDE too...

Lots of people like the auto-typing paredit intellisense style of tooling, I can't stand it personally (with the exception of megalithic Java projects where they seem like a necessary evil and both eclipse and intellij are utterly slow at it but that's another rant...), but I do appreciate vim's "%" command to jump between matching delimiters and combine that with other text editing forms, plus rainbow parens so everything is matched without having to highlight.

Anyway sometimes I wonder if a lot of newbies get turned off by the parens (in one of the various ways) then get turned off again when "use emacs with all these things!" is the only suggested way to manage that.

I personally would not want to edit Lisp code without some basic s-expression commands - especially the command to indent/reindent code. Indenting Lisp by hand is a waste of time.

I personally don't like Lisps, although I agree with you that when I learned them that emacs felt like the best editor to use (though I ended up using the Racket IDE for other reasons). To me, the biggest advantage of Lisps is that the syntax is so simple that building a macro system on top of the language is far simpler than doing so in other languages. Lisp has practically no syntax, so the macro system's complexity is much lower. Because the language's AST is simpler it is easier to deconstruct input syntax and generate output syntax.

Also, Lisps have no static type system, and for whatever reason static type aficionados have an aversion to macros or else want to make macros typed, and combining macros with type systems again makes macros far more complicated. One need only look at Scala's many attempts at adding macros to the language to see just how hard it is, and I personally have little expectation at this point that Scala will succeed in integrating macros into the language.

At the end of the day, the major selling points of Lisps are macros, and, for Schemes, baked-in delimited continuations. Most or all other features can be found in other languages in various forms.

Some lisp have static type system, you talk about racket which has typed-racket. Also i don't think that static type aficionados want to make macros typed, since the output of macro-expansion phase goes into the type checker anyways; what people may want is control over errors.

Racket is typed, but I don't think the macros themselves have types. Contrast that with both template Haskell and scala macros, both of which are typed and far more limited in what they can do. That said, it looks like OCaml macros are untyped, so perhaps it is just Haskell-influenced communities that generally look down on macro systems and want to subvert them under the of the type system.

> Lisp has practically no syntax,

That's not the case. You think of s-expressions as the syntax for Lisp, while s-expression is a data syntax. Lisp syntax is defined on top of s-expressions. Think about LET. It has syntax. LAMBDA: it has syntax. And so on. Most macros implement syntax.

> Most or all other features can be found in other languages in various forms.

It's not the amount of features, it's the integration which makes the character of a language.

I implemented an R5RS macro expander, and, at least from the perspective of Scheme and macros, the only truly base is application, and all other forms can be reduced to it. For my expander I actually implemented all let forms, for instance, as macros that reduced to lambda applications.

Now other lisps might have more forms that don't reduce quite nicely, I'm not sure, I still strongly believe in my original point that lisp practically has no syntax, and that's one of the reasons its easy to create macro systems for it.

That one can expand macros does not make them non-existing. The programmer writes the macro forms, not their expansions.

If you actually look into the R5RS document, you can see that chapters 7.1.3-7.1.6 define the syntax for that particular toy version of Scheme.

Lisp usually has three syntax types: function applications, built-in operators and macro operators.

That you can implement some constructs by expanding them to LAMBDA expressions says nothing about the syntax of Scheme or Lisp. Lisp users don't develop in pure lambda calculus. Each built-in operator and each macro operator are adding syntax.

One can reduce arithmetic to lambda expressions - this does not mean that Scheme has no arithmetic operators. R5RS documents arithmetic and not their lambda variant. Same as R5RS defines the syntax for DEFINE, COND, IF, CASE, SYNTAX-RULES, ... That's what the progammer sees, not LAMBDA.

Take any real world Scheme, like Chicken Scheme -> lots of macros have added syntax. That's what the programmer uses, not lambda calculus.

What is your rewrite rule for reducing (set! var value) to application?

For iteration and selection, did you just turn everything into lambdas? E.g.

  (if x y z) -> (__if_fun (lambda () x) (lambda () y) (lambda () z))

If you minimize the special forms, you have fewer cases in the expanding code walker; but then the compiler has little information for producing good code.

Note that a compiler doesn't necessarily just let every instance of function application be function application. If the above expansion for if is forced upon you as a compiler writer, you can recognize the __if_fun function as a built-in and open-code it, effectively undoing the reduction to function application.

Check that all arguments are lambdas (and diagnose if they aren't zero-argument lambdas) destructure them and turn into an open-coded branching construct.

The cost for fewer code walking cases in the macro expander ends up being more cruft in the compiler.

About let to lambda: in non-toy implementations, you almost always want to go the other way: recognize lambdas which can be turned into lets.

> For my expander I actually implemented all let forms, for instance, as macros that reduced to lambda applications.

> Now other lisps might have more forms that don't reduce quite nicely, I'm not sure, I still strongly believe in my original point that lisp practically has no syntax

This isn't an argument against Lisp having syntax: it does have those operators, and each one defines its own syntax. The fact that they can be written entirely in the language itself doesn't mean Lisp doesn't have syntax, it means that Lisp has extensible syntax: you can add more syntax whenever you like, and in some dialects, you can remove or replace the pre-defined syntax forms. Syntactic sugar is still syntax, too: in C, arr[i] is just sugar for *(arr + i), but I don't know anyone who would claim that array indexing isn't part of C's syntax.

I remember once reading a quote to the effect "Don't think of them as parentheses, think of them as handles."

For anyone who'd be interested in trying a Lisp (with Clojure probably being the most mainstream one at the moment) and who's been put off by the parens, paredit and/or Emacs:

- Parinfer [https://shaunlebron.github.io/parinfer/] - makes the parens follow indentation and vice versa and effectively makes them a non-issue at worst and a great advantage at best (you can combine it with paredit later on to perform all kinds of ninja magic when editing code)

- Cursive [https://cursive-ide.com/] - a Clojure plugin for IntelliJ IDEA for a more 'just works' experience. Integrates parinfer, among other things, and is generally pretty great (I'm a happy customer).

And last but not least - rainbow parens to keep nested expressions readable. Pretty much every Lisp editor has them more or less out of the box.

I would very much NOT recommend using something like parinfer; IMO if one is learning lisp, one ought not to try to avoid the parentheses, but rather learn the advantages of using them. So instead of parinfer, use paredit and learn the key bindings for moving up/down/formard one sexpr at a time. It's frustrating at first, but after a few days, you will never want to go back to line oriented languages.

Well, first of all let me emphasise again that it's not an either/or proposition, they work together perfectly fine. Personally I use paredit extensively and yes, going back to a language/environment without it feels like having to work with my hands tied behind my back, but I still make use of parinfer as well.

Sometimes just deleting a couple of lines in Vim (IdeaVIM in my case) and knowing that parinfer will take care of the parens feels faster and more comfortable to me than using paredit. YMMV.

And as far as beginners are concerned - for someone coming from an OOP/imperative programming/C-style syntax background, a LISP will seem very foreign and weird for many reasons, the parens being just one of them.

I think having the ability to get your feet wet and start dabbling without an upfront investment is a good thing and the "jumping into the deep end" approach isn't for everyone, especially not for many beginners who at this point have only a very vague idea that a LISP might be for them and are already facing a number of new concepts that they need to get their heads around.

Parinfer requires no investment, Paredit requires some. Not much, but still. And going from one to the other as one feels drawn to explore the language is, I think, a perfectly natural transition, which requires no unlearning or changing of habits whatsoever, and as such it's also a more accessible path to getting into LISP. In my book, that's a good thing.

But here we're beginning to move into a general attitude/philosophy territory that's hard to argue about, so I'd be perfectly comfortable with agreeing to disagree and leaving it at that.

Counterpoint: there is justification in the claim that the possibility of parinfer is one of the advantages of parentheses. So maybe: use parinfer, but don't overlook the other advantages of parentheses.

    While deep inheritance hierarchies 
    may seem like a great way of 
    modelling your concepts when you 
    have a blank slate, it’s a 
    recipe for some extremely tight 
    and non-obvious coupling that 
    can deadlock refactoring attempts
    in later phases once those concepts
    have shifted and the original 
    model is no longer a good fit.

This is probably the best critique of nominal inheritance I've read. It really does capture the problem and nicely alludes to the struggle of arguing against an inheritance structure. That it usually starts with many benefits but can easily grow to a constraint.

I kept trying reading this as a haiku.

Ha! Apologies. I gave a somewhat vain attempt at quoting so it would show on phones ok.

This is part three.

Part one: http://gmorpheme.net/lisp-in-small-pieces-of-clojure-part-1....

Part two: http://gmorpheme.net/lisp-in-small-pieces-of-clojure-part-2....

That's whats mentioned right at the start of the article?

  This continues a translation of various parts of Lisp in Small Pieces into Clojure. For earlier chapters, see:

  Lisp in Small Pieces of Clojure - part 1.
  Lisp in Small Pieces of Clojure - part 2.

Oh this looks great. I loved L.I.S.P. I did plan to dust off my French and try the second edition. But I'm sure I'll enjoy this just as much.

A testament about the quality of Lisp is the breadth and depth of its literature: SICP, PAIP, On Lisp, Lisp in Small Pieces, CLtL...

The only great book I'd like to see written in Lisp is CTM.

"CTM"?

https://mitpress.mit.edu/books/concepts-techniques-and-model...

I'd add Let over Lambda (LOL) to that list.

For those who are not francophones, there is also an English translation.

I applaud joyfully.

(2015)

Reply on news.ycombinator.com