A few tips for writing macros in Clojure | Hacker News Reader

A few tips for writing macros in Clojure

blog.altometrics.com

49 pointsdwighttk43 comments

I'm going to be contrary and suggest that, when you're first learning a Lisp, write a macro every time you think "hey this might be a good place to write a macro" - and then try to achieve the same behavior with a function. There's no better way to learn when macros are useful and when they over-complicate things.

Excellent tips. I'd advise most inexperienced programmers (say under 10 years of lisp) to stick to tip #1. You very rarely need a macro in Clojure.

If it's true that you rarely need macros then isn't the awkwardness of s-expr syntax a very high price to pay just to have access to them?

Sure. The syntax is distracting at first, but in my experience after a few weeks I saw past it. It's a non-issue for me at this point and I dearly miss it when writing Scala on another project.

What aspect do you dearly miss? (Honest question, I've never understood the S-expr trade-off, especially if macros are off the table).

Probably paredit. It's really so much better than non-structured editing. That said, there are structured editing plugins/modes/editors for other languages too.

However, looking back at Lisp/Clojure now I really don't like the syntax anymore. I don't care about the brackets, it's absolutely true that you learn to look past them in a matter of days, but the homogeneity and the "denselesness" really impact my ability to quickly parse an expression. Infix syntaxes make reading code much easier, imo.

I thought that at first too about infixes being better (I've only been using Clojure for a few months), but I'm beginning to find the opposite. The only times when I miss an infix syntax is when doing math

    1 + (2 * 10) - (12 / 3)

is so much easier for me to read to read than

    (- (+ 1 (* 2 10) (/ 12 3)))

But when working with functions, I much prefer the standard Clojure syntax.

I believe the parens are off by one in the Clojure example and that it should be:

    (- (+ 1 (* 2 10)) (/ 12 3))   ;=> 17

I've found the threading operator can help in these cases.

    (-> (+ 1) (+ (* 2 10) (- (/ 12 3))))

Or, line breaks can clarify:

    (- (+ 1 (* 2 10))
       (/ 12 3))

or at the risk of overdoing it:

    (- 
      (+ 1 (* 2 10))
      (/ 12 3))

Now I know at first glance that I'm subtracting the results of those two lines from each other. Indeed the last two might be almost as clear as the infix, and are without any order of operations rules.

Also some don't like colored parens but I find them very clarifying.

I have to disagree.

    f x . g

reads just so much easier than

    (comp (partial f x) g)

Structural editing, paredit, 100%.

I say that not being particularly accomplished with it. 15 years of programming Java, now 3 in Clojure, would dearly miss slurp and barf, the lesser appreciated virtues of homoiconicity.

I took some time to think about this because it started off as a feeling and not a list of reasons.

For me it's mostly centered around control structures - if, for, switch, case (scala's pattern matching). Clojure's control structures are syntactically composed of ()'s and []'s. Scala's aren't. They are a coded into the language's grammar[1][2]. Clojure's ns forms parallel Scala's package and import statements, but are again Clojure's are syntactically composed of ()'s and []'s. I know it's a relatively small point, but control structures make up a significant portion of programs.

As a side note, I took a look at a Clojure grammar also specified for antlr[3]. My initial thought would be that Clojure's grammar would be appreciably smaller. It is, but not by as much as I was expecting. There are a lot of very little additions to the reader that add up [4][5][6].

[1] - Scala's syntax http://www.scala-lang.org/files/archive/spec/2.11/13-syntax-...

[2] - Examples https://github.com/lrlucena/grammars-v4/blob/master/scala/sc...

[3] - https://github.com/laurentpetit/ccw/blob/3738a4fd768bcb03996...

[4] - http://clojure.org/reference/reader#macrochars

[5] - https://yobriefca.se/blog/2014/05/19/the-weird-and-wonderful...

[6] - See "Special Characters" http://clojure.org/api/cheatsheet

I'd say it's more true that you rarely need to write a macro than it is that you rarely need to use them. Lots of very useful things across the language, from builtins like the threading macros (->, ->>) to libaries like Compojure, are built on macros.

I've found s-expressions to be one of my favorite things about Clojure. I understand that many people dislike them, and that many people get used to them, but I loved the syntax from the get go.

I have a pet theory that someone will put a Python skin on Clojure and have significant success with it.

You can't. The reason you can use macros so easily in LISP is because of homoiconicity: https://en.wikipedia.org/wiki/Homoiconicity

Your macros will break all over the place in languages that don't have this property.

SRFI-110 says you can^1: "Sweet-expressions are general and homoiconic, and thus can be easily used with other constructs such as quasiquoting and macros. In short, if a capability can be accessed using s-expressions, then they can be accessed using sweet-expressions. Unlike Python, the notation is exactly the same in a REPL and a file, so people can switch between a REPL and files without issues. Fundamentally, sweet-expressions define a few additional abbreviations for s-expressions, in much the same way that 'x is an abbreviation for (quote x)."

http://srfi.schemers.org/srfi-110/srfi-110.html#wisp

^1 [ed: well, not "just python syntax", but if you wanted that, you would be better off using python. But python-like syntax/skin is absolutely doable. ]

Thanks, I learned something today. :)

Dylan also has macros.

As does Rust.

But Rust isn't a Lisp based on M-Expressions like Dylan is.

No, but it's also not a homoiconic language, and it supports syntactic macros just fine. Homoiconicity might make procedural macros convenient, but it's far from a requirement. And then pattern-template macros are just as convenient in a more "syntactically rich" language as they are in a homoiconic one.

See also: camlp4/camlp5.

And then there's the idea of pattern-match rewriting systems. iirc, GHC has something like that to rewrite certain expressions as optimizations, like:

    (map f) . (map g)  =>  map (f . g)

In a pure language, the two expressions are equivalent, but the latter is more efficient because it traverses the list only once rather than twice.

The Cat programming language had a similar sort of "macro" system, but its web site seems to have disappeared. Shame, it was a really neat little concatenative language.

Homoiconicity is certainly neat, but it's not about macros at all. It's about having a Lisp-style `read` function. That's it.

Also, Dylan's syntax wasn't based on M-expressions per se, at least not in the "classical" LISP 1.5 sense of the term. It was just an Algolesque syntax rather than a façade layered above S-expressions. If you can claim that Dylan was based on M-expressions, then you could just as truthfully claim that Lua, Ruby, JavaScript, etc. are based on M-expressions. I won't say that it's strictly false, but I don't think that's quite how most people familiar with M-expressions think of them.

> Also, Dylan's syntax wasn't based on M-expressions per se,

I know, my remark was more into the sense of being based as departure point, not as being a pure M-expression implementation.

Well, there's already wisp/sfri-110 for scheme, it appears the preprocessor should mostly work with clojure -- although one would probably want to port it over, or at least try to get it working on the jvm in order to have it "work where clojure works" (I have no idea how viable it would be to get it running with jython and something like kawa scheme out of the box):

http://www.draketo.de/english/wisp#text-table-of-contents

[ed: For Racket, see also: https://github.com/takikawa/sweet-racket ]

[ed2: See also the "readable" package for Common Lisp and for Scheme: https://sourceforge.net/p/readable/wiki/Install-howto/ ]

This is interesting, but also a little discouraging. Coming from a background of Python and JavaScript, and using a little bit of Elixir (which also supports macros) lately, the power of macros for defining DSLs is the main draw for me.

As an example, I've been using GraphQL quite heavily for the last 8 months, primarily with JavaScript and i've found it to be a fairly tedious experience. I tried out an Elixir GraphQL library that's heavily based on macros and was almost immediately more productive - despite not even knowing the language.

Similarly, coming up with a nice API for working with forms seems to be pretty much impossible with JavaScript. I've evaluated dozens, and not one comes close to the simplicity of Django's forms (which is only possible thanks to Python's metaclass magic).

Having sampled the power of languages with macros, I'm naturally drawn to Clojure, simply because it has a very decent JavaScript implementation. I'm sure there are other very good reasons for using the language, but right now macros are the motivation.

I'd like to understand why one would need 10 years of lisp experience before starting to author their own macros, it feels like a philosophy that's just going to stop people trying out lisp.

It's OK to be discouraging to a certain degree. It helps stop people from cooking up "clever" solutions.

Well I try to use the right tool for the job, and it's important that code can be understood easily when it's read by other developers. But I think the 10 years with Lisp argument needs to be better justified. :)

Oh, I was not making a personal comment about you. It was a comment in your first point. That being discouraged by others is a sign that one should thread lightly. I was not discouraged by anyone when macros appeared on my horizon. And boy did I write some bad code. :)

Makes sense :)

I guess i've just heard for so many years how badly wrong macro usage can go, that i've not even been tempted to try languages that use them - until very recently.

As long as it's not production code: You should actively seek out and learn/try as many things possible. Don't let anyone stop you. I once worked on adding a "maybe" keyword to python because boredom. It was a disaster. Fun though.

The article says that you shouldn't write them unless you need to, not that you shouldn't write them. He lists quite a few scenarios where they should be used, one of which is DSLs. :)

Also, given that Clojure is a lisp, I think you'll find you need macros left often than a language like elixir. They're still incredibly powerful, but you can do so much in Clojure without them that you would otherwise need a macro to do.

I've recently (about two weeks ago) picked up Clojure for use on a side project after getting frustrated with trying to use Haskell for something non-trivial -- not smart enough for Haskell, apparently :D

Haven't messed with a LISP language since college but clojure has grown on me considerably. I already like it more than Scala... though I did almost immediately find myself reaching for a monad library (cats seems good so far) :) Haven't felt the need to write my own macros yet, but certainly many of the libraries I'm using (compojure) make heavy use of them.

Too bad my day job is all Java, all the time :( My continual requests to use Scala have been shot down over the last couple of years -- "We want something maintainable, where am I going to find Scala devs when you leave?" So obviously Clojure isn't going to fly :)

That seems off by about 1 order of magnitude...

Lisp is getting on a bit, but even the greyest of Lisp hackers haven't quite racked up a hundred years of Lisp...

maybe in the other direction? :D

yes, I was thinking that a programmer with about 5-10 years experience in other (non-lisp) programming languages and 1 year of Clojure should be able to tackle macros effectively...

In my home-made lisp, a macro can be an anonymous first-class value.

e.g. this is early in the stdlib and is what sets up the ability to write 'foo as a shorthand for (quote foo):

    (extend-parser "'"
                   (mac (x) (list (quote quote) x)))

Haven't yet contorted myself into wanting to map a macro value over a list, though!

This is true in emacs lisp too, although the syntax for it is a bit wonky:

    ;; increment as a macro, anonymously
    (= ((macro lambda (x) `(+ 1 ,x)) 10) 11)

Oh, good. It's not a completely stupid notion then. Someone on another site told me an anonymous macro is a contradiction in terms.

I have a diagnostics toggle (SIGUSR2) that traces what's happening with macros for debugging. Can you think of a better way to convey it? http://i.imgur.com/kFzEvd6.png

That looks like a really handy feature. I'm assuming you still have macroexpand, macroexpand-1, and a macroexpand-all available?

I imagine that you could implement a similar feature for tracing, and if you were careful writing the tracing code, you could then use that code for this feature, as you could say "trace the complication e.g. (trace (eval (read ...)))" and then maybe feed in an optional list of functions to elide . This would be super handy to have as a toggle'able feature, for both macros and normal user code. I use clojure.tools.tracing a lot, and it's a life saver.

I don't have a macroexpand function and the self-imposed limit on the number of primitive functions (32 language-related + 16 OS-related) has already been reached! Having a whitelist for macros whose expansion should be traced would be good as there's a lot of noise.

"Non-intrusive" tracing of any expression can be done, using an @ prefix, based on this:

    (extend-parser
      "@"
      (mac (x)
          `(let once ,x
                (seq (eprintln "TRACE: " once)
                     once))))

e.g.

    (def third (div 100.0 3.0))
    => TRUE

    (add @(mul third 2.0) third)
    TRACE: 66.66666666666667
    => 100.0

That's a good point that you mention tracing needing to be toggleable - it'd be nice to be able to leave all the @s in code. I guess the macro for "@" could either be conditionally replaced with an effective no-op, or it could internally check for the existence of (or truthiness of) a global tracing-enabled definition.

I've heard it said that Clojure's macro system is most similar to Common Lisp's macro system and that Racket (and Scheme) style macros are more powerful. Can anyone compare the differing approaches?

It is more similar to CL's macro system in that it doesn't force you to write hygienic macros -- though I think CL's macro system is actually more powerful than Scheme's since it does let you write unhygienic macros.

Clojure doesn't allow user defined _reader macros_, which means it is less flexible than either CL or Scheme. Basically, you can't really define your own syntax parser in Clojure, you are limited to what the Clojure reader can already parse. In CL and Scheme you don't have this restriction.

Scheme has historically been used as a research vehicle for macros. When the fifth revised specification of Scheme (known as R5RS) was written there were no consensus on the best practices for writing a system supporting unhygienic macros.

Since then both R6RS and R7RS has support for unhygienic macros.

Note that there are two important properties of the macro systems of modern Scheme implementations: hygiene and referential transparency. The second property is often overlooked.

The standard syntax-rules macro system of R5RS Scheme is hygienic

    If a macro transformer inserts a binding for an identifier (variable or keyword), 
    the identifier will in effect be renamed throughout
    its scope to avoid conflicts with other identifiers.

and referentially transparent

    If a macro transformer inserts a free reference to 
    an identifier, the reference refers to the binding 
    that was visible where the transformer was specified,
    regardless of any local bindings that may surround
    the use of the macro.

The first property (automatically renaming of identifiers) is prevents common errors. This property is easy to hack around in macro systems without automatic renaming. The macro writer simply generates a new identifier (gensym) each time a new identifier is needed.

The second one referential transparency is the killer feature. Names inserted by a macro refers to binding where the macro is defined. This means that the macro writer has control over the meaning of the binding of identifiers in the expansion. In other words: a user of a macro can not by accident rebind or assign values to identifiers inserted by a macro use.

When referential transparency is not supported, then one must be careful to load libraries in the right order:

http://fare.livejournal.com/146698.html

Reply on news.ycombinator.com