Not sure if this helps you. But let's define a macro A:
CL-USER 32 > (defmacro a (b c)
(print (list :macro-expansion b c))
(list 'print (list 'quote (list :runtime :b b :c c))))
A
Now we can use this macro in some code:
CL-USER 33 > (defun test ()
(a 21 42))
TEST
CL-USER 34 > (compile 'test)
(:MACRO-EXPANSION 21 42)
TEST
NIL
NIL
CL-USER 35 > (macroexpand '(a 20 30))
(:MACRO-EXPANSION 20 30)
(PRINT (QUOTE (:RUNTIME :B 20 :C 30)))
T
CL-USER 36 > (test)
(:RUNTIME :B 21 :C 42) ; <- the printed output
(:RUNTIME :B 21 :C 42) ; print also returns its arg
Thus it has to do with code generation and code execution at macro expansion - nothing about 'delaying' something.
The example isn't useful, but imagine a macro INFIX
(infix a * b + c)
(+ (* a b) c)
And most practical uses of macros involve passing in expressions that will be later evaluated verbatim, e.g. (time (reduce + (range 100)))
I was only intending responding to this:
"That's a common misconception that Lisp macros are mostly used to 'delay' evaluation."
Which just seems to me confuses the matter in response to the article that was posted. At a very low level, macros operate by having the evaluation of their arguments delayed. At a higher level, they're used to implement sugared forms which usually contain valid code (though not necessarily code that makes sense in isolation).
I know you're very experienced with lisp, and I'm not trying to correct you technically, but to me this stuff only clicked when implementing a metacircular evaluator and learning when and how to perform evaluation. The blog post to me reads like the author was going through a similar process.
It doesn't 'delay' anything. It just rewrites at compile time
(infix (avg b) * (sgn a))
(* (avg b) (sgn a))
> At a very low level, macros operate by having the evaluation of their arguments delayed
Not at all. The main purpose of macros is to compute code from arbitrary expressions at compile time. There is no 'delay' -> it's mostly about rewriting expressions at compile time.
> implementing a metacircular evaluator and learning when and how to perform evaluation.
You can build a macro expansion phase into an interpreter. That's what Lisp interpreters do. But the main purpose of macros is to enable code transformations at compile time and thus only to have a cost at compile time, not at runtime. In an interpreter the macro expansions would also happen at runtime and have a cost there. But Lisp compilers are popular and to make the most out of code transformations at compile time, the idea of macro and their expansion at compile-time was introduced. Thus one gets for example new embedded languages, without the need to parse them at runtime.
I'm not arguing with you about what happens in any production lisp. I'm sure you know more than me, but I do understand how they're implemented in practice, and the warts that brings - eg no (apply or <list>), and first class macros are so esoteric purely from the runtime cost that they're barely even written about in the literature.
I'm just saying the mental model of where macros fit in in an interpreter has value, and that view has not changed. In my opinion you're an expert coming in correcting a bunch of people attempting to learn the basics, and you're not wrong, but you are muddying the waters.
If you mean run-time functions that act a bit funny about evaluation order, that mental model is incompatible with common introductory examples like `let`.
((fn [x]
((fn [y]
(+ x y)) 3)) 2)
fn: (a b c) => (call a (eval b) (eval c))
mac: (a b c) => (eval (call a b c))
Everyone jumped on it, as production lisps don't implement macros this way. But it's the natural way to implement them in a SICP style metacircular evaluator, and a good way to learn them.
This is about mental models, not implementation details. And in my opinion this thread is full of experts who have forgotten what it's like to not understand how macros work.
Everyone jumped on it, as production lisps don't implement macros this way. … This is about mental models, not implementation details
It looks to me like you're the one who brought up implementation details, talking about what happens "at a very low level." You're the one who has presented a questionable implementation strategy as a mental model and has been corrected on it multiple times by multiple people. Everyone else has been describing the semantics of macros. Having separate timelines for expansion and execution is a semantic distinction, not merely a common implementation strategy.
And in my opinion this thread is full of experts who have forgotten what it's like to not understand how macros work.
Your confusion is because you insist on a mental model of macros that is incompatible with their semantics. Letting that model go is necessary in order to understand macros. Otherwise, refusing to distinguish expansion time from run time, you are working towards reinventing fexprs.
http://matt.might.net/articles/metacircular-evaluation-and-f...
SICP is a great way to learn lisp and learning macros by extending the evaluator is a natural progression.
I don't care how many lisp experts are "correcting" me on the way it works in the real world, because:
a) I actually do understand how they're implemented in existing production lisps. No, really. I am not confused. In my own toy lisp compiler, I rely on first class macros and am attempting (and so far failing) to use partial evaluation to make the implementation cost somewhat sane. I am under no illusions this is normal.
b) Being an expert doesn't automatically make you good at teaching (often the correlation is reversed), and the article being commented on was a tutorial on macros.
The smug lisp weenie trope exists for a reason, and it's that the lisp community is too busy falling over themselves to prove how smart they are with arcane knowledge to give a shit about the experience beginners face. Clojure gets no end of flak from CL users who bristle at the suggestion that it even be called a lisp, but the community actually gets it and does a great job of teaching.
I have an opinion on how best to teach macros. Maybe I'm wrong. Maybe it's not a good idea to extend SICP with macros, or maybe there's a better way, and I'd love to hear arguments in those directions. But correcting it by explaining how they're implemented in CL is absolutely baffling to me.
I'm not sure why you continue to insist that responses to your comments are just about how macros happen to be implemented in Common Lisp.
But the point of a toy model is that you can read an implementation, build your own, reason about it and send code through it with little difficulty.
Once you've built eval and sent real programs through it, it's not a big jump from that to understanding how real tools work, which can generally be done incrementally.
That was certainly my lisp journey, and that includes macros (which were a separate step). After implementing them in eval, learning how they're in the real world was about as challenging as reading a restaurant menu.
SICP is a book for beginner computer science students, with a focus on functional programming as base.
As a Lisp book it's not really good, since it covers only very little actual Lisp and uses only a core Scheme language.
There is literally nothing about macros in SICP. They are mentioned only in a few places (for example it is mentioned that DELAY would be implemented by a macro).
> The smug lisp weenie trope exists for a reason, and it's that the lisp community is too busy falling over themselves to prove how smart they are with arcane knowledge to give a shit about the experience beginners face
Since you ignore the large amount of literature, teaching & research on macros in the Lisp community, what you write is rather strange. Surprise: people teaching Lisp in the 60s already 'gave a shit about the experience beginners face'.
> I have an opinion on how best to teach macros. Maybe I'm wrong. Maybe it's not a good idea to extend SICP with macros, or maybe there's a better way, and I'd love to hear arguments in those directions. But correcting it by explaining how they're implemented in CL is absolutely baffling to me.
Lisp teachers usually taught macros in context of actual Lisp languages and actual usage examples. Personally I prefer that as a first approach, over trying to explain how to extend a toy implementation with some homebrew macro implementation.
I regret my initial comment, which was written in haste. Had I known the sheer volume of people that would jump out the woodwork to demonstrate their superior technical knowledge while avoiding the argument I was actually making, I'd have been a lot more careful about what I was trying to communicate (or more likely not have bothered, which would have saved us all a lot of time).
That is where the "smug lisp weenie" trope comes from, and it's real. I don't know what it is specifically about lisp that brings it out; I can't imagine having this discussion about C#, TCL or SQL.
Were I disagreeing technically, it would be a different matter and I'd have long ago hit the books to see where I fucked up. The combined lisp experience on this thread is at least an order of magnitude more than mine, and most of you are probably smarter than me to boot.
There is already a bunch of good material out there, teaching the use of Lisp macros for both beginners and for medium-level users. Advanced-level users will find a lot of teaching&research in the Scheme literature/...
Paul Graham himself has written a good and well-known book on it. it does a good job explaining macros and their use. It's called 'On Lisp' and there is no need to replicate its content here.
Now we have gone full circle, again.
A good teacher is engaging and glib, and tells the right sorts of little lies at the right times to lead students through the steps of "Wittgenstein ladders"; but he or she can't be flat out wrong about the subject material.
I don't think it is overly useful to explain people how to extend a SICP style metacircular evaluator with some custom macro system. It's often a part of the Lisp journey, but not something I would recommend for basic Lisp teaching.
If I want people to program in Lisp, I give them a real Lisp and teach them how to program with it.
> This is about mental models
Sure, but you are teaching them the wrong mental models (macros tied to an evaluator), while I would focus on the model of general code transformations and the tools to make effective use of them.
> And in my opinion this thread is full of experts who have forgotten what it's like to not understand how macros work.
Not sure who you talk about, but I have been explaining macros to people already in the 80s.
Edit: In fact, I strongly suspect that this resulting unsuitability of Lisp for proprietary code played a major role in its commercial failure before the widespread advent of services.
Whether the compiler runs inside an 'image' or in some batch mode is also not important for macros.
Because macro expansion does something before evaluating the arguments, you can say that evaluating the arguments has been delayed.
I feel this is just looking at the same thing from two different directions.
Of course, macro expansion does _more_ than just delaying the evaluation of the arguments, and if people say that macros delay evaluating the arguments, you might think that's all they do.
Delayed (lazy) evaluation merely prevents an operand being evaluated at the time of call and until you want to read its reduced value, after which it is automatically evaluated.
It does something independent of evaluation. When the code gets compiled, the macroexpander already has transformed the code. The code might never be evaluated in this Lisp. It might be written to disk and later be loaded into another Lisp.
If something does not get evaluated, gets evaluated later, gets always evaluated or never -> that depends on the generated code.
Thus 'delaying' something is the wrong idea and it limits the imagination of what macros are used for. Think of 'general code transformation', often independent of execution in a compilation phase.