Hacker News Reader

Comment by DaniFong | Hacker News Reader

I don't claim that this is the only way to learn how to write compilers, it's just one surprisingly effective way.

And I actually think skill in generalised, simple, macro-type compilers is more useful, generally, than knowing the ins and outs of optimizing compilers, but that's just me.

For the sake of discussion, what are some of the important things for optimizing compilers that you can't do with this approach? I'm having trouble finding any -- both in a literal sense of possibility, and from a practical standpoint.

View on HN

I may be underestimating macros but I can't imagine any optimization technique that can be done with them when generating low-level output, be it native or virtual machine code. Can you demonstrate (just theoretically, of course) copy propagation, removing loop invariants, automatic inlining of functions, to name a few?

It's not hard to see how macros could do this. Macros just transform code into other code, which is what optimizations do too. For example, if you have an s-expr representing a loop, imagine a function which accepts that s-expr and returns a new one with invariants moved outside the loop body.

Ok, but my original point was, "you won't be able to write a compiler for anything other than a (suboptimal) Lisp derivative" (I was probably wrong about suboptimal though, because it only applies to the compiler)

Well, the easy way to use Lisp macros in a compiler is to start with an s-expr and successively transform it until you've got something that the target platform accepts. "Starting with an s-expr" implies that your source code is in Lisp format. So in that sense, what you're compiling is a Lisp derivative. Is that what you mean?

No doubt there are clever ways to leverage Lisp macros in compilers beyond this approach. But most people who like Lisp macros wouldn't bother. They'd just do it the easy way.

Edit: my understanding of what DaniFong is getting at is that in Lisp programming, there is no longer a barrier between application development and compiler development. This makes possible a lot of powerful things that you can't do when the application is written in a fixed language by different programmers than the ones who write the compiler. It's a different point, but one I find very interesting. It's not obvious what belongs to application development and what belongs to language development once this technical (and organizational) barrier is removed.

What about global optimizations? Macros operate on local expressions only effectively. Things like global alias analysis still needs to be done outside them.

Presumably you have the entire program represented as a top-level tree, and a global optimization would be a transformation of that. I don't see why macros wouldn't be just as applicable here as in the local case where you're transforming a subtree.

I guess u are right. Macros people usually write are local but macros are just AST transformers after all. I was confusing between what happens during macro expansion (arbitrary computation including lets say doing any analysis and building metadata) versus the result of the macro expansion (a transformed AST).

You're right that if you keep the implementations of Lisp compile time macros and use only that, you have as hard time reducing code that's already been macroexpanded, or applying global effects.

But when you might macros and the language itself, what you find is that you already have the pieces of what you need for a serious compiler: a symbol table, built in, a way to manipulate the parse tree, an easy way to do local expansions, and most importantly, a fully featured language.

You can do this with existing Lisps by a a few methods: making first class runtime macros, for example, or by saving the source code and working over it in passes.

Does this explain it?

Reply on news.ycombinator.com