Code generation isn't well defined other than it's a calculation in which the end product is a certain language. The input could be any format and the transformation could be anything.

Macro processing is one way to constrain and refine the concept of code generation, in a particular direction.

Arbitrary code generating programs have the problem that they don't play along. Alice has a code generator and so does Bob. Both of these generate the same language. Alice and Bob want to work on the same files of the same project, each using their code generator. The only way it can work is if their code generators recognize only certain delineated syntax and pass everything else through, so that Alice's code generator can be applied first to code that also needs Bob's generator or vice versa. Suppose Bob uses his generator in such a way that Alice syntax comes out of a construct. If the Alice generator has run first, that won't work. Running both generators repeatedly, until a fixed point is reached, might work. This will likely not scale nicely beyond a small number of code generators.

If Alice's and Bob's code generation are only doing text substitution, it would be a lot better if Alice and Bob used a common textual preprocessor and wrote their respective parts as macros. Then their work integrates and is expanded by a single application of the tool, and any number of team members can write macros independently.

It depends on the preprocesor.

Recently, I converted a C-preprocessed file into code generation.

Here I was merging similar code with a macro:

https://www.kylheku.com/cgit/txr/commit/?id=eb576809489a7b1a...

Then, short time later, I retract my decision and it gets generated:

https://www.kylheku.com/cgit/txr/commit/?id=feb8eefb2fc97e5f...

C macros don't have good tooling. They are not supported well under debugging, and even basic text navigation tools don't work well with them. Vim with tags will jump to the macro definition, but not to the definition of a function defined with a macro.

No. I'm not gonna sit there writing another tool to augment ctags so that the editor will know that sha256_init is written by the chksum_impl(sha256, SHA256_t, "SHA-256", SHA256_DIGEST_LENGTH, HA256_init, SHA256_update, SHA256_final); macro call, and jump to that line.

So the switch isn't motivated by a philosophical position on macros versus code gen, but by the concrete specifics of the state of the tooling, and the nature of the desired transformation. (E.g. are we generating large boilerplate spanning multiple functions? Or are we making syntactic sugar for walking over a list?)

This is a Lisp project; I'm not macro ignorant.

Other Lisp people have used code generation at the implementation level. The CLISP project dates back to the late 1980's, long before there was C99. Many of the C sources have .d suffixes, and are preprocessed by a script called "varbrace", which turns mixed declarations and statements into C90: statements before declarations (by emitting brace-enclosed blocks). E.g. puts("hello"); int foo = 42; becomes puts("hello"); { int foo = 42; ... }. There is no way you could do that with C macros, in any way that would be remotely reasonable, and not stray so far from the objective as to be comical.

Code generation is never off the table when we deal with languages that don't have good macros. But new language should be making provisions so that users don't have to resort to it. When language users resort to code gen, that's a sign that the language has failed in some way.

E.g. C failed for Bjarne Stroustrup by not providing support for OOP, or good enough macros to do it nicely, so he wrote a "C with Classes" code generator. Not everyone agrees; plenty still use "C without Classes" forty years later.