Hacker News Reader

> The macros inevitably become so confusing, pervasive and complex that the author winds up inventing his own undocumented mess of a language, unusable for anyone else.

This is not limited to macros, but any abstraction.

Let's step back and rewrite the quote:

> The abstractions inevitably become so confusing, pervasive and complex that the author winds up inventing his own undocumented mess of a language, unusable for anyone else.

The key fault that trips up so many talented programmers (if not all) at least some point during their career (and hopefully not continuously, though I certainly know a few cases) is confusing convenience with simple. Convenience is almost always more complex, not simpler. Macros happen to wear their complex hearts on their sleeves.

You can address any of these problems by _addressing the problems_ you succinctly listed: documentation and usability.

Take React for example. It's an abstraction, arguably a new "language", and requires ample documentation and usability concessions to be palatable. You could accomplish this with macros as well.

But like any abstraction, the real problems arise during maintenance, and thus it becomes a Software Engineering problem -- whatever that means. This is where Software Engineering as a poorly-defined pseudo-profession has failed to demarcate itself from simple programming: the quality, maintenance and longevity of living code bases.

The masses unfortunately still don't see the forest for the trees, so I agree that it's something we've never seemed to learn, if perhaps using a broader definition.

So I'm not a fan of metaprogramming or heavy abuse of tricks for convenience either, and I do get and agree with your point!

But for some anecdotal nuance concerning the React example: > Take React for example. It's an abstraction, arguably a new "language", and requires ample documentation and usability concessions to be palatable. You could accomplish this with macros as well.

I feel like the standard before React was just every (web) framework inventing its own templating language, whereas I found React to be really easy to learn, because the way it abstracted things were straightforward and well explained. It's tooling however is one of the scariest things I've ever seen haha.

I recently wrote a React app without using the “create-react-app” tool… just by writing JavaScript code, later TypeScript, and bundling it with Rollup.js.

Doing things this way shows you that the tooling doesn’t have to be complicated. I assume that there are very good reasons why the tooling is complicated—stuff like code splitting, modular CSS, etc. If you’re writing pure JS React and putting the CSS elsewhere, you can squeak by with fairly simple tools.

Disagree. There are abstractions and there are abstractions.

Macros are too big of a footgun of an abstraction. Because they can change the syntax of the language, they can hide functionality in non-obvious ways and they're just another band-aid on top of the languages limitations (the two latter points are especially true for C). And more often than not they're worse than just writing using the languages capabilities.

Yes I'm sure people writing C macros will sit down and document them, and make them usable. Not.

That being said, Rust macros are a world apart from C macros.

> another band-aid on top of the languages limitations

Those limitations are real though, and getting in peoples way. I'd also add most language designers would not consider C macros as "real" macros.

Think of abstractions as power tools, you want to use the least powerful one that solves the problem so you don't lose a hand. If you're cutting a string you pull out some scissors, if you're cutting a 10 foot thick steel rod, you grab some hydraulic shears; so it goes with abstractions.

Most people never need macros, the abstraction equivalent of hydraulic shears, because most of the time your problem isn't that difficult. That doesn't mean those problems aren't out there or that macros shouldn't exist.

There are macros, and there are macros.

You've used C and Rust macros. Have you seen Common Lisp macros?

I've read a fair amount of Common Lisp code written by other people, and every time I had difficulty understanding something, it was invariably plain CL, and not a macro.

Maybe C programmers just have difficulty exercising restraint when writing their macros. Or perhaps the design of the language and the macro system (which is basically just text pre-processing) don't mesh well together.

Regardless, I haven't noticed macros being a footgun in CL. Sure, there are doubtless people out there who have abused it (and will continue to do so) - but that's true for every language feature; meanwhile, it doesn't seem like macros themselves are actually a problem, in the sense that they're particularly easy to abuse compared to the rest of the language (a distinction that I might give to C raw pointers, for instance).

If anything, (good - CL) macros are easier to understand than many other features, because they're (usually) referentially transparent functions with a clearly defined input and output that runs once at compile-time, on known data, which you can manually expand and inspect, and they tend to rarely interact with other features of the language. With rare (but useful) exceptions, macros don't involve networking, async, threads, dynamic environments, databases, stack limits, or any of the other strands in the spider-web complexity that tends to make programming hard.

In other words, they're a completely different language feature than the one in C that happens to go by the same name.

Lisp macros can get super obtuse, no matter what people tell me. When I have used Common Lisp, I try to avoid them most of the time.

I addressed this in my comment:

> Regardless, I haven't noticed macros being a footgun in CL. Sure, there are doubtless people out there who have abused it (and will continue to do so) - but that's true for every language feature; meanwhile, it doesn't seem like macros themselves are actually a problem, in the sense that they're particularly easy to abuse compared to the rest of the language (a distinction that I might give to C raw pointers, for instance).

Nobody contests that CL macros can get obtuse - but that's not special. You can write obtuse code in any Turing-complete language. Does that mean that we should abolish Turing-completeness? Of course not.

How much Common Lisp have you read and written?

At one point, I decided to unwind all my use of macros in C. I was actually very happy with the result. My code was much better.

> This is not limited to macros, but any abstraction.

Disagree here. Unix process, file access api's, programming languages are all abstractions that make life easier.

Thanks for taking the time to reply Walter, genuinely appreciate it!

  > "Unfortunately, the dahk side of macros inevitably consumes the user. The macros inevitably become so confusing, pervasive and complex that the author winds up inventing his own undocumented mess of a language, unusable for anyone else."

Yes! Modern languages have all adopted this approach it seems, trading macros for proper compile-time expressions. Rust uses "#[feature]" I believe, pretty sure Zig and Nim has something for this as well.

I am very sparing even when using mixins/template mixins in D -- the second you do, it ratchets the debugging difficulty up to "11" and becomes am order of magnitude harder to reason about program state.

Great power, great responsibility. I'm an "avoid at all costs" kind of guy, unless you can really justify it.

If C++ didn't have macros (which invariably get abused, along with templates) I think I might not hate it so much. Let us pray that the "constexpr" idea they "borrowed" from you may slowly fix this a little over time ;)

I went through a phase earlier in my dev career where I thought "Metaprogramming is so powerful and cool!", and I've put that behind me now. I just want boring, easy-to-read + maintain, predictable code.

Ironically I used to poke fun at Java and .NET but I've come to appreciate the "stupidity". "Dumb" is easy to understand, "dumb" is easy to maintain. "Clever" is bad.

  > "My friend said "I can fix that" and fixed the bug and checked it in in two hours. The manager, astonished, asked him how he figured it out. He said "I didn't figure it out. I disassembled the object code using Walter's disassembler that turns it into assembler source code, found the bug, fixed it, and checked in the new asm source code!""

Ha!

I suppose while I'm at it, I ought to thank you for all the Digital Mars tools that myself and many others still use today:

https://www.digitalmars.com/ctg/ctg.html

The Digital Mars tools for working with PE/COFF files and libs/binaries are indispensable. I suppose they've helped many tens of thousands of people.

They're certainly indispensable to me! Some of them I wrote for the purpose of understanding the file format. Writing a pretty-printer is a great way to understand.

> One is the use of macros in programming languages.

Macros killed the Lisp. They never noticed.

I'm not experienced with Lisp, but I've heard that as well. A project will essentially invent its own language with the macros, and once the original developers leave nobody else wants to learn those macros, they want to use their macros. And so the project dies.

That's a myth. Actually there are lots of project with domain specific languages and in the case of Lisp they are often directly embedded. Having OO- or functional interfaces to learn is not much different from learning macros.

USING the macros is not a problem. MAINTAINING them might be a problem, because new people might have a hard time understanding to work with a language which has meta-level programming directly integrated. There are more languages which now are adding some kind of macro facility (example Rust), so there might be now more programmers getting familiar with meta-level programming with macros.

Projects like planners for public transport use special language extensions and if they have a market, they go for a long time. Other example, a specialized CAD system will have its own language extension for expressing generative CAD operations.

Sometimes there are projects which are replaced with different technology. For example a web store generator. But there the domain technology typically changes that some of these sites have three technology transitions in a few years. Every project owner has new tool preferences, influenced by the market. It might look like that old code is not understood, but the problem is actually that the code is no longer fashionable. Writing enterprise applications is largely done by Java and not C++. The reason to replace an enterprise application with one written in a newer language (like Java) has many reasons: technology no longer seen as fashionable, fewer trained developers available, trained developers to expensive, age bias, new libraries not available for the older language, etc.

I am experienced with Lisp, and this is a case of "Short answer yes with an if; long answer no with a but."

Plenty of junior Lisp developers are so enamored of macros that they over-use them. Particularly since macros in Lisp are so both easy to write, and mentioned as a feature unique to lisp, you see junior developers reaching for that as a their first tool. This leads to there being lots of projects as you describe, particularly projects written by a lone Lisp journeyman developer.

Most Lisp style guides say things like "Don't use a macro when a function will do." Most projects written by more experienced developers (particularly those who have experience working on a team) tend not to end up in this state.

> ...author winds up inventing his own undocumented mess of a language, unusable for anyone else.

For projects in which an embedded language makes sense, documentation is important! Also, again, this is a situation that tends to be peculiar to projects written by lone developers because it's very hard to get away with writing a tool that is unusable by other people if you are surrounded by other people trying to use it.

[edit]

This is also partly an artifact of the post AI-winter community being so small and fragmented. When smalltalk style OO was the next big thing, there were dozens of implementations for it in Lisp almost overnight. Reasonably quickly this whittled down to about 2 or 3 popular ones, some time after which people got together and standardized CLOS from the what was left. Arguably the Lisp community is insufficiently cohesive (and insufficiently funded) these days to maintain something like that that now.

> My friend said "I can fix that" and fixed the bug and checked it in in two hours. The manager, astonished, asked him how he figured it out. He said "I didn't figure it out. I disassembled the object code using Walter's disassembler that turns it into assembler source code, found the bug, fixed it, and checked in the new asm source code!"

Nice! But this trick doesn't work when a bad program is written in a HLL! I once speeded up a C++ program that took 3 minutes & was in the critical path for 40-50 engineers. It was badly written and the logic was spread all over the place. It was not fixable so I took the easy way out. I looked at its inputs+outputs, fed various smaller samples to figure out the IO logic and reimplemented it from scratch. The new program ran in 0.5 seconds. It wasn't a difficult problem but nobody wanted to touch the program because it was so badly written.

I think the moral is that bad programmers will misuse whatever features they have access to. The "dark side" is less that of the feature in question and more in the lack of early intervention and proper training! People need to be taught how to use power tools responsibly & judiciously. If you take away all the power tools for the fear of misuse, you will only see mediocre work. Or worse.

> People need to be taught how to use power tools responsibly & judiciously.

I would have agreed with you long ago. But decades of experience with it shows that:

1. Everyone agrees with that. And then they create a mess. They are generally unable to see the mess. One cannot learn without experiencing failure, and when one cannot see the failure, one cannot learn.

2. Some features are far more susceptible to this than others. Macros are pretty much at the top of that list. Other abominations are nutberger operator overloads, like overloading << to mean input (or is it output? I can never remember, and don't care, either).

There are many things in D meant to block or discourage these kinds of choices. I regularly get pushback on them. All I can say is "I understand exactly where you're coming from, and why you believe I'm wrong. But you haven't been living with this for that long, and will just have to trust me."

3. These also come about from my discussions with managers of programming teams and the problems they have to deal with.

> 1. Everyone agrees with that. And then they create a mess. They are generally unable to see the mess. One cannot learn without experiencing failure, and when one cannot see the failure, one cannot learn.

Hence "early intervention"! Usually one's own experience is not going to be enough but if one can learn from real experts what to avoid and what works well, one can develop a sense. You can get some of that by careful reading of such people's code.

> 2. Some features are far more susceptible to this than others.

True. This happens when the language developer creates a mess! I am reminded of Tony Hoare's "Hints on Programming Language Design".

> 3. These also come about from my discussions with managers of programming teams and the problems they have to deal with.

They should be listened to for the problems they face but not necessarily solutions :-)

Haha! Yes, every time I hear Lisp and Macros... really, you have to be careful with them. I would say it should be the last resort, and really careful when using them.