[1] https://en.wikipedia.org/wiki/Macro_(computer_science)#Synta...
I think Rust did two things right in that regard: firstly the macro invocations are always suffixed with '!' so you know it's not a regular function call right away. Secondly Rust macros are so quirky, ugly and painful to implement that you only ever use them as a last resort, so people tend not to abuse them too much.
And that, ladies and gentlemen, is called a framework.
Macros or no macros, if the problem domain is large and you stay on it long time, you end up with framework. Macros can make that framework easier, not harder.
In most languages you end up with large configuration files and huge number of strings as parameter. The program semantics might be defined by interpreting JSON structures, template languages and ungodly amount of string parameters that are in effect reserved words. If you persist long enough, you have code generators and state machines and define execution semantics with petri nets.
What you want to do us use the programming language that is best suited to solving the problems you have. Growing the language towards the problem you have is great opportunity to make the programming interface simpler to understand.
With Lisp, especially with Common Lisp it's possible to grow the language so that it stays familiar as much as possible.
>In most languages you end up with large configuration files and huge number of strings as parameter.
Not in the languages I'm familiar with at least. I don't understand what configuration files and macros have to do with each other.
>Growing the language towards the problem you have is great opportunity to make the programming interface simpler to understand.
Simpler to use, not to understand. You know what it does, you don't know how it does it or how it can be extended because it doesn't play by the language's usual rules. It's nice if you're copy/pasting stack overflow snippets or your use case is very standard, it sucks if you want to modify the code to do something a bit different and out of the box because now you have to learn the rules of this custom DSL.
>With Lisp, especially with Common Lisp it's possible to grow the language so that it stays familiar as much as possible.
So adding custom constructs to the language makes it stay familiar? That doesn't make a lot of sense to me
There are rules and conventions for writing macros. If you follow them, their usage will be clear for anyone who knows the language - because the language itself includes a lot of macros and they all follow the same set of rules. Common Lisp, Clojure and Elixir are very good examples of this (with Racket taking it up to 11).
Of course, you can write macros which work and behave in unexpected, weird ways. In practice, though, you don't - why the heck would you?
... which often includes yourself a year later.
> It's nice if you're copy/pasting stack overflow snippets or your use case is very standard, it sucks if you want to modify the code to do something a bit different and out of the box because now you have to learn the rules of this custom DSL.
This basically describes Spring, and I'm not just being snarky. To someone from the outside, Spring's annotations seem to have all the problems you list. I have trouble imagining that heavy use of macros could be much worse, and I can certainly see the overlap between macros and frameworks.
Seen in this light, the kind of DSL you can create with a full-blown macro system isn't all that different.
Programming, in large part, is an exercise in language creation.
Emacs-lisp is one example of a Lisp that has evolved into the domain of text editing.
It's different. Macros can change the language grammar (it's like inventing a new way to structure a sentence in English, instead of the usual object-verb-complement). Libraries just provide new vocabularies (new names, new verbs, etc.).
You can use the English language to express ideas on very different topics (food, law, engineering, romance, etc.) without changing its grammar.
But that's the point. When you can't change the grammar, you instead end up writing in a different language that doesn't even have a grammar, and encoding it in the base language plus a bunch of horrible configuration.
Look at Spring for an example.
This is bad use of macros, or an ugly macro system.
Macros, at least in Lisp, made code even clearer to understand; because they let you create the constructs that make the problem domain map more directly, straightforwardly, easily, to the programming language.
So they reduce line count, they reduce need for kludges or workarounds. They allow more straight code.
But this is within the Land of Lisp, where writing a macro isn't something "advanced" nor "complex" nor "esoteric". In the Lisp world, writing a macro is 95% similar to writing a run-of-the-mill function.
But really, this is a recognized problem of Lisp, and has been called the Lisp Curse. [0] One is never programming in "just Lisp", but rather in Lisp plus some half-baked DSL haphazardly created by whoever wrote the program in the first place.
Also, don't confuse readability with understanding. Yes, DSLs are typically easier to read, but only after you come to understand the primitives of the language. When every program has its own DSL with its own primitives, even programs that do similar things... That becomes quite a burden.
[0] http://www.winestockwebdesign.com/Essays/Lisp_Curse.html
This is also true of functions. Without reading the body, you don't know if it's just going to return the sum of the two integers you passed to it, or if it's going to change some global variable, launch the missiles, and then return a random int.
Yes, macros are more powerful, and therefore you need to be more careful with them. But they are still much better than what ends up being used instead. With languages that don't have macros, you end up with complex frameworks that use runtime reflection, or code generators that run as part of the build system (which end up being an ad-hoc, messy macro system).
Or, some horrible solution where you embed a DSL by interpreting trees of objects, which effectively represent an AST. In this case, the embedded language doesn't follow the language rules, but it seems like it does, because you're looking at the implementation of the interpreter, instead of at the syntax of the embedded language.
> but rather in Lisp plus some half-baked DSL
Why does it need to be "half-baked"? Why do you assume that writing a good DSL is impossible for most Lisp users? Are you sure it's actually the case?
Note well: This is my understanding of the claim. I take no position on whether it is true.
Writing and maintaining a good DSL is like writing and maintaining bug free code. You always start with the best of intentions, but human fallibility and entropy are always pulling you in the wrong direction.
This doesn't mean that the attempt is not worthwhile. But it does mean that you should expect eventual failure.
Also, yes I would propose that a good DSL would be difficult to write for most programmers of any type. Not because of any inherit deficiency in the programmer, but rather because we tend not to spend enough time in a single domain to understand it well enough to write a good DSL.
Quoting user quotemstr here:
"Every program is a DSL. What do you think you're doing when you define types and functions except make a little DSL of your own? (...) Programming, in large part, is an exercise in language creation."
Most lisp docs will tell you to use macros only when necessary, because as great as they are, they have inherent issues that aren't fixed just by having a good macro system.
This is bad use of macros, or an ugly macro system.
No true scotsman?
One also declares variables when necessary, and one also creates arrays when necessary, etc. But imagine a programming language that doesn't support arrays. It would be a nightmare if you need to do certain scientific computations.
So in the same way, yes, not having a (proper, Lisp-like) macro system surely hurts a lot, once you realize how it makes certain problems become really easy.
And, by the way, one should write macros when necessary. In Lisp, we're using macros most of the time!
>Isn't loop essentially a macro (probably special op) that a lot of people hate?
And other Lisp programmers like the LOOP macro, since it can allow for very readable and concise code to do something simple that should stay simple to read.
Examples:
(loop for x from 1
for y = (* x 10)
while (< y 100)
do (print (* x 5))
collect y)
5
10
15
20
25
30
35
40
45
(10 20 30 40 50 60 70 80 90)
(loop for x in '(a b c d e)
for y in '(1 2 3 4 5)
collect (list x y) )
((A 1) (B 2) (C 3) (D 4) (E 5))
Source of examples: http://cl-cookbook.sourceforge.net/loop.html
Today there is a related discussion in comp.lang.lisp: https://groups.google.com/d/msg/comp.lang.lisp/U5LqiM5nKq8/M...
For the first: There's an implicit x->x+1 hiding in there? Why 'for y' if y is derived from x and not incremented on its own?
For the second: It looks like a nested (cross product) loop, but it's actually a zip (dot product) loop?
loop doesn't do cross-producting; if you know that, there is no mistaking it. All the clauses specify iteration elements for one single loop.
"for x from 1" means starting at 1, in increments of 1.
"for y = expr" means that expr is evaluated on each iteration, and y takes on that value.
y could be incremented on its own, but then that example wouldn't show the "for var = expr" syntax, how one variable can depend on a combination of others.
This is what a LOOP macro looks like in code:
(loop for i from 0 below n
for v across vec
when (evenp n)
collect (* i v))
(loop ((for i :from 0 :below n)
(for v :across vec))
(when (evenp n)
(collect (* i v))))
The LOOP macro is historically coming from Interlisp (70s), where it was a part of a certain language design trend called 'conversational programming'. The idea was to have some more natural language like programming constructs, combined with tools like spell checkers and automatic syntax repair (Do What I Mean, DWIM). From there this idea and the FOR macro was influencing the LOOP macro for Maclisp. The LOOP macro grew over time in capabilities and was then transferred to later Lisp dialects, like Common Lisp.
There are actually Lisp macros which are even more complicated to implement and even more powerful, but which create less resistance, since they are a bit better integrated in the usual Lisp language syntax. An example is the ITERATE macro: https://common-lisp.net/project/iterate/
Thus it is not the complexity or the functionality of the macro itself, but a particular style of macro and its implementation. I personally also prefer something like ITERATE, but LOOP works fine for me, too.
The advantage of something like ITERATE or even LOOP is that they are mostly on the developer level, and not fully the implementor level. A developer or group of developers can develop such a complex macro and can integrate it seamlessly into the language, making the language more powerful and allows us to reuse much of the knowledge/infrastrucure about/of the language.
Implementing and designing something like ITERATE and LOOP requires above the usual dev capabilities. Generally macros require some of that, since it makes it necessary that the dev can use or program on the syntactic meta-level. It's there where language constructs are reused, implemented and integrated.
Lisp docs will tell you to use macros only when necessary? They tell you to WRITE macros only when necessary. Since many Lisp dialects have a lot macros, you have to use them anyway. Most of the top-level definition constructs are already macros. If we use DEFUN to define a function, we already use macros.
In my experience actual Common Lisp uses a lot of macros. I also tend to write a fair number of macros.
But generally good macro programming style is slightly underdocumented, especially when we think of various qualities: robustness, good syntax, usefulness, readability, avoiding the obvious macro bugs, ...
On Lisp from Paul Graham is useful, as a few other sources, ... Read On Lisp here: http://www.paulgraham.com/onlisp.html
Macros are very useful and I use them a lot, but at the same time one needs to put a bit more care/discipline into them and some help of the development environment is useful...
Macros are a tool for creating abstractions. In the mind of the author, abstractions are always clearer. Others who have to work with those abstractions legitimately may or may not agree.
In the case of authors who think that abstractions are an unmitigated good, I seldom agree with their abstractions. And I don't care whether they are implemented as deep object hierarchies, macros, or functions with a lot of black magic. If you are unaware of the cognitive load for others that are inherent in your abstractions, then you are unlikely to find a good tradeoff between conciseness and how much of your mental state the maintainer has to understand to follow along.
>But this is within the Land of Lisp, where writing a macro isn't something "advanced" nor "complex" nor "esoteric". In the Lisp world, writing a macro is 95% similar to writing a run-of-the-mill function.
I agree with you but I think there are two aspects to code readability: A/ is it clear what the code does (the intent) and B/ is it clear how it does it (the implementation).
I think macros can help massively with A (hiding redundant code and clunky constructs) at the cost of obfuscating B. The thing is that if you want to hack into the code at some point you'll need to understand B too.
To take a very simplistic example imagine that you're reading some CL code and see something like:
(let ((var 42))
(magicbox var (format t "side effect~%"))
(format t "~a~%" var))
So, without running the code or looking at what "magicbox" does, if you assume that it's a function call, then you can expect that this code will print "side effect" when magicbox is called (since the parameter will be evaluated before the call) and then whenever it returns the 2nd format will display "42" since var is not modified between the let and there.
You run your code and you see that it only displays "magic" instead. Uh, What happened?
Well there you're lucky because there's only a single statement to consider, the magicbox invocation. You ask emacs to find the definition and you see:
(defmacro magicbox (var unused)
(list 'setq var "magic"))
Not really. Since the call explicitly names var you could easily narrow it down to that by going through all the places in the code where var is mentioned. If all the other expressions concerning var are non-destructive, you have your smoking gun: it must be magicbox. If var is mutated all over the place in that code then you have to work out whether it is magicbox or something else.
Give a mad man enough rope and he can find a way to hang himself.
Where do you draw the line? At least with functions, there's only one syntactic structure that is user-definable, making it easy to distinguish foreground from background.
I don't understand what you mean by that. When I call a function in Common Lisp I have some garantees, I know it won't change the value of the parameters that I pass by value, I know it won't add or remove variables from the current scope, I know that the parameters to the function will be evaluated before the invocation. For macros anything goes, for all you know it could expand to a RETURN-FROM and exit your function prematuraly, it could implicitly modify a local variable etc...
> Operating overloading is another. They can all be used well or confusingly.
I completely agree and I consider that operator overloading for anything that's not "semantically" equivalent to the builtin operator behaviour is inherently evil. It makes sense to overload "*" and "+" on a matrix or vector type, it doesn't make a lot of sense to overload "<<" or ">>" to deal with input/output. But obviously nobody would be insane enough to do that.
When I read "a = b + c" in some C++ code I fully expect that "a" will be set to the value of a sum of "b" and "c", whatever makes sense for the types of "b" and "c". If adding "b" and "c" doesn't make any obvious sense semantically (adding database handles with strings) then it shouldn't be done.
I think that it's easier to draw the line and design sane guidelines for when you're allowed to implement overloaded operators. Macros have much broader use cases.
I meant that a whole bunch of specialzed functions can also be a dialect, which are opaque unless one takes the trouble to look into them. You can write very confusing code, unless it matches up closely to a domain (and even then, you need to be on top of the domain). Of course, the "unraveling" is easier, (and less literal) than for macros.
I was just trying to say it's a difference in degree not in kind - if you measure in terms of power and usability.
I take your point that it's not just syntax, but differences in (basic) guarantees, so there's less ability to treat it as a black box (information hiding within a module). But any module (e.g. a fn) can behave differently from how you expect.
tl;dr macros vs fn is a difference in degree, in terms of outcome (i.e. confusion).
The space of things that a function won't do is tiny compared to the space of what it may. You've taken a vast universe of possible behavior and eliminated from it a few possibilities like mutation of a local variable in the caller; that still leaves a vast space. Generally, to maintain code, at some point you have to grapple with understanding what it does more than with what it doesn't do.
No. When you define a function or a class, you define a new vocabulary (a new verb, a new name), but you don't change the grammar. Macros are a different beast because they can change the grammar.
If there's a sin, it's maybe that macros make it a bit too easy to (prematurely) optimize the conciseness of your code, and that becomes an enticement to play code golf.
This exact same thing happens with functions, too. At some point, a third party will have to read the code to understand it. You don't know what a function does until you read its implementation either.
Often, having a domain specific language (even if the domain is very specific to the application) will make the application easier to understand.
If you write code in say, Go, then sure, a third party will be able to look at a loop and be able to figure out the mechanics of what it does. But that person will still not understand how the application works. They will still need to learn both the domain and the structure of the application. And with the right macros, the application will likely be smaller and better organized.
Avoiding abstraction doesn't magically make it easier to understand the solution to a complex problem.
In any case, I would like to point out that there are other answers to the problem than macros. For example, Haskell has lazy evaluation, currying and strong type system, and many use cases for macros can be implemented just as ordinary functions.
Between debugging C++ meta-template programming and Rust macros, I am undecided what to choose as best.
(This is a joke, but it's also a serious point; people keep thinking of Lisp as the "king over the water" that will one day return and save them, which keeps people from looking at the reasons why Lisp never achieved mass adoption. Or arguing that Lisp cannot fail, it can only be failed.)
Edit: I think you're downvotes are bit harsh and as a Scot I appreciate the "king over the water" reference - people get all romantic about the Jacobites without thinking too much about what their policies might have been.
The recurring cost of maintaining and deploying a working Lisp system is perceived by you to be higher than the benefits the language begets?
I agree that lisp has issues marketing itself.
Which has some previous discussion about it:
https://news.ycombinator.com/item?id=11558007
I would imagine Martin has gone through the path of discussing macros long time ago, and for some reason or other does not refer to them here.
JS fatige is caused by a culture that creates packages for single functions.
If the language ast and syntax aren't iconomorphic then you add a learning curve and need to insulate users from compiler internal changesy quasy quoting doesn't work everywhere.
In a compiled languange first class macros also add significant compile time.
A couple languages have them like nim's macros , haskell's TemplateHaskell extension and Rusts compiler plugins, though.
Do you mean "homoiconic"?