Hacker News Reader

Comment by JimmyRuska | Hacker News Reader

Macros feel magical, this idea you can make your own language in no time so long as it's s-expression based, and make your own personalized vocabularies, expression flows, fit to purpose. Personal computing at its best!

But from a working-with-other people perspective, imagine opening some code and finding random invented constructs you have to reverse engineer and step through. These days it seems everyone creates yaml or json files, and then implements some kind of declarative engine or interpreter to implement their DSL. This can be both easier to read and debug. For more complicated use cases you can tie it to a graph database that supports reasoning, instead of json/yaml

Even so, is there really something that can be done with macros that can't be done, just as elegantly, with simple state machines, or object composition style in other languages?

View on HN

> But from a working-with-other people perspective, imagine opening some code and finding random invented constructs you have to reverse engineer and step through

This feels just like working with another API to me. Macros can be arbitrarily complicated I suppose, but is the typical usage any more complex than learning an API? I don't really know, I guess, but my experience hasn't shown this to be that challenging.

I can appreciate that programming languages are ultimately codifying a particular way to think about computation, and macros express more idiosyncratic ways of thinkingabout comouting that may not be readily understandable.

I do think Racket has issues, but this isn't the first one I would cite.

> finding random invented constructs you have to reverse engineer and step through

Like functions and procedures?

I would pray that a good number of them are compile-time code transformers that execute in the comfortable development environment, so I don't have to reverse engineer and step through them on the embedded target.

I'm very curious what you think macros are that they'd be analogous to "simple state machines".

Macro expansion occurs at, well, macro expansion time which is prior to compilation (in a compiled lisp, as is the case in the context of Racket). The result of the expansion is what gets compiled. You could certainly express a state machine using macros (or with the aid of macros), perhaps even offering macro/compile time optimizations (hypothetically, you could take a regex in some form and convert it through a macro into straight code removing some runtime overhead). But macros are much, much more than "simple state machines". And I'm not sure what you mean about "object composition style", unless you're talking about the OO sense of preferring composition over inheritance. Which, again, what does that have to do with macros?

You can use macros to personalize your language, but for the slight overhead of run time, it's often better to use state machines or object composition style to recreate that same vocabulary.

Eg somekind of cron-like event library, made from a macro

    (wait 6 'hr' (run './myapp'))

In some OOP you could create an object style expression flow that's similar

    Scheduler.create(6,'hr').then(() -> system('./myapp'))

Or state machine

    scheduler(6, 'hr', steps= [() => { system(./myapp) }, ...]);

Then you customize the state machine steps with whatever, monad-like modifications are needed for that expression flow.

In the end macros save you a tiny bit of typing at the expensive of limiting static analysis tooling, limiting readability since the language can be modified, and most the time the runtime way of doing it 99.9999% as fast, easier to understand, easier to debug by stepping through it. That being said I like using macros, especially in emacs. No one will ever read my dot files, and I can live in my own world.

That should be a function:

  (wait 6 :hr (lambda () (run "./myapp")))

a macro to hide the lambda doesn't buy you much, and could be done by a macro that condenses lambda syntax:

  (wait 6 :hr (op run "./myapp"))

Wait method could expand into some kind of systemd library api calls which are more complicated. That's the thing about macros, 99% of the time they could just be functions, and when they're control flow macros that specialize away some runtime checks, you rarely save more than a few nanoseconds.

If 99% of the time macros could be functions, you're reading a Lisp codebase whose authors don't follow good practices like in Google's Common Lisp Style Guide (by Robert Brown, François-René Rideau).

https://google.github.io/styleguide/lispguide.xml?showone=Ma...

"You must never use a macro where a function will do. That is, if the semantics of what you are writing conforms to the semantics of a function, then you must write it as a function rather than a macro."

Can you give me an example of good macros in your opinion? It feels like any macro could be replaced with functions that does the same thing, albeit with a bit or runtime checks, or without being as terse as the macro.

How about defun? I don't want to program in Common Lisp like this:

  (eval-when (:load-toplevel :execute) ;; don't evaluate functions in compiler
    (setf (symbol-function 'add)
          (lambda (x y)
            (block add (+ x y))))

when I can program like this:

  (defun add (x y) (+ x y))

The nice thing is that I could remove the eval-when dance and have functions defined in the compiler by default, and the block could be omitted when I obviously don't need it. The remaining syntax is still ugly; I'd want my-defun.

Usually people whip out the LOOP macro, so let's skip that.

How about pattern matching (TXR Lisp). A simple expression like:

  (if-match (:pre :amble (@x) @(oddp @y)) obj
    (list x y))

churns out quite a mouthful:

  5> (macroexpand-1 '(if-match (:pre :amble (@x) @(oddp @y)) obj
                       (list x y)))
  (alet ((#:g0051 obj))
    (let* (#:result-0064
           x y)
      (if (if (consp #:g0051)
            (alet ((#:g0052 (car #:g0051))
                   (#:g0053 (cdr #:g0051)))
              (if (equal #:g0052 ':pre)
                (progn (if (consp #:g0053)
                         (alet ((#:g0054 (car #:g0053))
                                (#:g0055 (cdr #:g0053)))
                           (if (equal #:g0054 ':amble)
                             (progn (if (consp #:g0055)
                                      (alet ((#:g0056 (car #:g0055))
                                             (#:g0057 (cdr #:g0055)))
                                        (if (consp #:g0056)
                                          (alet ((#:g0058 (car #:g0056))
                                                 (#:g0059 (cdr #:g0056)))
                                            (progn (set x #:g0058)
                                              (if (equal #:g0059 '())
                                                (progn (if (consp #:g0057)
                                                         (alet ((#:g0060 (car #:g0057))
                                                                (#:g0061 (cdr #:g0057)))
                                                           (progn (set y #:g0060)
                                                             (alet ((#:res-0063
                                                                     (alet ((y #:g0060))
                                                                       (oddp y))))
                                                               (if #:res-0063
                                                                 (if (equal #:g0061 '())
                                                                   (progn (progn (set #:result-0064
                                                                                   (list x y))
                                                                            t)))))))))))))))))))))))
        #:result-0064

I wouldn't write it in that mechanical way by hand; but writing it in a nice way would take thought.

The optimizer does a good job of reducing that kind of code, and the code walks through the list only once, using temporaries for successive cdr values.

The way I might write it might look terse, but involve redundant accesses to the list from the start.

Common Lisp's simple destructuring pattern operator destructuring-bind is also a macro.

  [1]> (macroexpand '(destructuring-bind (a (b c (d))) list (list a b c d)))
  (LET ((SYSTEM::<DESTRUCTURING-FORM> LIST))
   (IF (NOT (SYSTEM::LIST-LENGTH-IN-BOUNDS-P SYSTEM::<DESTRUCTURING-FORM> 2 2 NIL))
    (SYSTEM::DESTRUCTURING-ERROR SYSTEM::<DESTRUCTURING-FORM> '(2 . 2))
    (LET*
     ((A (CAR SYSTEM::<DESTRUCTURING-FORM>)) (#:G3318 (CADR SYSTEM::<DESTRUCTURING-FORM>))
      (#:G3319
       (IF (NOT (SYSTEM::LIST-LENGTH-IN-BOUNDS-P #:G3318 3 3 NIL))
        (SYSTEM::ERROR-OF-TYPE 'SOURCE-PROGRAM-ERROR :FORM SYSTEM::<DESTRUCTURING-FORM> :DETAIL #:G3318
         (SYSTEM::TEXT "~S: ~S does not match lambda list element ~:S") 'DESTRUCTURING-BIND #:G3318 '(B C (D)))
        #:G3318))
      (B (CAR #:G3319)) (C (CADR #:G3319)) (#:G3320 (CADDR #:G3319))
      (#:G3321
       (IF (NOT (SYSTEM::LIST-LENGTH-IN-BOUNDS-P #:G3320 1 1 NIL))
        (SYSTEM::ERROR-OF-TYPE 'SOURCE-PROGRAM-ERROR :FORM #:G3319 :DETAIL #:G3320
         (SYSTEM::TEXT "~S: ~S does not match lambda list element ~:S") 'DESTRUCTURING-BIND #:G3320 '(D))
        #:G3320))
      (D (CAR #:G3321)))
     (LIST A B C D)))) ;
  T

This example is from CLISP. Ut uses a lot of CADDR from the root of the list. Still a mouthful that is not nice to write by yourself, even in the nicest, tersest possible way.

Defining object, with complex bells and whistles (TXR Lisp):

  2> (macroexpand-1 '(defstruct myclass (these are super classes)
                       (:static static-slot 42)
                       uninitialized-slot
                       (initialized-slot 73)
                       (:method meth (me : optional-arg) (list em optional-arg))
                       (:postinit (me))
                       (:fini (me) (put-string `finalizer invoked on @me`))))
  ** expr-1:1: warning: defstruct: inheritance base these does not name a struct type
  ** expr-1:1: warning: defstruct: inheritance base are does not name a struct type
  ** expr-1:1: warning: defstruct: inheritance base super does not name a struct type
  ** expr-1:1: warning: defstruct: inheritance base classes does not name a struct type
  (sys:make-struct-type 'myclass '(these are super
                                   classes)
                        '(static-slot meth)
                        '(uninitialized-slot
                          initialized-slot)
                        (lambda (#:g0019)
                          (when (static-slot-p #:g0019 'meth)
                            (static-slot-set #:g0019 'meth
                                             (sys:meth-lambda
                                               meth myclass
                                               (me : optional-arg)
                                               (block meth (list em optional-arg)))))
                          (when (static-slot-p #:g0019 'static-slot)
                            (static-slot-set #:g0019 'static-slot
                                             42)))
                        (lambda (#:g0019)
                          (finalize #:g0019 (sys:meth-lambda
                                              myclass :fini
                                              (#:g0019) (let ((me #:g0019))
                                                          (put-string `finalizer invoked on @me`)))
                                    t)
                          (let ((#:g0020 (struct-type #:g0019)))
                            (unless (static-slot-p #:g0020 'uninitialized-slot)
                              (slotset #:g0019 'uninitialized-slot
                                       ()))
                            (unless (static-slot-p #:g0020 'initialized-slot)
                              (slotset #:g0019 'initialized-slot
                                       73))))
                        () ())

To write the function call, you have to know the API, which is ugly. You need to write a bunch of lambdas. You will not get a warning for the function that you're using undefined bases (and other useful diagnostics).

defstruct supports custom clauses you can define using define-struct-clause. There are some predefined ones for declaratively doing delegation between classes and what not.

The implementation of assignable places ("generalized variables") is done through macrology. All the place-mutating operators like setf, incf or push are macros. They analyze the syntax of the place and use the material from the back-end definitions to write the code, ensuring that complex places are not evaluated multiple times. Programs can define new kinds of places.

Object access? In TXR Lisp there is a macro qref. It corresponds to a dotted read notation:

  1> '(qref a b c (d arg))
  a.b.c.(d arg)
  2> (expand 'a.b.c.(d arg))
  (let ((#:g0024 (slot (slot a 'b)
                       'c)))
    (call (slot #:g0024 'd)
          #:g0024 arg))

If we suspect a.b might be null:

  12> (expand 'a.b.?c.(d arg))
  (let ((#:g0025 (slot a 'b)))
    (if #:g0025 (let ((#:g0026 (slot #:g0025 'c)))
                  (call (slot #:g0026 'd)
                        #:g0026 arg))))

Lisps are built on macros. And macros are something users have access to also, to do the same things for themselves.

You should use the quality, documented macros that are already provided. That's why that coding conventions document says that you should use macros liberally, just define them sparingly.

Lisp being built on macros has some nice advantages like if you have to work with an older version of some Lisp which didn't have some language feature, you may be able to backport those macros to polyfill that.

Wow, thank you for the detail

> Even so, is there really something that can be done with macros that can't be done, just as elegantly, with simple state machines, or object composition style in other languages?

Many simple macro use cases are covered by lightweight lambda syntax, reflection, or language constructs, e.g. Go's defer, Python's with, and C#'s using. Complex macros can create DSLs or rewrite code for optimization, though. Series[0] is a Common Lisp library that rewrites lazy streams into an imperative loop. LOOP[1] and ITERATE[2] are DSLs for iteration. Schelog[3] embeds Prolog inside of Scheme. Serde[4] generates serialization code.

Here are some old threads on the Lightweight Languages mailing list that discuss use cases for macros. Participants include Trevor Blackwell, Paul Graham, Dave Moon, Todd Proebsting, Guy Steele, Dan Weinreb, and many others.

• Macros Make Me Mad [5]

• macros vs. blocks [6]

• how expressive are they [7]

[0] https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html

[1] https://gigamonkeys.com/book/loop-for-black-belts.html

[2] https://iterate.common-lisp.dev/

[3] https://ds26gte.github.io/schelog/index.html

[4] https://serde.rs/derive.html

[5] https://people.csail.mit.edu/gregs/ll1-discuss-archive-html/...

[6] https://people.csail.mit.edu/gregs/ll1-discuss-archive-html/...

[7] https://people.csail.mit.edu/gregs/ll1-discuss-archive-html/...

> is there really something that can be done with macros that can't be done, just as elegantly, with simple state machines

My guess would be a C# using block. I think the Java equivalent is "try with resources" but I don't remember. In Common Lisp it'd be WITH-OPEN-FILE, or a similar macro made on top of UNWIND-PROTECT ...

> or object composition style

builder pattern comes to mind :)

> But from a working-with-other people perspective

There could be macro for that:

  (with-other-people ((bob (lookup :accounting "bob"))
                      (alice (lookup :human-resources "alice"))
    ... some workflow about about last payroll ..
    ) ;; people are nicely disposed at the end of scope

Reply on news.ycombinator.com