Hacker News Reader

Comment by DanielHB | Hacker News Reader

Half of the reason Lisp never caught on is that other languages have become better and added many of the Lisp features. First-class functions especially are is the killer feature you can't live without anymore once you tasted it.

The other thing that pg keeps hammering on about is Lisp Macros which I am not well versed enough to properly say how good of a feature they are. The alternatives most other languages have are much less powerful, but at the same time it seems that macros are very... abstract-y?

Modern software design thinking says that abstractions should be avoided when possible because they make the code harder to grasp for people not familiar with the abstraction on top of making the code too rigid and hard to modify once new requirements come in.

The alternatives to Lisp Macros like codegen, method annotations, Rust Macros. Are all much less powerful and usually harder to create, but maybe that is a good thing.

View on HN

Our 30kloc Julia codebase only has one key macro, but it's a really important one. It's ~70 lines of code that's saved us ~5,000 loc, and more importantly let researchers do a lot of stuff that would need normally need more programming expertise. We might be able to live without it, but it would have cost us a significant amount of time maintaining and debugging, and a much slower research feedback loop.

Other languages with less powerful macro systems often create a DSL (domain specific language) for this kind of problem. With Javascript it can even be mixed in with normal code through the use of tagged template literals. For example:

  const query = sql`SELECT * FROM User WHERE name=${name}`

It is so widely used in the JS ecosystem that editors often have plugins to add proper syntax highlighting to some tagged template literals and some frameworks pre-compute some tagged template literals at build/bundler-time.

It is definitely much harder to do it compared to Lisp-like macros though. But maybe the barrier to entry to create such powerful abstractions is a good thing. In my previous post I didn't say you should never use powerful abstractions, you should just not overuse them.

We did in fact start by creating a data structure that we parsed!

The main problem we were trying to solve is essentially that we had a lot of code like:

f(x) = sma(ewm(x))

g(x) = slope(ewm(x))

And we wanted to avoid recalculating ewm(x), while also not holding memory for it for longer than necessary. Our production use cases would have tens of thousands of intermediates, so the potential saving from getting this right is huge.

Naturally a DAG is a great fit for this type of problem. We started by creating an explicit DAG, where everything like ewm(x), sma(ewm(x)) would be a node in the DAG. We could then evaluate the DAG efficiently using multithreading, dropping nodes when everything that depended on them had been calculated, etc.

And I would usually stop there, but there were two main challenges. One was that the syntax was pretty clunky, and we wanted researchers (who weren't software engineers) to be able to use it. The other was that we wanted _any_ reference to something like ewm(x) to map to the node, even if they were used in very different places – and it was quite difficult to map these dependencies without explicitly tracking.

It's hard to quickly summarize our macro, but basically what it let us do was write the short syntax and track dependencies using the underlying Julia expressions. We mapped those expressions to nodes in the DAG, and that let us calculate the dependencies without having to track them explicitly. I don't think this would have been possible without some amount of homoiconicity and macros.

Can you elaborate? I'm using Julia macros to help make my research software easier to extend for others, so would be interested in your experiences.

See my reply here: https://news.ycombinator.com/item?id=40587847 Happy to answer any other questions!

> The alternatives to Lisp Macros like codegen, method annotations, Rust Macros. Are all much less powerful and usually harder to create, but maybe that is a good thing.

I kinda agree that macros can lead to a lot of (bad) abstractions if you overuse them, but when you need a macro a lot of times the alternatives a language provides can somehow feel more magical if you try to get them to do macro-y things (decorators/attributes... languages always seem to mix those two terms) or just cumbersome and in my opinion kinda an ugly hack to paper over the fact that your language is lacking in some way (codegen). I'd much rather read a simple (lisp) macro then understand how your ugly codegen system works.

I dont have enough knowledge to comment on rust macros though.

> First-class functions especially are is the killer feature you can't live without anymore once you tasted it.

Wouldn't it depend on application domain what's a killer feature & what's not?

"Include if lightweight & very useful" seems appropiate. But what's considered "lightweight" varies depending on who you're asking / where you're looking, doesn't it?

I disagree, first-class functions and closures are just so very useful to the point that even languages that have added it after initial release still feel gimped today because a lot of the older libraries (and probably most of the stdlib) don't use them extensively.

Like passing a function to any function that iterates over a data structure (like array sorting), or registering an event handler. Sure you can usually achieve the same thing without first-class functions but it feels so much less concise.

I just love first-class functions so much that the main reason I dislike python is that it doesn't include proper anonymous functions.

Reply on news.ycombinator.com