I would love to read more about "code as data" in general. I would appreciate any resources you can provide. Thanks.
When you actually start using these languages, you find yourself tempted to leverage this property and try writing lots of macros. But then you go back to the community and everyone tells you "the first rule of macros is you don't write macros." It turns out that if you use macros all over the place then your code becomes impenetrable. The problem is that macros allow you to circumvent the expected order of evaluation and produce your own novel syntactic structures. This puts lie to the old claim "Lisp has no syntax." In reality, Lisp has tons of syntax, it's just informal and buried within the definitions of macros.
It parallels the problem put forth in Jo Freeman's famous piece, "The Tyranny of Structurelessness" [1]. Any group that purports to adopt a structureless organization ends up having a hidden, informal structure. This is the way of Lisps as well.
I actually had heard of at least KRC and Miranda even before the first time I've read the paper, but even if I hadn't I would immediately be able to think of several languages with some or all of the additional features over the Lisps he was comparing them to for which he describes them as superior to Lisp for the purpose, including some modern Lisp derivatives.
Pattern matching, mathematical notation, and static typing with use defined types are all features that have become more, not less, common in languages since the paper was written.
Equivalent arguments exist for Ruby. Ruby isn’t bad because someone wrote a DSL that should’ve been a method and maybe a hash.
I happen to own a copy of HTDP which I bought for the course. It's a suggested textbook but not required at all. I haven't had time to go through it though.
At the end of the day, every line of code you write is some data in a bespoke data format. Most programming language implementations leverage this, and academic compiler courses make you do the same work. A program lexes, parses, modifies... that data structure and eventually rewrites it to some other programming language, usually something lower level.
If you exposed that underlying structure to the programmer, you give them a lot of power! The simplest way to see this is that any time you write a highly repetitive bunch of code, wouldn't it be great if you could ... write some code that wrote the code for you? You know how to systematically express the thing you want: you can probably say it a lot more crisply in a few lines of (especially functional) programming languages.
Once you get rid of simple repetitive stuff, you realize that you can do so much more with this. You want advanced pattern matching? Sure: you can go implement it as a library. async/await? Library. Type checker? Library. New convenient way of defining functions? Library. New syntax for matrix math? Library. Et cetera :-)
People in other programming languages have figured this out too. That's why we have code generators. The difference is that code generators are far enough removed from what you're actually doing (because e.g. they're glorified string concatenation) to be quantumly less useful. Generally speaking advanced IDEs will do this sort of thing: actually parse the code you're working on and let you do fancy tricks with it. Lisp is like having that power, all of the time.
It takes a symbol and figures out the name of a C function, how to most efficiently call it (with type hints for perf). If you screwed that up in one location it might have security consequences. But I _can't_ scre it up, because instead of copy pasting that code a gazillion times as I would have in Python, I just figured out what I want once and then use that functionality a gazillion times :)
(defmacro let (bindings &body body)
`((lambda ,(mapcar #'first bindings) ,@body)
,@(mapcar #'second bindings)))
(define-syntax-rule
(let ([id expr] ...) body ...)
((λ (id ...) body ...) expr ...))I learnt loads translating the Lisp to Clojure.