Hacker News Reader

Comment by diggan | Hacker News Reader

I think what I was getting at, is the power this homoiconicity gives you when combined with macros, and I'm not sure how template strings would help with either. Say you want to have a function that receives the code passed into it, instead of whatever the code evaluated to return, could I somehow achieve that with AST and/or template strings in Python?

Say I want `what-code(1 + 1)` to receive `1 + 1` as the argument, not `2`, would either of those things let me do that? At a glance, and without diving deeper, the Ast module could make it so `what-code("1 + 1")` would let the function parse the string and get an Ast, which we could do stuff with, but it pretty much ends at that and is non-ideal for many reasons.

View on HN

Well... what are you trying to do with "1 + 1"? Lisps are "homoiconic" in the sense that everything is a list, but honestly? When I worked with Scheme in the past, I almost never manipulated code as actual lists. It's not robust. I used match and quote/unquote. What's where I see template strings.

So what you could do is something like this:

    addition = "1 + 1"
    multiplication = t"{addition} * 7"
    result = eval(multiplication)  # result is 14

Instead of inserting addition as an exact string, producing "1 + 1 * 7" which is 8, it would parse as an AST and insert the addition node as a unit, so that you get the intended result. You could also do an alpha conversion if the injected code declares variables to avoid name clashes.

It wouldn't let you do true macros, but it would make codegen easier and less error-prone.

> Lisps are "homoiconic" in the sense that everything is a list, but honestly?

That's not how I understand homoiconicity, "everything is a list" or not has nothing to do with it. The point is that whatever structure the compiler uses to understand the code (slightly loose), is the same code you write, since code is represented as data. But if it's maps, lists, booleans or whatever isn't the important takeaway from that.

In your example, how would I, after the multiplication line but before the result line , manipulate "1 + 1" to read something else like "2 + 1"? In Clojure for example, it would be (+ 1 1) so if you wanna change 1 to 2, you use the typical "replace item in list by index" function, but with Python you're either stuck with AST or "code-as-strings". I think that's where a lot of the complexity and error-proneness gets in.

Okay, but even in Lisp, I'm not sure that's a good way to go about it. What do you want to do when you manipulate code, generally? Oftentimes you seek to find all instances of a specific pattern and modify part of that pattern. So why not do something like this:

    transform = Replace(pattern="$x + $y", replacement="($x + 1) + $y")
    transform("1 + 1")       # "(1 + 1) + 1
    transform("3 + 4 + 5")   # "(((3 + 1) + 4) + 1) + 5
    transform("10 * 1 + 1")  # no change, because the expression is actually (10 + 1) + 1

If you give me a primitive like that in Lisp, I'm going to use it. It's a lot more trustworthy than "replace item in list by index". Because it's nice and all that everything is a list in Lisp, but manipulating code as lists (or maps or whatever) like you describe isn't going to get you very far. If you want to do something as simple as replacing all calls to (f x) by (f x 1), you better hope you don't come across code like (let ((f 123)) ...), for example. Outside of toy cases and examples, you need robust primitives that understand the code's semantic structure.

There's actually a library called ast-grep which does something very similar to what you're describing. They have an example in their introduction which performs a find and replace operation on a JS AST using a pattern:

  ast-grep --pattern 'var code = $PAT' --rewrite 'let code = $PAT' --lang js

https://ast-grep.github.io/guide/introduction.html

Reply on news.ycombinator.com