Hacker News Reader

Comment by xg15 | Hacker News Reader

To be honest, what I don't like about this is that it once again operates on the character level. I feel this brings us back to all the issues we had with the C preprocessor and, in addition, makes any IDE analysis/assistance hard to impossible.

I feel the tool would be more useful if you could process the target language's AST instead. This would result in hygienic macros as well as making the code easier to analyse (and might solve the whitespace problems as well, as a formatter could render the whole tree in the end, after any code generation was already applied)

View on HN

What you want is something akin to Terra: http://terralang.org/

Looks similar to https://ivorylang.org/

What you want is access to all the intermediate forms in the compiler, not just the AST.

Didn't know about that yet. That indeed looks interesting. Thanks a lot!

https://github.com/pallene-lang/pallene https://github.com/titan-lang/titan

There is a big difference between Cog and the C preprocessor -- the generated code actually lives within the file, and your IDE / type checker / linter will process it. Sure, they aren't hygienic, but for lightweight macros (which, in non lisps, is the majority of the macros people write imo) hygiene isn't a must-have.

I have found Cog to be very helpful while writing Rust code. I love Rust, but I'd prefer to write my macros in a different language (like Python). I also find debugging macro expansions to be much more painful than debugging Cog macros.

Maybe I don't understand the nuances here, but I don't see how Cog code isn't hygenic: the Python namespace it operates in has nothing to do with the namespaces in the containing file as a whole. There's no way for names chosen by the Cog -writer to conflict with the names in the larger file.

Rather language-specific, but this is exactly the use case of Lisp's homoiconicity. The language's code and data are structured identically, so you can write code that takes other code as input.

The file containing the code is text. What AST do you mean? Cog doesn't have to understand anything about the structure of the containing file, so it can be used on any text file.

Processing the language’s abstract syntax tree is much less error-prone, for one. C macros (and any other text based macro system, including Cog) can be harder to debug than an AST version. While you could implement something complex, like list comprehensions in Cog, it would be a lot more complex and buggy than the AST-modifying equivalent[0].

That’s probably not what the author (you?) had in mind for Cog, but it is effectively a macro system so tying into the AST could be a big advantage. The only nice part about the text version is that it doesn’t presuppose a host language.

[0]:https://stackoverflow.com/questions/267862/what-makes-lisp-m...

Less error-prone in one sense, more error-prone in another. Any time you have an AST integration, you will run into versioning and integration issues. Consider for example a modern ES6 stack with Babel -- will all your third party tooling recognize the latest syntaxes recognized by Babel? And if so, will it all output AST to text in the correct way? Probably not. Same goes for versioning in languages like Python 2 vs 3 where the AST is only slightly different.

It's much simpler for a tool like this to be "dumb" -- leave the correct syntax to the human, since it's a lot easier for a human to deal with on a one-off basis than to have a group of developers writing and maintaining dozens of AST parsers and code generators.

Given the number of code injection vulnerabilities and escaping confusions we see, I disagree with the assertion that humans are good at understanding code in the same way that parsers are.

Also, any modern IDE already needs to parse the AST anyway to provide any half-decent inspection features. The grammars/specs of most popular languages are also readily available and well-maintained.

On the contrary, if you add search/replace style macro processing, you actually make the code more difficult to formally analyze because, then not even an IDE could build a meaningful AST without actually expanding the macros. (which in this case would mean executing arbitrary python code)

> Also, any modern IDE already needs to parse the AST anyway to provide any half-decent inspection features. The grammars/specs of most popular languages are also readily available and well-maintained.

But do those IDEs actually execute the in-language generators? Which AST are they reporting, the preprocessed or postprocessed?

I've heavily used M4 with C code before and what I liked was the ability to see and run tools against the postprocessed code. Textual replacement can be thought of as a worse is better approach, a principle behind many of the systems that people enjoy and even find to be elegant (at least as long as they don't look too closely).

The point with cog is that you will have the output in the file also. The ide doesn't need to understand the generator.

I like Cog. It is usefull for a set of problems. Eg. writing big enums for cpp with enum-to-string and back functions etc.

I can understand the priority of making the tool language-neutral, but by making it agnostic of the underyling structure, this can also cause lots of "code injection" like problems where the generated code behaves in many unintuitive ways. See e.g. https://stuff.mit.edu/afs/athena/project/rhel-doc/3/rhel-cpp... for examples.

But cog can be used for other file types. I can imagine adding it to documents to get a kind of poor man's Mathcad.

I wonder if this could be integrated with Emacs Org-mode?

Reply on news.ycombinator.com