I really hate the C/C++ preprocessor.
I really hate the C/C++ preprocessor.
It does mean that you have to know exact compiler invocation before you can start parsing. You can even invoke the original compiler to preprocess the file and treat it as completely separate phase that can be mostly ignored after it's done. There is plenty of complexity making extracting AST difficult after preprocessing. You could argue of how much of analysis belongs to syntax and what's already semantic analysis. But in practice and possibly at least partially due nature of C++ language the parsers making AST also do a lot of type analysis. Due to features like constexpr it they even execute the logic of code.
This is fine if you just want to analyze that particular instance of what the program can become, but it's not fine if you're trying to do something to the program as a whole.
Years back doing code analysis tooling (and struggling with, and being discouraged by, the TFA issue), I downloaded "a lot" (in an old pre-github sense) of code, and IIRC, found preprocessor use was so stereotyped that almost all could be easily parsed and included in the AST. Using a backtracking parser though, rather than the usual "having chosen a cripplingly inexpressive parser tech, difficulties with parsing came as a surprise". ;)
Despite being burned by gcc's "hide the AST" policy, it's not clear to me when/if it became the wrong thing. It's so easy to underappreciate the contingency of history. We're on a timeline where open source, and the internet, more or less succeeded - many perhaps aren't so lucky. On the other hand, emacs, at least with lisp on lisp machine, could be an immensely powerful integrated experience, that perhaps needn't have been quite so stagnant and lost.
Proc-macro in Rust is very tricky to analyze statically, some are even impossible (due to non-deterministic behavior)
One way to look at this is that partial evaluation of Rust or Scheme macros is very tractable, because there are very few side effects / side channels. But if you have a lack of hygiene or the C preprocessor, it's very difficult and almost everything becomes a "stuck term" whose evaluation is contingent on earlier evaluation.
Because of procedural macro, it becomes practically impossible to find all occurrences or rename a particular symbol regardless of the #ifdef or #[cfg] or whatever.
I am not sure whether Rust procedural macros are always that hygienic, so fair point if they aren't.