Hacker News Reader

libclang has been with us for ~15 years already.

> I'm not so interested in being able to compile but being able to parse

Chances are that you're not going to be able to walk the AST without getting errors unless the (lib)clang can cleanly parse the code. There are at least two prerequisites for that: (1) marrying your build system with the (lib)clang so that you pass the exact include directories, include paths, build flags, etc. for any given translation unit, and (2) ability to build the code with clang. If you use CMake then it's going to be a little bit easier. If not, I recommend first switching to it. You will also have to be somewhat creative about making it possible to correctly parse the code in the header files - a header file fundamentally isn't a translation-unit.

Back in the days, I wrote some of the utilities built on top of the libclang and the process was very hairy. Although I pulled it off, for the reasons above it required a lot of unsexy work exactly around the integration of (CMake) build system, patching the codebase itself, and working around the design limitations imposed by the language itself. Occasionally, libclang wouldn't offer me enough control over AST so I also had to work around those limitations by patching the libclang code itself. libtooling seemingly gives more control but I have never tried it.

If you still want to try it, I think your best bet is to find something similar that already exists and build upon it.

If all you want is pure syntax with no semantics, check out tree sitter. You can get it past macros by pre-processing via alternative implementations in some cases.

But most likely you want some semantics. The "one weird trick" you can pull here is that if all you want is syntactic manipulation, you mostly don't have to care about semantics and codegen, or anything past that (like linking or embedding resources) -- and you don't really have to care about most of the compiler flags, either. That takes out a fair bit of complexity. Moreover you can do the migration to Clang file-by-file, and you can do that with #ifndef __clang__ wherever you're certain the code is unaffected by your migration (which you can detect via compiling the file).

And on top of that you can use the most relaxed warnings possible in Clang - think MSVC compatibility flags and disabling all warnings and such.

I'm not suggesting it's trivial, but I've done it before, and it was not as daunting as I first imagined it. If you haven't tried these already, I would definitely give it a shot for a few files, and see how long it takes you on average per file.

> I was working on a big project with old code that needed a radical overhaul to make it ready for new boost and new C++ versions.

You might find Kristen Shaker's CppCon 2023 talk to be intriguing.

https://www.youtube.com/watch?v=torqlZnu9Ag

> I know there are several tools for clang that do rewrites and the unfortunate problem was that clang at the time wasn't able to build that code for some reasons that I cannot remember now.

Oh. Well that's unfortunate. You're probably going to have a bad time if no version of clang is able to compile (eg, libclang probably won't help). But GCC has something similar:

https://stackoverflow.com/questions/15800230/how-can-i-dump-...

Alas, I'm not familiar with either tool. I would find it interesting to read a long-form blog post from you with more information and what you end up doing!

castxml (https://github.com/CastXML/CastXML) may be what you want. It uses the Clang front-end to output an XML representation of a C or C++ parse tree. It is then possible to turn this into what you want. I've used it and seen it used to generate code to do endianess conversion of structures from headers, or RPC code generation for example.

It can be used from Python through pygccxml (https://github.com/CastXML/pygccxml). The name comes from a previous instance, gccxml, based on the GCC front-end.

Both castxml and pygccxml are packaged in Debian and Ubuntu.