Show HN: PyCParser, C parser and interpreter in Python | Hacker News Reader

Show HN: PyCParser, C parser and interpreter in Python

github.com

112 pointsalbertzeyer19 comments

All for projects in this space, but you're going to generate so much confusion naming it the same thing as Eli Bendersky's pycparser (which you even acknowledge the existence of in your Similar Projects section).

The alternate capitalization makes no difference to a search engine.

I know, that's unfortunate. I think I learned only later about Eli Bendersky's pycparser and then I was too lazy to change the name, although you are right, I should have. But as my project is mostly a toy project at the moment, and not sure if this will every get serious usage, I'm not sure if this is too critical. Also, whoever searches for pycparser, and ends up at this homepage, will find the link to the real pycparser project right there in the readme. Both projects are quite old btw. I just checked. Eli Bendersky's pycparser is from May 2010 (or that was the first Git commit, which seems to be an import from an SVN repo), and I started mine in June 2011. I never really got any complaints about confusions.

pycparser is actually from ~2008 :-) [http://eli.thegreenplace.net/2008/11/15/pycparser-v10-is-out], but it was probably less easily discoverable in those days - it was initially in SVN and the first release was a tarball drop.

pycparser doesn't do anything about interpreting C, it ends its job at creating the AST.

By the way, pycparser has no external dependencies. It uses PLY but vendors it. I had to port it to Python 3 before PLY did itself, and stuck to the vendoring because it's so convenient.

FWIW I was confused by this title because I know of Bendersky’s pycparser project.

Looks awesome - the interpreter seems really handy for learning C, too.

It'll be a neat milestone once it's complete enough to interpret Cython. :)

He's already started on it: https://github.com/albertz/PyCPython

Is python the optimal language for writing a C parser? It seems like text manipulation is not one of python's strong points (not that that is a strong point of that many langugages anyway).

Text manipulation is far easier in Python and a lot of other high-level languages than it is in C.

Incidentally, this ease (such as string concatenation with '+') also makes it easy to unknowingly/accidentally create a quadratic or higher complexity algorithm.

I certainly like text manipulation (well string manipulation at least) in python much more than other languages I use (C, C++, Matlab). It might just be the case that python is the best tool in the authors tool belt for this sort of work.

I'm curious what language you would consider optimal for writing a parser in?

> I'm curious what language you would consider optimal for writing a parser in?

I've been researching that question, and was hoping someone would reply and point me in a useful direction.

I've written a couple parsers in D using https://github.com/PhilippeSigaud/Pegged and they seem pretty fast and easy to verify. Of course since it generates a lot of code at compile time, you can end up with somewhat slow builds.

Can it run cpython?

from the readme: https://github.com/albertz/PyCPython

A problem with tools like these is dealing with the C preprocessor. You can either parse preprocessed code (in which case you can't deal with the source file as written, for example to do rewrites or searches on it), or you hack on it to somehow accept "common" preprocessor idioms (but that's fragile and imperfect).

It has a preprocessor. You'll start to see it at around 20% of the way through cparser.py .

I don't know how standards-compliant it is, because I just quickly skimmed over the code, but fortunately the correct algorithm has been documented:

https://www.spinellis.gr/blog/20060626/cpp.algo.pdf

Of all the pitfalls how can the text replace preprocessor be the hard part? Just apply the preprocessor on any input before the rest of the processing.

The project looks kinda cool. I wonder if I could hack it for "real time feedback" of c code in Eclipse some how.

That's fine if you want to compile or interpret the program for a particular set of definitions. It's not so good if you want to manipulate the program text as a structured object.

That's fine if you want to compile or interpret the program for a particular set of definitions.

That's what this does. No one said anything about your second point.

I was pointing out a problem with C parsers. This is not really the tool's fault; it's a problem with the design of C itself. One would not have this problem with, say, Java. I don't much like preprocessors for this reason.

Reply on news.ycombinator.com