Hacker News Reader

Comment by onan_barbarian | Hacker News Reader

From the article:

"In 1984, a compilation of ps.c, the source to the Unix ps command, was observed to #include <sys/stat.h> 37 times by the time all the preprocessing had been done. Even though the contents are discarded 36 times while doing so, most C implementations would open the file, read it, and scan it all 37 times. Without great cleverness, in fact, that behavior is required by the potentially complex macro semantics of the C preprocessor."

"Great cleverness" apparently means "the ability to detect that a file has a standard 'include once' pattern and only process it once". It hardly seems like rocket science.

View on HN

No, that's called "dirty nonstandard corner cutting that will cause you to curse up a storm at $VENDOR who wrote their compiler to be fast but it breaks on $POPULAR_LIB which gets cute with the preprocessor, and yes they shouldn't do that but are you going to rewrite it?"

That's incorrect. Optimizing the guard macros works in all cases as long as you use the standard pattern and the C Preprocessor memorizes the name of the guard macro.

How does this happen then?

"The construction of a single C++ binary at Google can open and read hundreds of individual header files tens of thousands of times."

An excellent question which deserved an answer in the paper.

Walter Bright (who created D to solve some of the same problems Go aims to solve) wrote something a few years back which touches on this topic:

http://www.drdobbs.com/cpp/c-compilation-speed/228701711

Off hand, I can imagine problems with: multiple guarded blocks at the top level, things outside guards, things that look like guards but aren't (#IFDEF conditional sections, perhaps), things that don't look like guards that are, things that affect which guards are switched on in a dynamic way (detect a feature and enable a lib)...

Some of those could be non issues. There could be other issues I lack the experience to guess. It's no use saying "so don't do that" - not all code is under your control. Honestly if it were simple, it would be solved.

As I mentioned, the cpp needs to recognize the standard pattern and act on it. If you have multiple guards at the top level, you are trying to do something clever which cannot be optimized -- and also cannot be expressed in Go.

You mean it cannot be expressed using Go's (non-existent) preprocessor. I am relatively confident that you'd be able to express it in Go (perhaps sacrificing speed, in the process).

I can see a reason why the potentially complex macro semantics of the C preprocessor would require this behavior, and why you would certainly need to re-evaluate certain macros you had already processed whenever it was included. Doing that without impacting any of the preprocessor's functionality? Great cleverness - for really no reason than faster compiles - which isn't that big of a deal to most people.

There's plenty of magic that can be done in the C preprocessor that makes it sometimes possible to #include a file twice with different effects than #include-ing it once.

#pragma once and Objective-C's #import declaration both do that, though.

1984 was a long time ago, I think most/all compilers ended up adding this optimization at some point.

It is somewhat complex to do it correctly, see: http://gcc.gnu.org/onlinedocs/cppinternals/Guard-Macros.html

Someone can still throw in an #undef __YOUR_HEADER_H__ somewhere, and while that may be playing with fire, the spec says that the header file should be included again. There's so many corner cases (and ifdefs are used for so much more than guards) that you really do need to reevaluate it every time.

It really doesn't seem that complicated. When reading an include file the first time, check if everything is enclosed in a preprocessor conditional, one without else-branch. If so, remember the condition and re-evaluate it before opening the file again.

And if you're looking for rocket science you clearly missed a section.

Reply on news.ycombinator.com