Notes on Programming in C (1989) | Hacker News Reader

Notes on Programming in C (1989)

lysator.liu.se

54 pointsscapbi51 comments

> Simple rule: include files should never include include files. If instead they state (in comments or implicitly) what files they need to have included first, the problem of deciding which files to include is pushed to the user (programmer) but in a way that's easy to handle and that, by construction, avoids multiple inclusions. Multiple inclusions are a bane of systems programming. It's not rare to have files included five or more times to compile a single C source file. The Unix /usr/include/sys stuff is terrible this way.

> There's a little dance involving #ifdef's that can prevent a file being read twice, but it's usually done wrong in practice - the #ifdef's are in the file itself, not the file that includes it. The result is often thousands of needless lines of code passing through the lexical analyzer, which is (in good compilers) the most expensive phase.

> Just follow the simple rule.

This is a ridiculous stance, in my opinion.

Part of the appeal of libraries is abstracting away implementation details. I want to be able to have foo.h #include a hash table, or a binary search tree, depending on #ifdef's or its internal implementation. The user of foo.h shouldn't even have to know about it.

Compilers can easily memoize the position and parsed content of preprocessor directives, and do a very quick skip of an #include file, as a much better way to resolve the problem.

Rob's resolution is an instance of worse-is-better: Instead of slightly complicating the lower-level tool (in this case, the preprocessor), let's complicate all user code in existence.

All Plan9 C programs are written in this way. Paradoxically the number of includes in your code is minimal compared with the usual POSIX aproach. All C programs begin with two includes: u.h and libc.h and usually libio.h and it is often all you need for the startdard library.

The man pages of the libraries describe the libraries you need to include.

Example:

Plan9 cat: http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/cat.c

GNU cat: http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f...

Plan9 du: http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/du.c

GNU du: http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f...

So basically the #include granularity is extremely coarse, because otherwise it is unmanageable.

I think it is nicer to have quite specific #include's:

  #include <stdbool.h>
  #include <stdarg.h>
  #include "hash_table.h"

Because it:

* Gives a list of dependencies at a glance

* Along with preprocessor optimizations to avoid reparsing protected headers, it can actually parse fewer headers in each compilation (reversing the performance argument by Rob)

* Can aid with greps for where modules are used

The comparison is really a strawman -- and the length of the GNU one and its many dependencies relate more to feature creep than to the #include issue here.

At least with GNU's #includes, I can see that it depends on the various stuff it needs for its feature creep.

Why do all the P9 files use `void main` instead of `int main`?

Yes. Instead of returning an integer you use the function exits [1] and the value appears in the shell variable status.

[1] http://plan9.bell-labs.com/magic/man2html/2/exits

This is an introduction to programming in Plan9 C:

http://doc.cat-v.org/plan_9/programming/c_programming_in_pla...

> Compilers can easily memoize the position and parsed content of preprocessor directives, and do a very quick skip of an #include file, as a much better way to resolve the problem.

There are reasons why actual C/C++ compilers don't do this. Additionally, recursive includes is a major reason why C++ compilers are slow.

For more on this, see http://clang.llvm.org/docs/Modules.html

> There are reasons why actual C/C++ compilers don't do this.

http://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html#Onc...

This construct is commonly known as a wrapper #ifndef. When the header is included again, the conditional will be false, because FILE_FOO_SEEN is defined. The preprocessor will skip over the entire contents of the file, and the compiler will not see it twice.

CPP optimizes even further. It remembers when a header file has a wrapper ‘#ifndef’. If a subsequent ‘#include’ specifies that header, and the macro in the ‘#ifndef’ is still defined, it does not bother to rescan the file at all.

Note that this only partially helps with a subset of the problem (and modules as proposed don't fix the issue, they just help a bit more): CPP still needs to make separate passes for each translation unit, for example. I'd recommend reading the link I put in my original post...

Which breaks #includes written to be included multiple times and provide different outcomes based on defined macros.

C based toolchains are just broken.

If a file is intended to be read multiple times with different results, then obviously you don't want to prevent it being read twice. However, we were talking about preventing the hit from reading a file twice unnecessarily. The quoted text says it doesn't bother rescanning if the macro is still defined. How do you get a different outcome from scanning the same file with the same define? Can you provide an example where this breaks things?

Yes, it's an optimization, it'll just make the common case (include guard) faster, without breaking anything.

How do you force the file to be re-read multiple times when the outcome is not idempotent?

I do not understand why you are arguing about this.

There are two scenarios: in one you want to prevent a file from being read multiple times because it wastes resources (and in some sad cases, might cause issues with redefinition). In this case, you use the include guards and the compiler can do a trick to optimize out unneeded scanning.

You have this other hypothetical scenario in which you purposefully want a file to be read multiple times, with a different outcome (this kind of trickery isn't usually possible at all in other languages with module system so it is funny to argue about C being broken). Now I don't think this is a common scenario or even a good one to be in, but it is possible. The solution is simple: don't try to prevent the file from being read multiple times. Why would you try to prevent it, if it is meant to be read multiple times?

You are arguing that a door is broken because you cannot walk in through it if you close the door. Don't close the door if you want to walk in...

Because it is an optimization that relies on well behaved developers and we all know how good such assumptions end, specially in big teams with high attrition.

As I read this, if the compiler determines that (with the #define set) the file is effectively empty, it knows it can just check the #define next time rather than re-reading. Without this optimization, you get the same result slower. It doesn't depend on well-behaved anything to get a result just as correct as without the optimization.

With the single possible exception of a file that changes on disk between the multiple includes in the same build step. That is not a situation I have ever encountered and I am not at all sure it is supported and there aren't a bazillion other things that would break it in any given compiler chain (certainly at least line directives would be wrong, but that's arguably minor I guess).

It doesn't rely on anything. It's a correct optimization that does not change any behavior at all. It only speeds things up.

Just #include it again and it will work like expected.

This preprocessor optimization is only for the case where include guards block off the entire #include file. Ex:

    #ifndef INCLUDE_FOO_H
    #define INCLUDE_FOO_H

    #endif

When the preprocessor sees that, it marks it off and never reads the file again (Unless you were to #undef INCLUDE_FOO_H, but that's an extremely unlikely scenario since it's likely to break your code). If you want to be able to include a file multiple times, just don't use include guards. Even without this optimization your idea still wouldn't work if you placed include guards in your #include file. All this does is speed up preprocessing when we already know that the file has no new lines to be added.

I am not talking about include guards, rather macros that affect the outcome of specific file.

It is not a very clean way of coding, but there are many enterprise developers that provide files without include guards and include them multiple times.

An example would be generic data structures in C done with macros.

    #define DATA_TYPE int
    #define DATA_NAME intList_t
    #include <list.h>

    typedef struct { /* ... */ } Node;

    #define DATA_TYPE Node
    #define DATA_NAME nodeList_t
    #include <list.h>

It's pretty unlikely that list.h would have a trivial include guard, so I don't see how the above optimization would break it. An include guard in list.h would prevent you from using it properly with or without the optimization.

Not when you #undef it. That was just a possible example.

If list.h has no include guard, it gets included with or without this optimization. If it has an include guard and you #undef it, then it gets included again with or without this optimization. If you it has an include guard and you do not #undef it, it does not get included again, with or without this optimization.

In no case does this optimization change the code emitted.

But also: http://clang.llvm.org/doxygen/classclang_1_1MultipleIncludeO...:

"Implements the simple state machine that the Lexer class uses to detect files subject to the 'multiple-include' optimization.

The public methods in this class are triggered by various events that occur when a file is lexed, and after the entire file is lexed, information about which macro (if any) controls the header is returned."

Modules improve things, but that doesn't mean one can do nothing to speed up lexing without them.

> This is a ridiculous stance, in my opinion.

> Part of the appeal of libraries is abstracting away implementation details. I want to be able to have foo.h #include a hash table, or a binary search tree, depending on #ifdef's or its internal implementation. The user of foo.h shouldn't even have to know about it.

I agree with your premise (re: abstraction of implementation details), but I think you've applied it incorrectly to reach the wrong conclusion.

If the implementation of foo (contained in foo.c) uses a hash table or binary search tree, then foo.c may of course include whatever it likes. On the other hand, foo.h is (by definition) the interface to foo; any headers the interface requires (e.g. for function argument types) ought to be explicitly understood by the user of the library.

It depends on whether you delegate all memory allocations to the C file.

In the C code I work on, dynamic memory allocations are considered expensive and problematic (fragmentation, runtime errors) so we prefer to move the responsibility for allocation as far up the call-stack as possible, where it is often possible to minimize or coalesce dynamic allocations.

To do this, the interface files must expose the correct sizeof() so that the caller can allocate. We do this by splitting the header file into foo.h and foo_private.h (foo.h #includes foo_private.h). This separation is not for enforcement, but for API clarity and convention (if it's in a private.h, you have no business assuming anything about it).

This forces us to rebuild when the sizes of private structures change (which can be a significant downside) but allows us to avoid dynamic allocations, their costs and their many potential error paths.

In this approach, foo_private.h may very well #include a hash_table or whatever header which should not* be exposed to the #includer.

It seems to me that all this fuss over multiple includes and include guards would be solved by something like a #stop directive:

  #ifdef STDIO_H
  #stop
  #endif
  #define STDIO_H

which would cause the preprocessor to skip the rest of the file. How would this not be a better solution? What am I missing?

It would be faster, but this would still cause the compiler to open and start parsing the file unnecessarily (possibly multiple times from multiple included files).

Another approach would be to have a pragma that means: "just don't bother including this file again if you see it", and have the compiler remember it. Perhaps there is - its a long time since I programmed in C.

There is #pragma once

It's not standard, but it is supported by many compilers.

I wish for a standard #once directive. It should be very simple to implement, increase preprocessing speed and reduce the size of visually disturbing boilerplate in header files.

True, supported by GCC, Clang and Visual Studio and probably other compilers ...

Good point. Opening a file has non-trivial cost, so perhaps the added cost of burdening the compiler with avoiding needless includes is worth it. As a long time C hacker I hate to say this, but I guess the whole include mechanism is a bad design or, perhaps, a design that didn't scale well.

If the compiler can remember there's a pragma like that, then it can also remember there's an include guard wrapping the file.

Todays compilers may be fast enough so that using #ifndef that way would not make much of an impact, but back in 1989 when this article was written, presumably the time spent was much greater.

Have you looked at the code for plan9? It follows this principle and I think it works fairly well. Its much nicer than wading through a large number of preprocessor directives.

The ugly "includes hack" seems ridiculous today, but in the 90's such techniques were the only way to compile large C/C++ systems in a reasonable time. Messing with includes and reducing file dependencies while less important nowadays are underappreciated elements of software design (called physical design as opposed to logical design). For better historical context see: John Lakos - "Large scale C++ software design" (1996)

It can also be dangerous. Take (for instance) a header file that relies on a specific integer size. Using this scheme one could include a different/wrong typedef and get yourself into trouble.

Objective-C's #import directive basically pushes this work to the compiler, and is quite effective.

> This is a ridiculous stance, in my opinion.

I always find it humorous to find posters here and reddit and other boards say these things and I believe they would make the same statement about the Ten Commandments from God.

Yes, at least a few of the Ten Commandments from God are ridiculous. What's your point?

My point is most people could learn a lot from him and aren't in the position to make claims against him.

There are plenty of things Rob says I strongly disagree with, and even if you're very impressed with his work, your comment is still a very silly appeal to authority.

In this life, you must find someone to believe in. Dismissing this as a "silly appeal to authority" is even sillier.

I have plenty of people I believe in. In the sense that almost whatever they say, I'd think really really hard before disagreeing. Even then, if I find them unconvincing, I'll eventually disagree.

I personally don't hold Rob to be such a figure. I believe he's quite ignorant when it comes to many important topics in computer science (e.g: he made some comments that make it clear he does not understand parametric polymorphism). I find his work on Go to be a minor variant of Algol69 (and not necessarily an improvement, at that!), throwing away 40 years of real improvements in PLT.

I'm not a fan of Go, its syntax, or Rob Pike in the slightest but aren't go-routines as first-class/ default at least an improvement over most Algol-derived languages' support for concurrent execution models?

In the final analysis, this is hardly enough to convince me to use the language (as I've never found concurrency difficult in any other language to begin with and there are better ways than go-routines to make it 'easy'); however, I'm probably not in the target audience.

My impression is that Pike is endeavoring to create for his new corporate overlords a language that fungible enterprise developers can be at home in. And, it probably is better than Java.

Maybe you need someone to believe in. Stronger minds decide for themselves (interestingly enough which is the advice given by Rob in the intro of this article).

A strong mind listens to others who may be smarter than themselves and have more experience and doesn't worry about who the stronger mind is.

I think you've merely re-stated my point. A strong, independent mind makes determinations on ideas objectively without respect to the sources of those ideas.

And, I find Rob Pike completely unimpressive. I would suggest that you find other folks smarter than yourself (and preferably smarter than Rob as well) to blindly follow (if that is your thing).

And now you've just proven my point.

I think you're confused. What is your point exactly? This whole discussion has been about how your original post was a baseless appeal to authority (which you still seem intent to make). My point is merely: if that is your sort of thing, then you can find someone better than Rob Pike to fill the role. How is this in any way the same point that you yourself are making?

It's interesting to see how some of these opinions are reflected in Go, because Rob Pike helped design Go about 20 years after writing this essay:

Typography: Automatic formatting is part of the standard Go tooling and culture.

Function pointers / Protocols: This section seems to be decsribing a technique similar to Go's interfaces.

Include Files: Rob Pike talked about this in his presentation at SPLASH [1,8:00-12:30]. He mentioned that Plan 9 had fast compile times in part due to following this discipline. He contrasted that with one of Google's C++ code bases in which the compiler had to read 2000 bytes for every byte of source. That was largely because of includes, and was a problem for compile times. A little later he talks about Go's method of importing dependencies, and presents them as a big improvement to the situation.

[1]http://www.infoq.com/presentations/Go-Google

Nice list of books they have in their library: http://www.lysator.liu.se/lib/boklista.txt

" Procedure names should reflect what they do; function names should reflect what they return." - Never thought about naming this way, but is what I do (:

Reply on news.ycombinator.com