Hacker News Reader

Comment by weeber | Hacker News Reader

Simply add:

  -include $(OBJS:%.o=%.d)

in your makefile (with -MMD in CFLAGS).

View on HN

Don't you need a pattern rule for %.d? GNU Make doesn't seem to come with this rule.

There is a profoundly ugly example here:

https://www.gnu.org/software/make/manual/html_node/Automatic...

    %.d: %.c
        @set -e; rm -f $@; \
         $(CC) -M $(CPPFLAGS) $< > $@.$$$$; \
         sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@; \
         rm -f $@.$$$$

I copied it into a running example here for anyone curious:

https://github.com/oilshell/blog-code/tree/master/dotd

If you have -MD or -MMD in your CFLAGS, GCC (and Clang) will generate the .d files during compilation without you having to add anything else to the Makefile, and without requiring ugly sed magic.

I just tried this, and it seems wrong on two counts:

1) The sed magic is required for adding the dependencies of the .d file itself. For example, if the timestamp of your header changes, it may have acquired new dependencies, and a new .d file has to be generated BEFORE make determines if the .c file is out of date.

See:

https://www.gnu.org/software/make/manual/html_node/Automatic...

The purpose of the sed command is to translate (for example):

    main.o : main.c defs.h

into:

    main.o main.d : main.c defs.h

The first line isn't correct because the .d file itself has no dependencies.

2) By the time you are compiling, it's too late to generate .d. The .d files are there to determine IF you need to compile.

EDIT: I am trying to generate a test case that shows this fails, but it seems to actually work.

Hm yes I'm convinced it works, but I have to think about why. I guess one way of saying it is that the previous .d file is always correct. Hm.

(2) is not quite correct. The old .d file from the previous compilation is actually all you need to determine whether the .c file needs to be recompiled. It works in all cases. If the .c file is new (or you're doing a clean rebuild of the whole project,) it will always be compiled, because there will be no corresponding .o. If the .c file, or any of the .h files in the old .d gain new header dependencies, they must have been modified, so their timestamps will be newer than the .o file from the last build, hence the .c file will be recompiled and a new up-to-date .d file will be generated (because a new .d file is always generated when the .c file is compiled.)

If (2) is not correct, then (1) is not needed either. The old .d files from the last compilation pass are sufficient to know which files need to be recompiled in the current compilation pass. Make does not need to know the dependencies of the .d files themselves, it just needs to load all the existing .d files at startup.

EDIT: Yep, I'm fairly confident this works :D. I don't know if whoever wrote that manual page knew about -MD, but I think it might be newer than -M, which would explain it.

Hm yes I just figured that out the hard way -- <scratching head>.

This feels hacky, but yes it seems to work. I'll think about it a bit more. (I might clone this feature for a build tool -- since the gcc/Clang support is already there, it seems like any serious build tool needs this. Although some have their own #include scanners which is odd.)

Thanks for the information!

I guess a simple way of explaining it is that if there are any new header dependencies, one of the files that make already knows about must have been modified to add the #include statement, so make will correctly rebuild the .c file and generate the new .d file, even though it's working on outdated dependency information.

Though, I guess I wasn't quite correct either. See plorkyeran's sibling comment re: generated header files.

> Although some have their own #include scanners which is odd.

I once worked in a place that had its own #include scanner (partially because it used so many different compilers and other tools... suffice it to say that Watcom C++ was in the mix). To make it work, you had to install a custom filesystem driver that intercepted disk reads during compilation and logged them. A rather... bruteforce approach. But it had the advantage of working with everything.

The problematic case is with generated header files. Suppose foo.c includes foo.h, where foo.h is generated by a separate command. On a clean build, there's nothing telling Make that it needs to build foo.h before foo.c, so it may not happen (and worse, it may usually happen but sometimes not when doing parallel builds). A separate invocation of `gcc -MM` works for this, as when it generates the dependency information for foo.c it will see that it needs foo.h before you do the actual build.

Personally I've never found it too burdensome to just manually specify dependencies on generated files.

Wouldn't the header need to be explicitly listed as a dependency to prompt it's generation anyway?

There's a -MT flag which does what the sed line does. From the gcc man page:

  Change the target of the rule emitted by
  dependency generation... An -MT option will
  set the target to be exactly the string you
  specify...

So in your example one might do something like

  -MT "$@ $(basename $@).d"

which would output

  main.o main.d : main.c defs.h

for the main.o target.

This is what I do too, and it seems perfect to me for adding set-and-forget dependency awareness to an ordinary Makefile. The first build with a new .c file will work. Its .d file won't exist, but the -include directive silently ignores this because of the - prefix, and it will be built anyway, because its corresponding .o doesn't exist either. Subsequent builds will use the dependency information in the .d.

Also consider adding -MP to CFLAGS to prevent errors if you delete a .h file.

'include' being one of those features that isn't in basic POSIX make, though.

Reply on news.ycombinator.com