GNU Make in Detail for Beginners | Hacker News Reader

GNU Make in Detail for Beginners

linuxforu.com

154 pointszhiping36 comments

I used to write a column called "Ask Mr. Make" for CM Crossroads. I ended up turning it into a self-published book called "GNU Make Unleashed" and it contains lots of information like this (all the way to very advanced topics).

http://www.lulu.com/shop/john-graham-cumming/gnu-make-unleas...

If you don't like the book format then the original articles (which I cleaned up and improved for the book) are here:

http://blog.jgc.org/2010/01/update-list-of-my-gnu-make-artic...

Also, I created a useful 'standard library' of functions for GNU Make which can be found here: http://gmsl.sf.net/

I've used gmsl in the past. It's good stuff.

A digression but .. CM Crossroads has gone downhill for a while now. Recently it took three or four emails from me for them to remove some spam hacked into their site.

Thank you for a good article. Even I learned some new stuff. However, there was a major omission: automatic dependency generation. Every now and then I forget how it's done and try to find a good source on the net, but they're really scarce. On the other hand, there are tons of small "your first makefile" tutorials (this article is better than the average).

Here's my Makefile boilerplate. It build single executable from all .c files in this directory and scans for include file dependencies to properly cause recompilation when a header file is touched.

  INCS=-I./
  LIBS=-lglfw -lGLEW
  CFLAGS=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
  CFLAGS+=$(INCS)
  CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
  SRCS=$(wildcard *.c)
  OBJS=$(SRCS:.c=.o)
  DEPS=$(SRCS:.c=.d)
  EXECUTABLE=main

  .PHONY: all clean
  all: $(EXECUTABLE)

  $(EXECUTABLE): $(OBJS)
  	$(CC) $(CFLAGS) $(LIBS) -o $@ $^
  %.o: %.c
  	$(CC) $(CFLAGS) -c -o $@ $<
  %.d: %.c
  	$(CC) $(CFLAGS) -MM -o $@ $<

  -include $(DEPS)

  clean:
  	rm -f $(EXECUTABLE)
  	rm -f $(OBJS) $(DEPS)

If anyone has a better one (preferably with out-of-source build to put .d and .o files to a better place), please share it.

I don't know about out-of-source building, but I know Make has a bunch of built-in rules that you can use to cut down your boilerplate.

For example, I'm pretty sure that a filename depending on objects will automatically be created by linking those objects (although you'd have to specify your libraries in LDFLAGS rather than LIBS. Also, your rule for building .o files from .c files would be automatic if you put your include-directives in CPPFLAGS instead of INCS.

You should put something like this:

    ifneq ($(MAKECMDGOALS),clean)
    include $(DEPS)
    endif

So you don't regenerate all the deps if you re-run clean.

You might want to provide separate CFLAGS and LDFLAGS.

Also, I prefer adding -MMD to CFLAGS instead of having an additional rule for dependency generation: This way, you won't unnecessarily trigger dependency generation on targets which do not build objects like clean.

(Sorry, I misread the above post, but I believe LDLIBS is the Makefile built-in variable for libraries to link.)

Actually, in this case, I believe LDLIBS would be more appropriate since the OP isn't passing any flags to `ld` but rather the libraries for `ld` to link. I ended up getting hit be this in a simple Makefile a few weeks ago that worked in Fedora but failed in Ubuntu.

The target was "%.o: %.c", which make auto-expands to be (something like):

    $(CC) $(CFLAGS) $(LDFLAGS) $@ $^ $(LDLIBS)

But I had placed the libraries in LDFLAGS, instead of LDLIBS which caused Ubuntu to fail at finding a function in one of the libraries. Moving the libraries to LDLIBS solved the problem.

I was more concerned about passing CFLAGS when linking - he already has a LIBS variable; you're right that the 'canonical' name for his LIBS variable is LDLIBS (as well as LOADLIBES - anyone an idea where that name came from?).

EDIT: stop editing your answer while I'm writing my own ;)

According to the manual, the built-in rule for linking an executable `n` from a single object file is

    $(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS)

I used this (on an admittedly unfinished and more or less abandoned project) https://github.com/adam-a/Simple-Sampler/blob/master/build/M...

I never found a good way to silo the .o and .d files though, the way I do it is put the main makefile in a separate dir from the tree so all the clutter can accumulate there.

Here's a makefile: http://code.google.com/p/min-game/source/browse/makefile

Here's the dependency script: http://code.google.com/p/min-game/source/browse/dep.sh

Both inspired by code in "Recursive Make Considered Harmful", always worth a read: http://aegis.sourceforge.net/auug97.pdf

The built-in rules and variables for gnu make are worthless for anything but the most trivial project; don't bother with them. I'd be so happy if there was a makefile directive that forced make to run in "make -rR" mode.

A quick note:

> INCS=-I./

This is generally a bad idea, unless you're explicitly trying to override #include <stdio.h> with a local file (and there are better ways to do that). <> are for system files, "" are for local files.

So off the bat, your makefile example is better than most makefiles out there. I still see things that look like some dumb IDE wrote them with explicit .d and x.c: x.o rules littered about.

Though I do note you aren't taking advantage of the built-in makefile rules (which are defined in POSIX), and you aren't taking advantage of gcc's -MMD flag, which does the dependencies during compilation instead of a separate step. That eliminates much of the file:

  LDLIBS=-lglfw -lGLEW
  CFLAGS+=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
  CFLAGS+=-MMD
  CFLAGS+=$(INCS)
  CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
  SRCS=$(wildcard *.c)
  OBJS=$(SRCS:.c=.o)
  EXECUTABLE=main

  .PHONY: all clean
  all: $(EXECUTABLE)

  $(EXECUTABLE): $(OBJS)
  	$(CC) $(CFLAGS) $(LDLIBS) -o $@ $^

  -include *.d

  clean:
  	rm -f $(EXECUTABLE)
  	rm -f $(OBJS) *.d

Furthermore, I like to name my objects explicitly instead of using * .c, which lets you order them on the command line, if you find that makes a difference. In particular, if your executable is named the same as one of your C source files then you don't even need the link line, as long as the C source file is the first dependency:

  LDLIBS=-lglfw -lGLEW
  CFLAGS+=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
  CFLAGS+=-MMD
  CFLAGS+=$(INCS)
  CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
  OBJS=main.o obj1.o obj2.o

  .PHONY: all clean
  all: main

  main: $(OBJS)

  -include *.d

  clean:
  	rm -f $(EXECUTABLE)
  	rm -f $(OBJS) *.d

I tend to like to prefix variables with the thing you are making so that different executables in the same Makefile can have (drastically) different compilation parameters:

  .PHONY: all clean
  all: main

  main: LDLIBS=-lglfw -lGLEW
  main: CFLAGS+=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
  main: CFLAGS+=-MMD
  main: CFLAGS+=$(INCS)
  main: CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
  main: OBJS=main.o obj1.o obj2.o

  main: $(OBJS)

  -include *.d

  clean:
  	rm -f $(EXECUTABLE)
  	rm -f $(OBJS) *.d

If you have an embedded project that needs both compiling and cross-compiling that trick can work wonders.

Fair warning: GNU make cannot handle directory names or file names which contain spaces. This is a show-stopper defect which has been present since the day it was written and shows no signs of ever being fixed.

Why would you have a space in a directory/file name though? I wouldn't call this a show-stopper.

Program Files

My Documents

Also, why wouldn't I? Why shouldn't I? There isn't a single file system that prevents me from doing so and there isn't a single other computer program in the world which has a problem with it.

I understand you are making a point, one which I largely agree with, but:

there isn't a single other computer program in the world which has a problem with it

is a bit too much hyperbole -- there is some really really bad software out there.

That's a fair point. Strawberry Perl, I'm looking at you.

I stand by the rest of what I said, though. If a piece of software can't handle spaces in paths, the software is broken. I'm not adding a top-level directory to my C: drive just to appease it. I'll use something else.

Make was made for *nix, it's not exactly surprising that it doesn't conform with Windows's preferences :-) I certainly wouldn't call that show-stopping.

Oh and there are some file systems that don't allow spaces by the way, though admittedly not in common usage.

Sigh, yet another article giving advice about how to use make as a kind of general-purpose language. It's not. Forget all the crap with wildcards, conditionals, filename globbing, etc. and use it for what it is: a program that takes as input a DAG and a timestamp for each node, and executes rules when predecessor(s) of some node are newer than the node itself.

Take make for what it is and use a proper programming language to handle Makefile generation, installation, etc.

  c:\>copy con m.bat
  cl foo.c bar.c baz.c boz.c
  ^Z

I really appreciate that this article talks about using Make in contexts other than just compiling software. I've used it in the past to automate bioinformatics analysis pipelines. Unfortunately, I ultimately found that it did not really have the flexibility that I needed for more complex analyses, because of various limitations such as the inability to handle multiple output files from a single rule. I also couldn't figure out how to do automatic dependency generation in the general case (not just for C files).

I figure you already solved that problem, but maybe http://gittup.org/tup/ is general enough for your use case?

No, I haven't solved that problem. tup looks really useful. Thanks for the suggestion!

It's worth highlighting here how Tup differs fundamentally from Make in that you specify the DAG from the bottom-up, rather than the top-down as in Make.

That is to say, with the top-down scheme used by Make you think about and write things down in this kind of order:

  1. executable <- object files
  2. object files <- source files

Whereas in Tup you think and write bottom-up, like this:

  1. source files -> object files
  2. object files -> executable

... which is more akin to how you would write a simple shell script for your build, with all of the commands in the order that they need to be executed. But you still get the time saving of files not being rebuilt if their sources haven't changed, which is the point of using a build system over a shell script in the first place.

To put it another way, Make is declarative like a functional language, whereas Tup more like an imperative language. If you're like me, building is a chore and a build system is supposed to make my life easier, so if it's very much more difficult than a shell script then I can't be bothered with it. Like most programmers I'm more at home with imperative languages than functional; and looking at other people's Makefiles tends to do my head in.

Having said that while Tup has less cruft and some better features (eg. multiple output files as per the parent poster's wish) than Make, it's still a domain-specific language and as such its overall features are limited to what its creator decided were sufficient to his definition of "building programs" for the sake of terser syntax. So, if you're like me you may prefer to hurry on to the 'ultimate' step which is to just use a general-purpose language with a helper library for running build commands. I currently use fabricate.py (http://code.google.com/p/fabricate/) which provides a 'run("shell command")' function to a Python script that runs the command while hooking system calls to spy on the command's inputs and outputs, then saves the command-line and the input/output file names to a file ('.deps') so next time it can check whether the command needs to be re-run or not.

Of course in a general-purpose language such as Python you can do just about anything you like and don't have problems with silly little things like spaces in strings (though see below). While the resulting script may not be as short as the 'ideal' Makefile or Tupfile which could be nearly empty with all their implicit rules taken into account, I'm of the view that explicit is better than implicit.

Couple of specific fabricate.py caveats - being on Windows, I had to do some hackery on my copy of the library to make the dependency detection work better, and to allow for supplying the dependency filenames manually as a fallback. And, the documentation of that 'run()' function could be better (in short, you shouldn't just supply a full string as you would type it at the command line; you should convert those space-seperated command-line arguments into comma-seperated Python arguments, and Fabricate will automatically quote any argument that contains a space. I have no connection with the project but I should really see if I can get into their wiki there and add a little doc laying out the subtleties of it; it does work out in the end). That said, I couldn't be much happier now; it works like a champ everywhere I've tried it including C++, Java, ActionScript and Haxe.

The other downside of bottom-up/imperative style is that it doesn't lend itself automatically to parallel builds (cf. an oft-noted difference between imperative and functional languages in general). I hear it's possible if you plan for it (http://code.google.com/p/fabricate/wiki/ParallelBuilding) though personally my projects have not been big enough to bother trying this so far.

Thanks for another interesting suggestion. Reading the fabricate "how it works" page, I see that you can subclass it to implement your own dependency-determination code, which is cool.

A rule for multiple output files:

    output1 output2: deps

Yes, you can do that, but it doesn't actually work the way it should. I don't remember the details any more, but it breaks in nonintuitive ways.

Great article! I could have only wished for such fine introductory material when I was starting off with Make. It would be nice if the docs themselves had a similar chapter.

One thing that I've always wanted from Make was a way of writing long commands with less escaping and without continuation characters. Something like heredoc for Make.

For example, the following is just too arcane and I would by all means move the body of the command into a separate file:

    compress:
      find -iname '*.html' -print0 | \
      while IFS="" read -r -d "" fn; do \
        old=$$(stat -c "%s" "$$fn"); \
        java -jar htmlcompressor.jar "$$fn" -o "$$fn"; \
        new=$$(stat -c "%s" "$$fn"); \
        delta=$$(echo "scale=2 ; 100 - ($$new/$$old) * 100" | bc); \
        echo "compressed $${fn}: $${old}/$${new} $${delta}%"; \
      done;

Assuming there is no interpolation between Make and Bash, why not provide a way of writing the command as naturally as possible:

    compress:
      %cmd -flags -that -control -escaping << EOF
      find -iname '*.html' -print0 | while IFS="" read -r -d "" fn; do
        old=$(stat -c "%s" "${fn}")

        java -jar htmlcompressor.jar "${fn}" -o "${fn}"
        new=$(stat -c "%s" "${fn}")

        delta=$(echo "scale=2 ; 100 - (${new}/${old}) * 100" | bc)
        echo "compressed ${fn}: ${old}/${new} ${delta}%"
      done
      EOF

Of course, shell and Make have a lot of conflicting syntax that would make this into a hard problem.

This paper provides really valuable insights about Make's /modus operandi/ and desired usage:

http://miller.emu.id.au/pmiller/books/rmch/

I used to be a pretty good GNU Make 'expert'. I must admit that I'm glad that I've been lucky enough to be able to purge this arcane and esoteric knowledge from my head. Long live .PHONY!

Is recursive make not considered harmful, i.e. best practice is to instead include makefiles from subdirectories ( http://aegis.sourceforge.net/auug97.pdf )?

Including makefiles from subdirectories is not recursive make. Recursive make is reinvoking make on a subdirectory.

Ehhh, that's what I said? In the article he takes about reinvoking make using $(MAKE), having mentioned changing to a subdirectory in the previous paragraph.

It would be nice to Make tutorials to mention the simplest possible Makefile ... none at all.

1. Create something.c

2. make something

Look, no Makefile!

What about when?

"Large projects can contain thousands of lines of code, distributed in multiple source files, written by many developers and arranged in several subdirectories."

Assuming you have GNU Make installed, go to your terminal and run `info make'. Quite a read but comprehensive.

Reply on news.ycombinator.com