Comment by erlehmann_ | Hacker News Reader

Comment by erlehmann_ | Hacker News Reader

Parent Full thread

I know at least three reasons why makefiles are not working well for common use cases. Many people seem unaware of these cases, even though they seem to come up quite often.

• make(1) has a hard time rebuilding target files when source files have changed. The problem is that users have to declare the dependencies of a target before the build, but due to file search paths, it can be that dependencies are only known after building a target. To know about all dependencies, make(1) would have to build any target at least twice.

• make(1) by default does not rebuild a target when its build rules change. While this seems really weird to me, it seems to be a consequence of having a single makefile. When a build rule is changed, the makefile changes. So should all targets depend on the makefile implicitly? One could argue that they should – but then each change in a makefile would rebuild all targets.

• make(1) can not handle non-existence dependencies. Imagine a file foo.h being searched for by a compiler in directories bar and baz (in that order). If the header file baz/foo.h is found, then bar/foo.h should be considered a non-existence dependency: If at any future point in time it exists, the target should be rebuilt.

These problems are all solved by DJB's redo approach, which I implemented in a few hundred lines of Bourne shell (/bin/sh): http://news.dieweltistgarnichtso.net/bin/redo-sh.html

Those don't seem limitations of make itself, but rather the compiler and linker you use with it...

The compiler may know paths, the linker may know paths, the shell might, make might, or you can supply them.

I've seen and used nested Makefiles before.

These things aren't really make things. If I use tcc, Haskell or Go in my Makefile, then they mightn't apply.

If you can write it in Bash, you can integrate it into your Makefile.

I think that all of these are limitations of not only make(1), but all utilities that expect dependencies to be fully known before a target is built. What makes you think they are not?

How would you solve all three problems I listed using make(1)?

Why should make solve the dependency expectations of what it is building?

If I use pip as part of my buildchain, then it has a series of places it can lookup, or you can supply one. Same with npm, cargo, and even Go.

If instead, I use a tool that has dynamic lookup, then it will only look in those places after the event.

So, if I used make to send a command like:

    $(echo 'require "foo"' > bar.lua)

Then it can lookup those when the lua file is called, later in the process.

So how would this be a limitation of make itself? The limitation only exists in what you call with make... Which could be anything.

A makefile consists of rules. Each rule contains a dependency line which defines a target and an enumeration of prerequisites. This means that the dependencies have to be known before the target is built. By design, it is impossible for a single-pass make(1) invocation to derive dependencies for a C program, as dependencies are output by the compiler.

By contrast, redo builds the target first and then records what was used to build it. For example, when compiling a C file with “gcc -M”, gcc will output dependency information. With redo, you normally record those dependencies after the target has been built. With make(1), that information has to end up in the makefile somehow, possibly leading to further builds.

I mentioned m4 before as a way of using more passes, and is how the Linux kernel approaches this, but looking deeper at make, I'm not even sure you need it.

From the manual, on implicit rules:

foo : foo.o bar.o cc -o foo foo.o bar.o $(CFLAGS) $(LDFLAGS)

Because you mention `foo.o' but do not give a rule for it, make will automatically look for an implicit rule that tells how to update it. This happens whether or not the file `foo.o' currently exists.

I do not understand. I know that make(1) has many implicit rules – but which of the problems I mentioned does this solve and how? Please be specific about your solution.

Reply on news.ycombinator.com