A Love Letter to Make
kmaasrud.com
kmaasrud.com
That's almost never exclusively what I want to do.
I want to chainload a dependency manager, I want to download, compile, and link those dependencies. I want to run a code generator and integrate that generated code into the IDE's intellisense without ever letting that generated code out of the build tree. I want to generate a compile_commands.json to communicate to my IDE the relationships between the in-source libraries, executables, dependencies, and generated code.
For most build systems it is sufficient to describe the relationships between these components and the rest you get for free. make would require manually specifying every single command, every single tool, every argument. Miserable.
Npx is just for us lazy command-line users who can't be bothered to make sure a Node package is installed before trying to run a command, right? Make would either have to run npx itself, or you'd have to make sure the package you want to run was installed before running its command, in which case you can skip npx... but that's the same with or without make.
- No tracking of changes to build rules. The build rule of a target should be one of its dependencies.
- No protection from dirty builds. Your sources root is also your output root, and make does not help you keep it clean.
- No options for extending the build to use alternate up-to-date checks (e.g. not mtimes), caches, execution mechanisms, etc.
These all have workarounds, but they require the developer to manually include the workarounds in every build rule.
Only if you learnt to program by reading Microsoft examples. Unix developers do not mix compiled files with source files.
Seriously, it’s very project-dependent, but you’re lucky if your thing of choice that you need to build often supports a separate build directory, let alone enforces it.
$ cd /tmp/build
$ ../bash-5.2.15/configure
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
Beginning configuration for bash-5.2-release for x86_64-pc-linux-gnu
checking for gcc... gcc
... etc ...
$ tree
.
├── builtins
│ └── Makefile
├── config.h
├── config.log
├── config.status
├── doc
│ └── Makefile
├── examples
│ └── loadables
│ ├── Makefile
│ ├── Makefile.inc
│ ├── Makefile.sample
│ └── perl
│ └── Makefile
├── lib
│ ├── glob
│ │ └── Makefile
│ ├── intl
│ │ └── Makefile
│ ├── malloc
│ │ └── Makefile
│ ├── readline
│ │ └── Makefile
│ ├── sh
│ │ └── Makefile
│ ├── termcap
│ │ └── Makefile
│ └── tilde
│ └── Makefile
├── Makefile
├── po
│ ├── Makefile
│ ├── Makefile.in
│ └── POTFILES
├── stamp-h
├── support
│ ├── bashbug.sh
│ ├── bash.pc
│ └── Makefile
└── tests
17 directories, 24 files
(Recursive make, eww!) ;)
Make worked very well for this use case. It easily auto-generated dependencies, ran tests, and supported parallel execution. In a tiny fraction of a second it determined what needed to be done, and only did that.
We created a small collection that we released as open source software. You can get it here: https://github.com/david-a-wheeler/make-booster
I think a lot of hate on make is due to poor use. If your makefile is complex, refactor it. Auto-generate dependencies (it only takes a few lines in GNU make). And don't use recursive make, that way lies madness. I also think GNU make is the wiser tool; POSIX make lacks too much in many cases.
Care to give an example? The manual is quite frustrating to piece this together, when it comes to dynamic dependencies, and god forbid, dynamic targets.
Many years ago, I transplanted the process image saving code from GNU Emacs into GNU Make. I keyed that to a new option in make, --dump, which would save the executable image and quit after reading all the makefile material, instead of executing them. The resulting executable would come up with all the Makefiles already in memory, and so dispatch the build very quickly.
This was on a project where reading the tree of Makefiles took 30 seconds. If you touched some C++ file in the tree, it would take half a minute to begin to dispatch the command to rebuild it and then link. With the dumped image, it was nearly instant.
---
OMG, I just found the code for this. I thought it had been lost for 13 years. Turns out, I have the whole damn SVN database from that folded startup, of the Linux distro and toolchains stuff.
I only wish for something which works like Make but has a more ergonomic syntax and an ability to define your own functions to determine if a target is up-to-date.
And for the love of everything that’s good, stop pigeonholing make into being a tool for compiling C-shaped programs. It’s useful for more than that.
Absolutely. When you need to transform one set of files into another and have the parallelism taken care of and the cleanup able to be expressed as a target, it's a great tool. One of many one uses on the jobsite every day.
Yes! A modular make-like would allow the tool to fit any scale. Imagine not only being able to define the up-to-date function, but being able to plug into remote build caches, and perform remote execution of build rules. There are already tools like this, such as Bazel, but they are less appropriate for the small scale than Make. The closest that I know of is BuildKit and related projects.
There are “alternatives” like some Python frameworks, but they are more “command runners” rather than graph resolvers, they lack inbuilt or simple ways to determine that a target is up to date, or they promise paralellism* (with fine print that maybe someday in a future version).
One thing make is lacking (arcane stuff aside) is dynamic dependency graph, which could have made it viable for things like javascript transpiling and bundling. Even for C projects which are sorta kinda match made in heaven, you need a “makedepend” tool to traverse your sources.
Run
I now use `xmake` for building (found love at last) and basic scripts or `just` for command files
> There are so many footguns.
That's true. That sorta comes with flexibility and power.
> `make` experts always tell you that your makefile is wrong even when it does what it is intended to
I wouldn't do this. "If it's stupid and works, it's not stupid."
> Correct handling of header files needs files generated by the compiler the alternative being recompiling everything when you change one header file.
I'm not sure I understand what you mean here.
Never used Xmake, looks nice and clean - the Zig way seems promising as well…
It has revolutionised how I build my projects, particularly with monorepos and is a massive productivity booster when compared to manual task running.
(in which case, why not just use a bash script to begin with?)
Make is terrible. It's just widespread, not very hard to start with and better than all the alternatives (which often are better, but not as easily found or preinstalled)
But it is obtuse for no good reason other than being a piece of software born in a long-gone era of computing. Whoever loves Makefile has never had to debug the compilation process of any non-trivial build system, with three dozen ways to define variables, rules and targets, spread over multiple files.
When in doubt, I'll create a Makefile (actually, I use `just` for my personal projects because 99% of the time in 2023 one just needs a task runner). There is much better software to write love letters about.
Whoever hates Makefile has never been forced to use ant.
Compilation is the use-case that helped make to gain traction. Make has long become outdated for the specific use-case, but it otherwise remains a fantastic general-purpose automation tool.
It is not though. It's a barely passable automation tool that's just slightly better than bash as a tool runner.
...can you recommend a static analyzer and linter for Makefiles?
I kinda get the feeling they're mainly complaining about autogenerated Makefiles from the output of cmake and similar...
If that's the case, then I'll join the chorus complaining about it. Autogenerated makefiles tend to be nasty.
When maintaining recipes for building packages, i'd always prefer to debug a Makefile where i can insert a `sh -x` than some magic tools that attempts to do everything for me and then fails without observable reason.
Ok, but then what about autoconf/autoreconf, aclocal, and automake used to generate makefiles for what seems like the entire GNU ecosystem?
At least the "magic" tools that are more modern have documentation. You can print debug a cmake build pretty dang easily, and it at least doesn't pull in the entire user environment implicitly.
I thought we were talking about make, though. Autoconf and the ilk is an entirely different conversation.
I dunno. I've done that many times over the years, and I still love make. It certainly has its flaws, but it also has benefits. For me, the cost/benefit of make is better than any other such system that I've tried.
I personally reach for Makefiles in exactly one situation: in research, when I’d like to collect lots of results from many computational experiments to make some plots or tables. I might have the whole process of re-running the experiments and rebuilding the final manuscript driven by Make. But even for this task, it feels pretty awkward and painful. I did try using Scons for the same purpose but didn’t like it as much.
CFLAGS = -g -O2
all: foo
foo: ; # how to build foo
install: install-foo
install-foo: ; # how to install foo
clean: clean-foo
clean-foo: ; # how to clean foo
all: bar
# etc.
The ability to override variables from the command line is also pretty sweet: e.g. with the above you can get a non-optimized build with `make CFLAGS=-O0 clean all`. But I guess by using ${CFLAGS=...} you could get `CFLAGS=-O0 ./build.sh` to work as well.
main() {
if [[ $1 ]]; then
cmd_"$@"
return
fi
cmd_all "$@"
}
cmd_all() {
cmd_something "$@"
cmd_some-other-thing
}
cmd_something() {
...
}
cmd_some-other-thing() {
...
}
main "$@"
At work, on the other hand, every project uses something like this in the Makefile:
-include $(shell 'pwd')/makefile-modules/*/*Makefile*
Was very recently looking for an alternative for automating frequently used commands for our Elixir app – especially for local dev (without using docker) and came across Taskfile (https://taskfile.dev/) and have been liking it quite a bit.
That works very well for compiling C. But make is often abused for things that don't match that pattern at all, like a Python project when there are no intermediate targets and they are all .PHONY.
The only downside is that the last part requires a small addition, the install doesn't change the virtualenv dir's last-modified, so that recipe has to end with "touch $@".
The problem is that the dirty real world of software construction doesn't match this ideal because the tooling is full of OS and compiler and library oddities that require explicit non-declarative interventions.
And that's how we ended up with source trees littered with shell scripts and M4 and autoconf and automake and it was a disaster.
One of the joys of operating in a net-new ecosystem like Rust or Go is having modern build tools built explicitly to avoid all this.
Make filled the "batteries not included" niche for C compilers in the 80s. But the world was too heterogeneous for it to scale nicely for larger, more complicated projects.
I asked on stack-overflow if there was a way to do this. https://stackoverflow.com/q/73816110/3568
(Short version of answers: No.)
One could probably write a "how to speak MSBuild for people who understand Make" page based on https://learn.microsoft.com/en-us/visualstudio/msbuild/msbui... ; basically
Make -> MSBuild
Item : File (e.g. foo.c)
Target : Target (e.g. %.o:%.c)
(environment) variable : Property
tab-indented shell : Task (usually a C# assembly, although there's a generic "run command" one as well)
Free tip for anyone trying to debug MSBuild: https://msbuildlog.com/ . AFAIK there isn't anything quite as good for Make.
I've had so many issues with builds failing when the number of jobs increases, random shit breaking between versions of make, issues with timestamps causing false positives for reconfiguration, and so on - it's really not reliable at all when doing big builds on modern systems where you care about caching, distribution, and parallelism.
I asked ChatGPT to write the CmakeLists.txt for me and after some debugging it worked.
Things every modern language with a deps manager infers completely automatically without having to ever think about it. Sure it's understandable that C++ is still so far behind the times for legacy compatibility reasons, but I don't see why we haven't yet gradually moved towards something less tedious.
A prayer that the result is correct is not included.
I used to curse having to trawl through docs older than myself just to get something working but now I just send the recipe I need to write/edit to ChatGPT and it does it for me.
My computer knows what files I read and write. Use that to calculate what you should do.
Some systems head in this direction — there is “tup”, and a Python program I love called fabricate ( https://github.com/brushtechnology/fabricate ), that records what a program reads, then doesn’t rerun it if no input has changed.
similarly, there's nothing stopping you from autogenerating them in some way that you prefer.
I've used dozens of different build systems over the decades, and while each has their own strengths and weaknesses, make remains my favorite.
I'm inclined to think that people who shiver at its name, either haven't bothered to read the manual and go beyond basic features, or have encountered some pretty shoddy makefiles (made possible by make's power and flexibility in the first place). Or, more likely, both.
SHELL = python
.ONESHELL:
And I'm curious why you say this, as I wouldn't have thought it to necessarily be the case, so I thought perhaps you had some nice insight about the use of python as a make shell that I hadn't considered.
You can tell GNU Make to use a leading character other than tab for recipe lines, like the > symbol or whatever:
.RECIPEPREFIX = >
Make works better than anything else for what it does.
Once your project grows, then sure, other tools is less work, but they still require arcane and obscure knowledge to make thos big things work.