The Unreasonable Effectiveness of Makefiles
matt-rickard.com
matt-rickard.com
Most of the targets in the Makefile had a command to kick off the job and wait for it to finish (this was accomplished with a Python script since kicking off a job involved telling another application to run the job) followed by a `touch $@` so that make would know which jobs it had successfully run. If a process had dependencies these were declared as you'd expect.
The other targets in the Makefile lashed those together into groups of processes, all the way up to individual days and times. So "monday-9pm" might run "daily-batch", "daily-batch" would have "daily-batch-part-1" (etc), and each "daily-batch-part-..." would list individual jobs.
It was awful. It still is awful because it works so well that there's been no need to replace it. I keep having dreams of replacing it, but like they say there's nothing more permanent than a temporary solution.
All of this was inspired by someone who replaced the rc scripts in their init system with a Makefile in order to allow processes to start in parallel while keeping the dependencies in the right order.
> Airflow was started in October 2014 by Maxime Beauchemin at Airbnb. It was open source from the very first commit and officially brought under the Airbnb GitHub and announced in June 2015.
I believe I starting building my tool somewhere around 2010, possibly 2011. The core mechanism has been completely unchanged in that time. If Airflow was a thing at the time, I'd have hopefully looked into it. I looked at a handful of similar products and didn't find anything that was a good fit.
Based on a really quick skim of the Airflow docs it seems like it checks all of the boxes. Off the top of my head:
* LocalExecutor (with some degree of parallelism, assuming the dependencies are all declared properly) seems to do exactly what I want.
* I could write an Operator to handle the interaction with the system where the processes actually run. The existing Python script that does this interaction can probably get me 90% of the way there. Due to the nature of what I'm running, any job scheduler will have to tell the target system to do a thing then poll it to wait for the thing to be done. To do this without any custom code, I could just use BashOperator to call my existing script.
* It's written in Python, so the barrier to entry (for me) is fairly low.
* Converting the existing Makefile to an Airflow DAG is likely something that can be done automatically. We deliberately keep the Makefile very consistent, so a conversion program can take advantage of that.
I think my dream of replacing this might have new life!
* There's no reasonable way to do cross-batch dependencies (e.g., if process X in batch A fails, don't run process Y in batch B). I've got a few ideas on how I could add this in, but nothing has been implemented yet.
* There's no easy way to visualize what's going on. Airflow has a Gantt view that looks very useful for this purpose, our business users would absolutely LOVE the task duration graph, and the visualization of the DAG looks really helpful too.
* Continuing a failed batch is a very manual process.
None of these are showstoppers because, as you said, this has been running fine for over a decade. These are mostly quality-of-life improvements.
Sometimes the most interesting thing is not the story itself, but the story behind the story.
This has my interest peaked. Is there anywhere else I can read about this?
peak comes from Old English pīc, meaning just "peak" (e.g. of a mountain).
The two are completely different words that just sound similar.
pique may possibly go back to Proto-Germanic, and peak does, but the two go back to two separate words (*pīkaz, *pikkāre) though both are then related to sharp things and possibly onomatopoeic.
https://www.gnu.org/software/make/manual/html_node/Job-Slots...
Basically you can specify a “-j 24” (e.g.) option to make, and it will run as many as 24 build steps in parallel. If your Makefile is correct, that’s all you need.
Because make knows the dependency graph, it can correctly handle cases where some build steps have to be done serially, while others can be fully parallelized. E.g.,
a: b;
b: c;
x: y;
x: z;
It’s quite a powerful capability and it feels great to see 24 build-steps start in parallel from one simple command.
I didn't pursue this very far as there were other problems with doing this, but I'd like to pursue it again. The problems are all with the target system, `make` does the parallel execution work perfectly.
https://web.archive.org/web/20110606144530/http://www.ibm.co...
There’s a reference to some sample Makefiles to start and stop some Linux services in parallel. It’s obviously not complete, but this (or something similar) was what inspired my system.
Any sufficiently complicated init system contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of systemd.
Heh. The text that follows this sentence is likely the most beautiful and elegant use of a Makefile ever.
I love the humble bragging of this site.
Being able to drop into any repo at work and expect that `make init`, `make test` and `make start` will by convention always work no matter what the underlying language or technology is, has saved me a lot of time.
In the project repo, either work.
#!/bin/sh
case $1 in
init)
... do whatever for each project init
;;
start)
... do whatever for each project start
;;
test)
... do whatever for each project tests
;;
*)
echo "Usage: $0 init|start|test" >&2
exit 1
;;
esac
The advantage could be, maybe, no need to install make if not present, and no need to learn make stuff if you don't know it.
Sometimes they don't match the usual words (deploy, start, stop, etc) but then I know that if I don't remember them, I just type ./runme and get the help.
For my scenario, it's perfect because of it's simplicity.
I've never rated Make for building C programs, but it's pretty good as a convenient cross-platform shell-agnostic task runner. There are also several minimal-dependency builds for Windows, that mean you can just add the exe to your repo and forget about it.
But it is still a great task runner.
task1() {
echo hello
}
task2() {
task1()
echo world
}
"$@"Meanwhile, in Make that's
task1:
echo hello
task2: task1
echo worldI've used in in the past with python/django roughly like so (untested from memory, there may be a "last modified" bug in here that still makes something run unnecessarily):
.PHONY: runserver
environ:
python -m venv $@
environ/lib: requirements | environ
. environ/bin/activate && pip install -r $<
touch $@
runserver: | environ/lib
. environ/bin/activate && python manage.py runserver [foo]
It might not be necessary for this example, but I've found being more liberal with this feature and manually using "touch" has been more reliable in stopping unnecessary re-runs when the recipe target and dependency are directories instead of just files.
I don't think Makefiles are a bad way to go but a bash script is likely more accessible and easily reasoned about in most places.
The above won't.
For many junior colleagues, this pattern is the first time they've ever encountered make -- hijacked as some kind of imperative command runner.
It's quite rare to run into someone who is aware that make can be used to define rules for producing files from other files.
I find it all a bit odd. Of course, no-one is born knowing about middle-aged build tools.
Is it, though?
That's literally what Make does as part of its happy path.
GNU Make even added support for pattern rules, as this use case is so pervasive.
What do you think people think make is about?
i'm talking working on projects with people whose first encounter with make is in a project where someone else has defined a Makefile to wrap imperative actions, e.g. `make run-unit-tests`, `make deploy`. If they think about make at all, there's a good chance they think make is for performing imperative actions, and has nothing specifically to do with producing files from other files using rules and a dependency graph, or the idea of a target being a file, or a target being out of date.
That command (to run an Angular/nodejs dev instance has staved off carpel-tunnel syndrome for me for maybe another 5 years.
The benefit of this is it's just shell scripting so you can use shell features like $@ to pass args to another command and everything else the shell has to offer.
I've written about this process at https://nickjanetakis.com/blog/replacing-make-with-a-shell-s... and an example file is here https://github.com/nickjj/docker-flask-example/blob/main/run.
If I had to pick one nit, and it’s a stylistic choice, you use braces around variable names where they aren’t strictly needed.
I also like to add “set -u”.
My thought process around using braces when they're not needed is mainly around consistency. If you pick and choose when to add them then you need to make a decision every time you add a variable. I've written about that here: https://nickjanetakis.com/blog/why-you-should-put-braces-aro...
That is a good call about `set -u`, it's something I've been using more recently but I haven't added it into that script yet but thanks for the reminder, I will soon. I ended up making a post about that here: https://nickjanetakis.com/blog/prevent-unset-variables-in-yo...
Another small thing I've been doing recently is defining options like this:
set -o errexit
set -o pipefail
set -o nounset
foo=$bar # quoting not needed but I'd still do this:
foo="$bar"
${TTY}
dc_exec=(docker-compose "${DC:-exec}")
dc_run=(docker-compose run)
if [[ ! -t ]]; then
# we have no TTY so we're probably running under CI
# which means we need to disable TTY docker allocation
dc_exec=("${dc_exec[@]}" --no-TTY)
dc_run=("${dc_run[@]}" --no-TTY)
fi
_dc() {
"${dc_exec[@]}" "$@"
}
_build_run_down() {
docker-compose build
"${dc_run[@]}" "$@"
docker-compose down
}
cmd=(
/usr/bin/some-command
--option1="foo"
--option2="bar"
--option3="baz"
)
"${cmd[@]}"
Another thing that goes along with set -u is to fail early. So for example, your script seems to require global POSTGRES_USER, etc, so why not:
set -o nounset
# fail early on required variables
: "${POSTGRES_USER}"
: "${POSTGRES_PASSWORD}"
The postgres variables are sourced in the line above I use them, they will always be set. It's a hard requirement of the Docker postgres image. I did end up pushing up the nounset change and it didn't complain about it since it's set from being sourced.
Aside, but I gotta say, lots of good stuff here:
https://nickjanetakis.com/blog/
I'm not personally a fan of videos, but I have plenty of collegeues who are that I'm going to happily start pointing at your videos. Some of these will be very handy references I can add to code reviews.
OK, maybe I have 3 [I mean 4] reasons. 3) if you always put the braces in, it won't break your script when they aren't required. However, if you don't put the braces in when they are required. it will break your script. 4) often putting the braces in when they are not required makes the script easier for me to read. I often use spacing that is not required for the same reason.
I'm not saying I never break my own "rules" (they are really more guidelines than rules). You will find variable names used in my shell scripts that have no surrounding braces, but I probably use more of the "unnecessary" ones than a lot of people do. And yes, I'm aware that sometimes not having them makes me less consistent. Everything has a balance, people just differ on what style provides the balance they prefer.
If most of your make usage is a bunch of .PHONY nonsense and tricks to make it so developers can run a simple command to get going, check out just. You will find it's not difficult to immediately switch over to its task format.
I've thought about trying it out a few times but can never see its value over scripts in ./bin.
Could your write a bash script that does stuff like enumerate all the bin scripts, pull out documentation comments, etc.? Absolutely, and people have followed that pattern for a while (see https://github.com/qrush/sub) but it's a bunch of boilerplate to copy between projects. Just pulls out that logic into a simpler config file.
tic-tac-toe: tic.o tac.o toe.o
cc -o '$@' $^
%.o: %.c; cc -c $^https://github.com/TekWizely/run
It's better a managing and invoking tasks and generates help text from comments.
I'll have to poke around this a bit more later, thanks for the link!
Even if `just` was installed on a standard Linux box, I don't see the benefit of it over a bash script.
If bash works for you, stick with it. It does not work with large teams and people who aren't deeply experienced in all the footguns of bash in my experience.
The advice on output sentinel files for rules creating multiple files helps keep rebuilding dependencies reliable. Avoiding most of the cryptic make variables also helps Makefiles to remain easily understandable when you're not frequently working on them. And using .ONESHELL to allow multi-line statements (e.g. loops, conditional etc) is great. No need to contort things into one line. or escape line breaks.
Seems like you could even use a more serious programming language instead of sh/bash by setting SHELL to Python or similar. That may be a road to madness though...
# Makefile
SHELL := bash
.ONESHELL:
.RECIPEPREFIX = >
thing1:
> FOO=bar
> echo "$${FOO:?}"
.PHONY: thing1
thing2:
> echo "$${FOO:?}"
.PHONY: thing2
$ make thing1 thing2
FOO=bar
echo "${FOO:?}"
bar
echo "${FOO:?}"
bash: line 1: FOO: parameter null or not set
Edit: Maybe I misinterpreted, do you mean you'd want to choose whether a given target is ONESHELL or not?
Yep, exactly! I would like to have some targets use ONESHELL and others in the same Makefile not use ONESHELL. So that I can choose the most appropriate for each target.
So far I have managed by avoiding ONESHELL and doing the typical "backslash, next line continues" thingy. But it puts some people off.
TIL.
SHELL=/usr/bin/python
.ONESHELL:
all:
@from plumbum.cmd import ls
print(ls["-a"]())
My response to this article would be, if make is so great why did they have to invent 'configure' and 'xmkmf'? And why do people continue to create new build tools every couple of years?
Yeah, I mean I guess it worked, but unreasonably effective? Hardly.
Cross-architecture and linux distro compatibility, mostly.
And even then, it handled even some non unix environments as well.
Is there some older implementation/standard/practice of ./configure before autoconf?
Seems like a rite[0] of passage to some degree. Perhaps similar to people talking a stab at The Next Actually Correct CMS, and The Next Object System That Doesn’t Suck, or The Next Linux Distro For Smart People.
[0] edit: corrected “right V. rite” per https://news.ycombinator.com/item?id=32442473
We developed a lot of Python scripts. To manage them I created some helper tools to integrate them via make. People are welcome to reuse it, I got it released it as open source software. I named it make-booster: https://github.com/david-a-wheeler/make-booster
From its readme:
"This project (contained in this directory and below) provides utility routines intended to greatly simplify data processing (particularly a data pipeline) using GNU make. It includes some mechanisms specifically to help Python, as well as general-purpose mechanisms that can be useful in any system. In particular, it helps reliably reproduce results, and it automatically determines what needs to run and runs only that (producing a significant speedup in most cases)."
"For example, imagine that Python file BBB.py says include CC, and file CC.py reads from file F.txt (and CC.py declares its INPUTS= as described below). Now if you modify file F.txt or CC.py, any rule that runs BBB.py will automatically be re-run in the correct order when you use make, even if you didn't directly edit BBB.py."
This is NOT functionality directly provided by Python, and the overhead with >1000 files was 0.07seconds which we could live with :-).
Make provides a way to handle dependencies as DAGs. Using it at scale requires that you call or write mechanisms to provide that DAG dependency info, but any such tool needs that info. Some compilers already come with generators for make, and in those cases it's especially convenient to use make.
In this pipeline use case, you have base data, and a series of transformations that massage it into usable results. You are always revising the pipeline, usually at the output end (but not always) so you want to skip as many preprocessing steps as possible. Make automates all that.
This works great for image processing pipelines, science data pipelines, and physical simulators for a few examples.
There have been a few blog posts and ensuing HN discussions about this use pattern for make. The discussion generally gets mixed up between make’s use as a build system for code, alas.
When you're integrating many different data sources with a complicated set of scripts, it's important to automate what you can. The easy but impractical thing to do is rerun everything. Make, properly used, will rerun everything that needs to be run in the correct order... and nothing else. GNU make is also awesome for running things in parallel.
https://www.factual.com/blog/introducing-drake-a-kind-of-mak...
I don't see any evidence that it handles refunding of steps when transitive depended on code is changed, though. E.g., if script BBB.py includes CC, and CC is changed, then all steps transitively depending on BBB should be rerun. My make-booster specifically deals with that case.
I also expect drake to have a slow start, which slows development.
I hate building system that don't use Makefile, or that use but don't respect the variable convention. It makes really quite annoying to do things like changing allocation library, add compilers flags, etc.
You don't need to use tab.
.RECIPEPREFIX=>
foo:
>echo barAutotools uses Make as the backend, but so do a lot of other things too. You don't need Autotools to use Make.
"Make originated with a visit from Steve Johnson (author of yacc, etc.), storming into my office, cursing the Fates that had caused him to waste a morning debugging a correct program (bug had been fixed, file hadn't been compiled, cc *.o was therefore unaffected). As I had spent a part of the previous evening coping with the same disaster on a project I was working on, the idea of a tool to solve it came up. It began with an elaborate idea of a dependency analyzer, boiled down to something much simpler, and turned into Make that weekend."
If they had rebuilt the whole project, make would not have been needed, However because they wanted to do partial builds and the manual process had downsides make was invented.
And all that just for executing a set of tasks in a DAG much slower than ninja
…until it is.
I usually reach for make because it’s simple to get a simple project built but once I start throwing stuff like source code generators at it then I have to spend a bunch of time getting the dependency tree right so I’m not chasing bugs I’ve already fixed but didn’t get recompiled. Or recompile the whole thing when I change some trivial thing.
Still, for the stuff I do, it gets the job done without too much trouble.
In case someone want to see all the defaults, run `make -p`.
I found Earthly[0] to be a great replacement. Everything runs in Docker. Your builds are reproducible, cache-able, and parallelizable by default. I've heard Dagger[1] is another good tool in the space.
[0]: https://earthly.dev/
[1]: https://dagger.io/
https://scottmcpeak.com/autodepend/autodepend.html
And this, avoid recursive make:
These days, I would absolutely not use Make to compile code written in C, except for the smallest personal projects. It is just too fussy to construct the Makefile correctly, in a way that you can get correct incremental builds. Nearly any other build system is better at building projects written in C, in the sense that it is easy & straightforward to get your build system to do correct incremental builds.
I've found that to not matter that much these days - unless your projects is hundreds of thousands (or maybe millions even) of LOC, full builds are instant on modern machines.
- Most C++ or Rust projects
- Medium size or larger C projects
- Anything which is built with tools written in JavaScript (due to Node startup time overhead)
Stuff where a full build is close enough to instant:
- Most Java, C#, Go projects
- Small C projects
- Tiny/trivial C++ or Rust programs, or C++ programs written for embedded systems
This is just my experience. YMMV.
Yes, exactly. Incremental compilation! This is what Make is not good at.
For literally everything else, I found myself using it more as a task runner - and Make doesn't do a great job at it. You end up mixing Bash and Make variables, string interpolation, and it becomes really messy, really fast. Not to mention the footguns associated with Make.
I found bake (https://github.com/hyperupcall/bake) to suit my needs (disclaimer: I wrote it). It's literally just a Bash script with all the boilerplate taken care of you - what a task runner is meant to be imo
I like it very much.
When I worked with latex more, I kept a ~/.Makefile-latex and a shell function that would pretty much just do
inotifywait -e modify -e move --recursive --monitor . | while read; do make --makefile ~/.Makefile-latex; done
edit: I used this complicated setup instead of LyX or whatever other "(la)tex IDE" because I had ancillary files like xfig diagrams that would get compiled to .eps files and gnuplot scripts that would render data into graphs, and the makefile knew how to generate everything.
latexmk -pvc -pdf foo.tex
I usually add something like this command as the `auto` target in my latex Makefiles, which works pretty nicely.
I never really understood most of the TeX world. I cargo-culted the bits I needed to make:
* articles for journal submissions (easy, they all provided packages or templates),
* my thesis (somebody had written a latex class that took care of all the school's formatting requirements a ~decade prior), and
* a resume,
and never dug into the differences between tex, latex, miktex, texlive, auctex, lyx, pdftex, xetex, luatex etc etc etc etc etc.
I was just to ask you if you knew of a 40000-foot overview of these kinds of things but stumbled across this article over at Overleaf: https://www.overleaf.com/learn/latex/Articles/What%27s_in_a_... , it looks pretty apropos.
But. I'm kind of proud. My shell script monitors the tex files for character changes and then once a (configurable) threshold of changes is met, it kicks off the compilation process. But the real game changer is that every time it compiles, if the compile is successful it commits the recent edits to a git branch. Then if I want, I can go through the git branch and see the entire history of edits for only versions of the document that compiled. It's a game changer in a big way. When I finish a section, I squash the minor edits into one commit and give it a good message and then commit the full thing to the main branch. Then there is where I can make sure my manuscripts look the way they should and do major revisions or collaborative edits.
The icing on the cake is that the fish script monitors for a special keypress and will take actions. So I hit "c" to compile the tex, "b" to rerun the massive bibliography compilations, "s" to squash the commits into the main branch, and "l" to display log errors. It's a dream! Now I don't think about compilation at all, or when I should commit something minor to git (and fiddle with the commands). I just type away and watch/request the pdf refresh when I need it... and _actually_ get work done. My god. So happy.
I just finished this today.
Tying together multiple projects, source from different locations, etc Id probably use make or a script.
Seems like one version out of 2 of windows being trouble stills stand.
make test
make format
make clean
make docker-stack
Quickly ran into some massive limitations - one of which is that it completely broke apart when filenames had spaces in them.
"But why would you do that you're doing it wrong" - don't care wanted spaces in filenames.
Ended up switching Rake (a make like DSL written in ruby) and never looked back. Not only can you do all the make declarative stuff, but you get the full power of ruby to mess around with strings.
And the feature bloat slippery slope is sliding towards Bazel!
If I had to use some barely POSIX conforming thing from BSD thing or wherever, I'd instead write a write a top-to-bottom linear shell script full of conditionals.
It makes sense if you ship both a GNUmakefile and a Makefile.
In general, for most people, make is GNU make. And GNU make is actually pretty decent at a lot of tasks.
Many ridiculous makefile problems I see stem from not using it well. I suggest:
* Use compiler dependency generators (generate .d files and "include" them via make). This eliminates many lines and errors.
* Don't use recursive makefiles.
* Set macros with := or ::=
* Use substitutions so definitions can be changed in just one place and the rest automatically works.
There is no perfect tool for all circumstances. But GNU make is still useful for many circumstances.
The last time I saw a lower case 'm' makefile was eons ago.
The manual itself recommends Makefile rather than makefile, right in the same paragraphs where it tells you makefile is checked first.
Why look for makefile before Makefile, a file next to nobody has.
Maybe lower-case makefile was a predecessor to GNUmakefile; a way of having a GNU-specific makefile. (If so, that's a noteworthy thing the manual ought to mention.)
https://github.com/TekWizely/run
It feels like a makefile but is optimized managing and invoking small tasks and wrappers, and auto-generates help text from comments.
A raw Ninja file is verbose but easy to read and understand, and there is basically no magic happening behind the scenes (well, except for "deps = gcc", but that's minor). Any action that's too complicated to go directly in the Ninja file goes to a "rule command \ command = ${command}" that just runs a python or shell script.
The only thing I'd like to add to my setup would be "Ninja with glob support", but again that can be handled in a few lines of Python that spit out another chunk of .ninja rules so it's not really a blocker.
I combined learning make with learning TDD, or rather develope some individual style of TDD. I am astonished how effective make is in a rapidly evolving code base with not insificant amounts of unit, component and integration tests. That said I try hard to use the makefiles both, heavily recursive and highly structured.
Of course, I am learning, so my set up is far from optimal. But wow.... Makefiles are so powerful for testing. Its amazing. On the other hand, that the first time I am properly using a build system.
At its core all a Makefile is is a dependency graph, which is necessary in all the more advanced config managers, only those others are much more heavyweight than the vast majority of projects ever need. Most code doesn't need to vary arguments or parameters based on the target environment, and so that complexity is unneeded, in which case a Makefile can do it all.
If you are a python dev, give doit a try.
[0]: https://www.gnu.org/software/make/manual/html_node/One-Shell...
[1]: https://www.gnu.org/software/make/manual/html_node/Choosing-...
Like JS is so popular because it's the main language on the web clients, and by default on most of them.
Also, because doit assumes you have a python project, and make is language agnostic, the latter will have a broader potential user base.
However, doit is not that far from make in the sense it doesn't assume each language workflow. It just gives you a declarative syntax to setup tasks and optionally your dag of dependencies and targets: you then do whatever you want with it. This gives it simplicity and flexibility and power, which are similar qualities that I think make nails.
I do think it would gain from being a stand alone executable so at least you were not expected to have python installed on your machine to run it.
for d in $(SUBDIRS); do cd $d && $(MAKE); done
It would be really nice if make could read all subdirectory Makefiles, build one global dependency graph and then have at it.
Correct me if I'm wrong, but I always assumed `make` to be one of those build tools that could incrementally build targets based on dependencies.
The arcane and esoteric language that constitutes `make` is almost universally avoided, surely if people need a simple task runner there could be better options?
More ubiquitous? Not really.
I use it for executing docker as well, because the docker command is actually about 5 commands with long lists of parameters. But it's just `make docker-stack` for everyone now.
Here's the PDF in case anyone else is interested: https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
I can't find any actual conversation about it on HN so I won't post the HN link
It is a very important paper, I think, mapping out the landscape of build systems designs. But I'm still waiting for something novel to emerge from it. I know the authors planned on integrating the improvements they found into (Cloud) Shake, but I don't know if that's production-ready.
Perhaps Bazel is the best we can hope for at this time.
Curious for those who published source with autotools,etc... do you really sit down and figure that out. It's just a mystery that works or doesn't to me. I never have or would go beyond a simple make file. It really seems tedious on top of the actual code you write. A bit impressive tbh.
I like the elegance of pure make, and do use it when appropriate, but I wouldn't really want to reimplement the things autotools does myself in it.
[0]: https://github.com/jbboehr/handlebars.c/blob/master/configur...
Once I figured out how to get it to build python extension modules my life became a lot simpler. The python way (distools?) is easy enough until it isn’t then I reach for cmake.
Archived copy:
https://web.archive.org/web/20220812135641/https://matt-rick...
tcc -run mybuild.c
*WRITING* CMake is the 11th circle of hell, but CLion is gradually getting more support for it because they use(d?) it as the first-class project definition when it launched
I would pay good money for CMake 4.x to switch to skylark/starlark
if os.stat(object).st_mtime < os.stat(dependency).st_mtime:
...
rules["a.c"] = (["b.c", "c.c", "b.h", "c.h"], action)
...
def make(rules, rule):
(deps, action) = rules[rule]
run_rule = len(deps) == 0 # if there are no dependencies, the rule always runs
for dep in deps:
make(rules, dep)
if os.stat(rule).st_mtime < os.stat(dep).st_mtime: run_rule = True
if run_rule: action()
----------
An amendment: You'll probably want to pass both the rule name and the dependencies into the action function/lambda/object so that you can parameterize it and maybe reuse the action (like a common compiler command):
if run_rule: action(rule, deps)Makefiles on the other hand, are not unreasonably effective in this sense. Makefiles are, in fact, _reasonably_ effective... like a coffee maker that brews coffee well.
That was the deal-breaker for me last I checked.
Just make the program do useful things for the user. Make the computer work for the human rather than the other way around.
Declaration is absolutely a coherent and functional design principle. A declarative system is one in which the outcome is specified and the process is not.
This has big payoffs in domains where it's natural. A good example being grammars. It also has hazards, a good example being the performance of algorithms to parse grammars.
We can see where a declarative build system might be a mixed bag, because the process itself is imperative: make is an early attempt to reconcile imperative build processes with a declaration of what circumstances require their triggering. Basically every build system since make has improved on make, but they all do more-or-less what make does.
The design, in short, is proven, as well as declarative but not purely so.
And the ability to reason about the degree to which make is declarative shows that 'declarative' is in fact a coherent idea. But you can't declare software into existence, you must compile it.
Makefiles are simply configuration files that use whitespace and a couple characters to create the configuration, and what Make's inbuilt functions do with that determine the extent to which the result becomes "more intelligent". Yes they are used to build a graph and execute it, but so is Dotfile notation, and software package configuration files. But we don't call those declarative programming. Many of those configuration files create multiple levels of instructions and require several passes to execute properly. But we just call them "config files" because we don't feel they are intellectually superior enough to be called a form of programming. And on the other hand, we don't call declarative programming "configuration", but they're often the same thing.
Nobody says they "imperatively configure" some software, but they do consider themselves "declaratively configuring" it. Because they've overloaded the word "declare" as if it means something other than "write down a thing I want a computer to eventually do with some automation". People bring up the declarative thing because they want to imagine there's some intellectual value to considering it, but there isn't. You're basically saying "I want to configure a program rather than write one". Which is fine. But just say that and stop pretending that's going to immediately lead to a better result.
Declarative vs. Imperative is a spectrum. For instance, there is a declarative language hiding inside most imperative languages : Infix Math. 1+2*3/71**7 is declarative because it under-specifies the order of operation, only the data flow dependencies implied by operator precedence needs to be respected. In the precedence hierarchy I had in mind when I wrote it, You can do 2*3 first or 71**7 first, it's unspecified and irrelevant. I only ask that you do both before you perform the division of their results, and that the addition is the last operation. Meanwhile, in Forth, math is imperative, you have to unroll the expression tree into an exact sequence.
Declarative is any language that under-specifies the task being described. Therefore, every language worth using is declarative to some degree or the other. After all, that is the very purpose of a high level language : to under-specify a task by describing only the most essential of details, all the abstracted details are taken care of by either inference (compiler figures it out, possibly according to rules that you need to be aware of) or exhaustive checking (compiler generates all possible cases and code to select among them at runtime, or very generic code that can handle all cases uniformly). If, like Alan Perlis says, "A low level language is that which requires attention to the irrelevant", then every good language is already declarative in some sense, you omit things and they get taken care of automatically, that's what Decorative means.
You can say you hate buzzwords, I empathize. You can just say that make is bad software (trivially true, or we wouldn't have needed software to generate makefiles, effectively making them a machine code that isn't meant to be written by humans) and that being declarative doesn't make it any less bad. Declarative vs. Imperative are just names for design decisions, they guide a language designer but don't have the power to make a language good single-handedly.
Ironically is a very concise definition of `declarative`.