I would like to call attention to `just` (https://github.com/casey/just) which is a modern take on "make if we didn't have to keep it backwards compatible", and more focussed on just task running rather than tracking dependencies.
The nice thing about dates is that they can be checked instantaneously, and independently of file size. If a file has changed, its date has, so it's all the information you need.
time find ./vendor -type f -print0 | sort -z | xargs -0 sha256sum | sort -k2 | sha256sum
Note if using this code for practical purposes, you should be sure your collation is set appropriately. That's taken with 1280 files that are in total 19MB.
For raw throughput, you can use BLAKE2 for even more speed https://www.blake2.net/
This might be approaching niche but I’ve seen a few mega-projects with 100 times more than that (also mostly due to vendored deps). You really don’t want to read 10G from disk for every incremental build. The actual sha computation isn’t the bottleneck, disk is. Ninja on the other hand, using mtime, handles this without a sweat.
The best solution would probably be to integrate the change detection with git, it is already content addressable and has lots of other smart things built in to make it fast.
Unless the date hasn't changed -- there is no hard guarantee that mtime will be accurate.
Yes, file hashes might be another way. On the other hand, if I modify a comment, the file hash will change, but re-compilation is not necessary. In this case, you might need yet another way...
You can always write your Makefiles to compute and depend on hashes or whatever else you might consider appropriate for your exact use case.
I mean those things work, but they're by no means nice. Parallelization works, but output of different tasks (and even the log that certain commands have been run) are interleaved, making it impossible to view progress. Language agnosticism works by operating on the level of shell commands, but that's really about it. If shell commands need to be customized on a per-system or per-OS level... well it's possible to hack around that with enough external scripts and even esoteric if statements inside the Makefile, but it's not nice.
Documentation for make feels like it's from the 80s. Which it is.
All these things (with the exception of having to rely on arcane sets of external scripts and if-statements-in-Makefile) are perfectly fixable. But they don't get fixed.
IME make’s model is often too simplistic to be of use, so it simply gets in the way of tools already doing things internally with proper insight, or requires force-cleaning everything because it's too dumb and just breaks the build when you e.g. switch branch.
Just’s author is also looking at integrating `redo` for building / dependency tracking, though the issue has no activity since it’s been created.
IMHO, this is the primary weakness of Make. The standardization efforts never really happened so you have various forks of 1980s code with annoying incompatibilities. BSD Make is missing some of the implicit build rules from GNU Make (only able to compile simple single source file applications without an explicit rule), and Windows Make tends to be extremely primitive. Worse, Make still in 2022 can't handle spaces in file names, like c:\Documents and Users\.
I would be so much happier if everybody could standardize on one flavor of Make. Probably GNU Make.
With included Guile scripting support!
https://www.gnu.org/software/make/manual/html_node/Guile-Exa...
I really wish it was more common because having it would be great.
What's built wrong is make itself. Because the signal it uses for "this dependency has changed" is the mtime, any change which does not move the mtime forwards will be ignored. Switching between concurrent git branches, for instance.
> Usually it means you've tried to manually re-create the inherent Make rules and have gotten it wrong somewhere.
And pray tell how do you "manually re-create the implicit make rules" when such rules don't exist? GNU Make has implicit rules for C, C++, Pascal, Fortran, Raftor, Modula-2, and assembly. That is, frankly, not much. And mostly outdated.
I don't understand how it can work in parallel if targets are not somehow associated to files. What's the advantage to a shell script with separate sub-commands, then?
So I assume it doesn’t need/want to implement parallelism.
I've (ab)used Make for server pushes, and sometimes you want those to run one at a time, and sometimes you want -j 100 (sometimes -j 100 is even effective).
Sophisticated dependencies are nice to have of course, maybe make push-all transitively includes building the thing locally, but it's still useful to have just the push step (or the reload step, etc), have optional parallelism. If you start to build a tool that does do these things first, in order; then these things in any order; then these other things, that does look a lot like Make. but if you just handle ordered steps and independent steps, and let a person do the mixing, then it's not nearly as much work as Make.
What are you talking about? Make does not deal with "sophisticated" dependencies. Make is insultingly simple, not sophisticated at all. It's just a damn directed tree. The simplest data structure in computer science after the array.
There are also some fancier shortcuts for expressing rules in makefiles, but you can ignore them and use only the simple stuff. That's already a much more powerful tool than "just".