Hacker News Reader

> If you add a new source file, you must rerun autoreconf to regenerate your Makefile

No you don't: automake generates rules for `Makefile.am -> Makefile.in` (`automake`) and `Makefile.in -> Makefile` (`config.status`). You should just be able to edit `Makefile.am` and run `make`.

> Is it really less effort to write Makefile.am than to write a straight Makefile? Wouldn't that be simpler?

I don't think so. Not if you want to get DESTDIR support right, and proper dependency tracking (dep info as a side-effect of compilation, so you get all the headers and don't slow down one-off builds), and support for all the standard Makefile targets, etc.

As for your "which build tool to actually use?" question, I wrote a longer post about that in the context of bootstrappable software: http://jackkelly.name/blog/archives/2024/04/01/which_build_t...

It really does seem like autotools remain one of the least bad options, but maybe muon (C99!meson) and samurai (C99!ninja) might be a workable option.

>I don't think so. Not if you want to get DESTDIR support right, and proper dependency tracking (dep info as a side-effect of compilation, so you get all the headers and don't slow down one-off builds), and support for all the standard Makefile targets, etc.

With any make, yes. With modern GNU make you can embed sh code inside a makefile with the define hack, so no, ugly, but no.

I'm be written lots of C code that runs on lots of architectures & platforms: PDPs, x86, ARM; Windows, BSD, Linux, VMS. My projects have spanned them gamut from tiny to gargantuan (millions of lines of code). I've never needed autoconf... what's it even for!?

Well, as archaic as it is, and as awful as its execution model is, autoconf is at least somewhat useful:

* determining availability of different syscalls (yay, printf)

* determining compiler

* verifying dependencies are installed and available (e.g., openssl)

This last one is made much easier with pkg-config and the pkg-config macros in autoconf. There's an argument to be made that pkg-config is bad because it will fail the build if you're just missing the wrong .pc file somewhere, even if your system satisfies the needs of the program - true enough. But pkg-config _does_ make checking for dependencies in autoconf much easier, and it does something essential for correctness, which is gathering up the necessary compilation and linker flags of your dependencies to pass along to the compiler so that you can use them correctly and match ABIs.

A good configure script will guide you through getting dependencies installed, which is much nicer for users than confronting them with compilation/linker errors.

Unfortunately, pkg-config couldn't handle C++ files nor the transition from lib32 to lib64; as Captain Kirk heard from Bone: "he's dead, Jim".

In decades past, if you were developing software for Unix you would inevitably run across compatibility problems. Even with the POSIX standard, every Unix was trying new things, inventing new things, or just implementing the same thing in slightly different ways. So your programs had to have a bunch of macros to switch between different variations. Some API would have two arguments on Unix A but three on Unix B, so you would have to turn that api call into a macro that would insert the missing third argument or whatever. But now your users needed to know which configuration of your software to use. You could try documenting all the little knobs and switches and make them choose the correct values, but someone hit on a clever idea.

Just write a little program that tries calling the API with two arguments, and see if it compiles. If it does, you could automatically define THAT_API_HAS_TWO_ARGS and your code will do the right thing. If it fails to compile, then define something else.

So you start writing a shell script that will run through all of these tests and dump out a header containing all of these configuration choices. Call the script `configure` and the header `config.h` and you are good.

Except that actually writing a shell script that is really actually POSIX correct is pretty hard. And the code is pretty repetitive. Mistakes start to creep in, bashism break things when it’s actually run under `sh` and so on. And it turns out that all the little tools we rely on, like find and grep and sed and awk, are all different on different platforms too! Turns out that nice option you were passing to grep is just an extension, only present on your specific flavor of unix.

So the next clever idea was to write a program that generates the configure script. The repetitive bits can mostly be solved by text substitution, so someone had the bright idea to use M4. Thus autoconf was born.

Autoconf solved a lot of problems. It ensured that the configure script was actually POSIX correct. It did this even though the generated script was even more complex than ever, with extra logging and tracing and debug output that most people never look at, but which is actually super useful for debugging. The configure script you had written by hand didn’t have any of that!

And autoconf came with baked–in knowledge about all the things you might need to test for. Any time someone discovered some new difference between Unices, new tests were added to autoconf for it. Autoconf users hardly had to do any more work than to look up the name of the macro that it would define for you.

Of course, M4 causes actual brain damage in regular practitioners, and nobody cares to write software for any Unix except Linux these days. Even the BSDs are an afterthought. Technically OpenSolaris is still out there, with superior features like ZFS and Zones and whatnot, but that’s a lot of extra work. So autoconf is a dinosaur, solving problems that nobody really has any more.

Rachel is not wrong that it’s crazy to rerun the same configure script over and over, but really only developers have to do that. Most people just run the thing once when they install your software, and never see it again. And most people don't even do that any more, because they download binaries from their package manager most of the time. It’s just the distro packagers that actually run your configure script.

And she’s right that the answers configure gets should be cached system–wide. Autoconf does support that, but it’s harder to do correctly than she remembers. And by the time it was possible usage was already waning in practice.

It always seemed to me that most of these differences and macros could be defined in a static header file that has a million #ifdefs for different OS and compiler versions. That seems simpler and more reliable, and usable for cross compiling, than running dynamic checks with autoconf.

Hmmm. Nope. The file would be near unusable and contain a pile of things you don’t care about where autoconf can cope with the one or two cases you are trying to manage.

Despite the description above of the genesis of autoconf, at the time when there were dozens of nearly compatible unices, it was a key enabling technology for free software to be distributed and shouldn’t be discounted just because it’s a little hard to understand.

It was absolutely a boon to all developers who used it. If you didn’t use it you were doing things the hard way. Just don’t look too hard at the implementation, because like I said M4 causes actual brain damage.

That's a very nice summary. But you're missing one crucial point. These are GNU tools and if you use them within that ecosystem, you also often use gnulib.

Gnulib is effectively the GNU Portability Library. Your fancy, new, auto generated configure script can find all the differences, but someone still needs to account for them and write alternative code to support the various platforms. This is where gnulib comes into play. It reads all the configure checks and plugs in replacements / stubs for whatever is different. This allows you, the developer to simply target GNU/Linux in your code while gnulib handles everything else (and pretty automatically) making is portable across all the unices and even non UNIX systems.

Yea, I completely forgot to mention gnulib.

Yeah, there’s a lot of software that uses autoconf without gnulib. It’s common to include a load of default platform tests in the configure script that are used by gnulib, but without gnulib the software never uses the results of the tests. All these futile tests are an appalling waste of time and energy.

This comment should really be promoted to the top level and pinned. I wanted to write something similar, but if I had, it would have been less detailed than this.

That’s pretty kind of you to say. I feel like I could have come up with a better example though.

It seems more, then, that devs need to just move on from gross C (and C++?) at this point to modern mandates without all this insecure baggage and constant security issues from lack of memory management.

IMO Cmake's "killer app" is that it will generate projects for Visual Studio and Xcode, which allows you to get access to the Windows/Mac specific tooling that those products offer. It's also a LOT faster on Windows than Autotools because of how NTFS performs poorly when dealing with writing a ton of small files. I agree that its syntax is ugly but once I got beyond that I found it fairly simple to work with. The most annoying part was figuring out which MSVC flags corresponded with the GCC flags I use.

Out of curiosity, what does your C++ development workflow look like? Do you stick to using a text editor rather than an IDE?

Rather than the "killer app", that's the only thing that cmake does better than other systems, and the only reason for anybody to use cmake.

It's kind of weird that these days, many (most?) projects use Cmake directly as their build tool. They don't generate a VC or Xcode project and then work from that; they generate a VC project and then immediately build it. The CMakeLists.txt effectively is the project definition. VS now has support for opening CMake projects directly, without creating a VS project file from them. So the thing that made CMake useful (and what also makes it complicated...), is now used less and less.

Honestly I think cmake got a foothold early on because it had color.

It also hides stuff so users look at simple, colourful lines with progress indicators instead of naked, overwhelming calls to the compiler.

That said, as a user I prefer autoconf. The semantics of --with-foo or --enable-bar are simple and well documented. And ./configure --help shows all the possibilities. Faffing around with -DCMAKE_STUFF is horrible. Ccmake should be nice but in practice never works properly.

As a developer, both autoconf and cmake are horrifying and a huge time sink to get them to behave.

We should just write Makefiles. If you want to be really nice, have it source a Makefile.inc file and keep all the user configuration stuff in there. If you want to be really really nice, consider accepting pull requests for .inc files that work with popular distros.

I just write (gnu) makefiles for my projects these days and invoke pkg-config to detect libraries and features. It's easy for me because I've been doing it this way for decades. But there are way too many footguns for me to recommend it to new developers (although I'm not sure what I would recommend instead either).

I think Make is exactly what you want, and I do recommend it to everybody, since the default alternative is usually something heinous like CMake which isn't really an improvement. You want the bit of logic to create a "build system" out of Make abstracted into a library, and then it's perfect. I use this: https://github.com/dkogan/mrbuild/ but there're many other ways to do it

What I want is a DAG, but with better syntax than make.

The closest I have come to ideal is a custom ninja build file generator, which is pretty simple. It's been easier for other people to maintain than make or even rake.

Makefiles are too brittle and allow all kinds of user errors. Targets with too many dependencies, too few dependencies, race conditions. The default doesn’t even handle transitive #include dependencies without you yourself plugging in complex .d-file generation using the compiler.

Are they though? The "complex .d-file generation" is "gcc -MMD" and a "-include *.d" at the end of the Makefile. You specify the dependencies, and it's on you to get those right, as it is in every other build system. I've never seen a race condition.

The problem with Make, is that it works just well-enough to allow people to sorta get it to work without requiring them to learn how to actually use the thing. Then they don't read the manual, and complain that Make doesn't, in fact, work.

Nice job, you just hit a footgun: try to remove any .h file (and the corresponding #include directive) and see your build just break. You need an empty `%.h: ;` directive for make to ignore missing .h files.

Make is full of this stuff and as a distro packager having to deal with each's individual quirks is prone to drive one insane. Please consider using meson, at least.

I do not believe that adding such rules for ignoring files is wise.

I prefer to just run "make clean" after I delete, rename or move source files.

With a properly written "Makefile" nothing else needs to be done and there will never be a broken build. With a properly written "Makefile" nothing has to be done when adding source files.

I don't know how you can think CMake is worse than autoconf. At least it has a declarative approach with good mechanisms for reusing common logic. Its much easier for IDEs/editors to work with it.

I don't think it's worse, per se, but I hate it/myself _more_ when I contemplate switching from autotools to cmake -- it's like upgrading from CP/M to DOS, when what I'd like to have is a Mac.

And as much as autotools blows, every time I try to build some project that uses cmake I inevitably go through some rigmarole making a build directory (not that out-of-source tree builds aren't the right thing, they are, but why make me create one?) and finding some obscure set of -D options to get the sucker to build on my system. configure && make still has cmake beat from a user's perspective.

Yeah it’s not intended to be used directly for end user builds. Its ultimate product is a set of makefiles (or Visual Studio project files) and the developer should distribute those.

libtool does serve a purpose: it causes plain 'make' to produce shell script wrappers as outputs, which is maybe kinda sorta vaguely useful when developing and utterly and completely obnoxious if you want an actual ELF file.

So you end up running 'make install' when you didn’t actually want to install or trying to remember how to find the actual ELF files. And you then wonder why 'make install' appears to be building things instead of just installing them, and you curse at libtool.

In its slight defense, the magic incantation to convince gcc (or, yikes, pre-gcc cc or ld or…) to competently produce a DSO is both nontrivial and platform dependent. Last time I tried to get all the options right off the top of my head, sensible defaults were not a thing.

Oh right, how could I forget that awful aspect of using libtool? I hate my life; maybe 2024 will be the year I find a decent build tool and the motivation to move projects to it.

Contra your slight defense, is it really so hard to produce a good DSO these days? -fPIC is half the battle and a couple other weird flags for macOS/darwin will do you. Cross-compiling for Windows, sure, a few more flags — but you look them up once on stackoverflow, you have three options, and you're done.

100% this. Libtool is awful.

In the early 2000s, I ripped libtool out of a product that built on Linux, Solaris, FreeBSD, Aix, Tru64, Mac OSX and Windows.

> Contra your slight defense, is it really so hard to produce a good DSO these days? -fPIC is half the battle

I did say “slight” for a reason :)

I always thought that purpose of libtool was to incorrectly guess that it was smarter than the developer in figuring how to link files together. When I've been trying non-default link magic stuff (something like lto), trying to figure out how to get libtool to actually pass the damn necessary linker flags to the linker was damn near impossible.

> Bazel's obviously a terrible Google project and won't work if you're not sufficiently Googly

I don’t even know what that means, sounds like a bunch of fud to me. C++ and Java are first-class languages in Bazel. It works great for these, then Go, then you start venturing into the alpha land

Works great if you follow Google's style of vendoring all of your dependencies. If you try to rely on third-party package managers like Maven or Ivy or whatever C++ uses, you're in for some serious pain. The more you venture into OSS tools like Spring, gRPC, containers, the more you have to incorporate shoddy and poorly-maintained third-party Bazel tools which inevitable stomp on each other in subtle ways.

My team abandoned Bazel for Gradle, and it has been a tremendous quality-of-life improvement.

> Works great if you follow Google's style of vendoring all of your dependencies. If you try to rely on third-party package managers like Maven or Ivy or whatever C++ uses, you're in for some serious pain.

It sounds like you've mainly dealt with Java because it's not a pain in C/C++ - if you wish you can link dynamically with system libs or vendor .so binaries. And at least for Maven I used rules_jvm_external which worked fine.

I’ll grant that container rules were a mess for a long time but new oci rules are much improved.

> My team abandoned Bazel for Gradle, and it has been a tremendous quality-of-life improvement.

I hear as much if not more gripes about gradle of the same variety - "hurr durr it sucks bc i don't like it". Personally I'm thinking this is more appropriate critique for an art piece not a build system.

I've done some C/C++ with Bazel before. It's nice, except that you end up having to vendor all your dependencies if you want repeatable builds :). Had more success sharing projects and building a community with CMake, horrible as that tool is.

The reason Gradle is a quality-of-life improvement is that we had boxed ourselves into a corner. rules_jvm_external, rules_docker, rules_proto, and a set of others all relied on internal details of other rules (accidentally, because pre-MODULE.bazel everything gets loaded into one giant global namespace and collisions happen). We basically got ourselves locked in, unable to move off of rules_docker, upgrade our JVM beyond 17, or even upgrade Spring, without breaking and having to upgrade everything else.

Gradle, which has first-party support for dependency management, and which is the primary system supported by large OSS vendors (Spring, etc), doesn't have these pitfalls. Poorly-written Gradle plugins can sink you as surely as Bazel packages can, but there is less opportunity for you to discover them.

I'm well aware that we were Doing Bazel Wrong, and I've certainly had my share of pain with Gradle, but never in my life have I seen so much time wasted on a build system. For plain old JVM stuff, Bazel didn't work well for us.