Autotools Mythbuster | Hacker News Reader

Autotools Mythbuster

autotools.io

90 pointsnbaksalyar85 comments

If you want to get started with autotools, try to build yourself a minimal autotools project. Once you have a working stub, you already took the biggest hurdle.

You only need to focus on two files: configure.ac, where you check for features, and Makefile.am, where you list your build targets and their sources. You can also write normal Makefile rules there.

After editing these files, run `autreconf -fi` to create/update the configure and Makefile script. Then run

    ./configure
    make

to build your project. Have a look at Appendix A/Basic Autotoolization on the submitted website or checkout libabc by Poettering/Sievers:

1. https://autotools.io/whosafraid.html#idm117385102076160

2. http://0pointer.de/blog/projects/libabc.html

Learning about autotools is an amazing source of fremdscham. The whole compilation/packaging stuff is hard to get right but autotools looks like someone completely gave up on producing a good, decent or even bad solution and just tried whatever didn't completely and utterly fail at the task. That everyone else just kind of seemed to go with it... there are just no words to describe how embarrassing that looks.

Well, is that really fair? autotools' scope is great, like you concede. But it was designed around 1991, right?

> That everyone else just kind of seemed to go with it

There was nothing else better available, and this was effective -- is effective.

If you haven't seen the effect of momentum in software, then this is nothing. Look at how x86 systems get bootstrapped; look at how we interact with important platform devices like the RTC (I/O ports). Virtually none of this has changed because it doesn't introduce significant limitations in operating systems or hardware but it's downright odd.

cmake is pretty bad, too, though at least it runs on Windows. (And I don't think it's got quite the same goals as Autotools, though their scopes overlap in many respects.)

I've never met anything build-related, scope large or small, that wasn't somewhere between "PHP" (horrid, workable if you're careful, will rot your brain in the long run) and "Cthulhu" (beyond evil, insofar as normal human modes of comprehension can be said to apply).

I wonder if this is some kind of, like, Heart of Darkness shit.

The big problem is that no one did a better solution. All alternatives are missing one or more crucial features of the autotools (most prominent being portability). And that is sad as well.

Cross-compilation is another. When I was building packages for the Fedora MinGW (Windows cross compiler[1]) project, it was mostly trivial to get things working if the package used autotools or cmake, and an impossible nightmare for everything else.

[1] https://fedoraproject.org/wiki/Features/Windows_cross_compil...

IMO it doesn't matter if there are better alternatives, as that's mostly a concern for new projects. The thing is, there's a good chance that you'll have to use autoconf at least once in your dev life, sooner or later, because very large number of major opensource projects use it (with no clear intent to switch). So at least you've got to be prepared and know how autotools are working.

  but autotools looks like someone completely gave up on 
  producing a good, decent or even bad solution and just 
  tried whatever didn't completely and utterly fail at the 
  task.

That is more or less what happened, by their own admission, "why does it look like gorilla spit?"

https://www.gnu.org/software/autoconf/manual/autoconf-2.65/a...

Although I disagree that they didn't produce a tolerably decent solution. It's a near hopeless task to account for the differences of dozens of operating systems, even if most of them are all Unix-like, yet autotools still tried and mostly succeeded. Nowadays some build tools don't even attempt to solve this problem and only target Windows, Mac OS X, and Ubuntu, and usually only the latest version of each.

A solution can be no simpler than the essential complexity of the problem it is solving. It is not clear to me that the essential complexity of the problem is significantly simpler than the autotools solution. Across the totality of all C/C++-using systems, across all of the mostly-POSIX and vaguely-POSIX-ish systems, across all the libraries, the toolkits, the OSes, across the decades, across all of what it is trying to span, the essential complexity is quite large. Contemplating all that probably makes autotools come out looking pretty good in context.

I'd observe that every major modern language since then has taken an approach that in one way or another, obviates the need for a full autoconf. C's contribution to large-scale code reuse was largely showing all the ways in which it could go wrong, and encouraging future language developers to get it more right to start with, which they mostly have.

(Note I'm not saying there aren't any problems with the way any modern language works. But they're still all simpler, generally more powerful, and even in their worst case, nothing like as bad as the autoconf-world's worst case.)

Even as an autotools critic, I totally understand that argument about the steps that were necessary for dealing with those portability problems. Well, way back in the day when they existed to the degree they did. Today, there should be much less complexity as the systems are more alike than different. What complexity is there is mostly a relic from days gone by that never got cleaned up because nobody wants to invest the effort.

As someone who maintains a hand-written configure script for a project that I build on just a handful of platforms, I have to disagree. There is plenty of cruft there to detect and take care of.

For example, recently I discovered that -D_FILE_OFFSET_BITS=64 works on 32 bit Solaris 10 for large file support. However, fseeko and ftello are not declared. For that you need something special, namely -D_LARGEFILE_SOURCE --- this is not just assumed from _FILE_OFFSET_BITS=64.

New developments in POSIX have created a fuzzy landscape which resembles some of the confusion in 1990 Unixes. This platform needs _POSIX_C_SOURCE=<at least such>, another has the same thing under _XOPEN_SOURCE. Apple won't declare anything useful to you without -D_DARWIN_SOURCE.

Some platform needed __EXTENSIONS__ to declare struct winsize. I don't remember which one, but my script detects that.

Systems are not becoming more alike. Even when it comes to standard stuff, they vary in their conformance to POSIX, and in how they declare identifiers and the exact headers that have to be included and such.

Appreciate the insight. So, are you on the side of the tool testing for every feature that might exist going back decades or modules that apply to each platform or version like we handle most dependencies? I think autotools' approach is overkill in terms of complexity and efficiency.

I'm not the parent commenter, but experience has taught me that testing for features, not versions, is definitely the right way to go. The alternative ends up with you creating a hodge-podge database of which features are present in which operating systems / versions, which is inevitably out-of-date, incomplete and wrong.

Only test for the features you actually care about, though. And at this point, it's perfectly fine for a new project to assume some baselines like POSIX-1996 and even C99 (unless you're targeting Windows...).

Testing for features is the right way to go. If you add support for a new platform, there is no guarantee that the tests will all magically work. But a lot of them, in fact will. That reduces the amount of remaining work.

You could have a canned set of files, one per platform, containing some canned settings of variables. (The Pine e-mail program was a good example of this; its Alpine successor uses Autoconf.)

If you are porting to a new system, you have to guess which of the existing canned configurations is the closest one that will work and use it as a template.

Since there is no logic to detect if any of the features related to the configuration variables is actually present and in what form, it's just guesswork. Okay, it doesn't build. Now which of the relevant variables in the canned config is related to that breakage?

I'm on the side of testing for features in a realistic, reasonably modern context which takes into account a set of supported platforms.

There is no point in testing for things for decades-old systems if you don't have any users on such installations.

The Autotools approach isn't overkill; it's overshoot. To declare something overkill, it has to solve the problem fully. Most of the Autotools-build packages in a GNU/Linux distro will not work on ancient systems, so they don't actually kill that problem. Even if the configure script actually runs through (unlikely), it will fail to detect things that the programs need. There won't be any fallbacks and so things won't build. If things happen to build, they might not run properly.

Porting requires work. You don't develop on system Y and assume it works for users of system X because you're on Autoconf. To call system X supported, you get a system X box with a development environment and port X. That is why, subsequently, the configure script works on system X flawlessly and everything builds.

Right. I have a 250-line utility that is currently testing for FreeBSD, OpenBSD and Darwin in the Makefile (due to differences in which libraries need to be linked to get a function, and differences in the include and library paths), and tests for FreeBSD, OpenBSD, NetBSD and Darwin in the .c file (due to differences in which header file functions are declared).

If someone tells me it doesn't work on, say, Solaris, my next step will be to convert it to autoconf. The essential complexity is still there.

To the extent that that is true in some abstract sense, it's difficult to manifest it in real actions. Pick a feature you think is unused. Suppose hypothetically you could wave a wand and remove it. What are the odds that A: at least one current user would be impacted B: at least one major platform and C: a majority of users will be impacted at some point (i.e., something in their Debian stack ended up using it and perhaps needing it)?

The problem isn't even that the answers to those question are bad and preclude all changes... the problem is that there isn't even a concrete method for determining the answer! And you only need to have experience with trying to remove one or two features before the desire will be seared out of you by firebrands wielded by angry crowds.

I've tried to remove much simpler "obviously unused features" out of much smaller codebases in environments all under the control of one corporation and gotten burned pretty bad, to say nothing of autotool's scope.

When I'm talking about relic cruft, I'm talking about things like PHK described here:

https://queue.acm.org/detail.cfm?id=2349257

A few highlights:

" yet the 31,085 lines of configure for libtool still check if <sys/stat.h> and <stdlib.h> exist, even though the Unixen, which lacked them, had neither sufficient memory to execute libtool nor disks big enough for its 16-MB source code."

"This is probably also why libtool's configure probes no fewer than 26 different names for the Fortran compiler my system does not have, and then spends another 26 tests to find out if each of these nonexistent Fortran compilers supports the -g option."

"so the input files for autoconf happen by copy and paste, often hiding behind increasingly bloated standard macros covering "standard tests" such as those mentioned earlier, which look for compatibility problems not seen in the past 20 years."

Looks pretty horrible to me. I think removing all that might piss off someone but they're refusal to upgrade shouldn't hold back everyone. Besides, why would I want all those tests running if most of them make no sense and it just adds overhead? And in an otherwise-unused language (M4)?

Looks like crap to me. They should get rid of it.

Basically, Autotools is really stable software that has a ton of bugfixes in it. You can't just throw that knowledge away. The overhead of unnecessary tests is negligible compared to compilation costs for anything but the most trivial project. And, I'm pretty sure if you ask the mailing list, there's a way to turn off unnecessary checks for a Fortran compiler, probably by specifying the languages to support with AC_LANG or something.

Yeah, it's clunky and obtuse, but it works to give you portability to just about every system, POSIX or not. And the free support is first class if you ever have a problem.

While it's quite popular to bash autotools, how about instead working towards a successor? One that's as easy for the end user to use as autotools, but with less technical debit.

It's a hard problem, as evidenced by the inherent complexity in autotools, but there are a lot of smart folks who could apply their effort towards that goal.

there _is_ a huge amount of smart folks who worked or works on build systems. Sadly, it seems everyone is 80% better than the others in something and 20% lacking in something else.

For example, the SCons wiki has some interesting comparisons.

https://bitbucket.org/scons/scons/wiki/SconsVsOtherBuildTool...

I really liked SCons, but it seems like it never really hit critical mass.

Ubiquity is important for a build tool.

Doesn't CMake count as a successor?

I think more like "differently bad". However of all the build systems out there (and as a Fedora packager, I have had to deal with many), I'd prefer if you used either autotools or cmake, because those are the ones we're most familiar with and we can fix things.

Of course there are many alternatives (of the alternatives, I can only really recommend cmake).

The problems broadly speaking are:

* Assumptions: autotools assumes that only a basic Bourne shell is available, and a basic make, and that make is a good way to build projects. All of those are questionable, and certainly if you get rid of the assumption that you're only allowed to use ancient tools, you could immediately make autotools better, if only by removing all the shell cruft. But then someone will complain because your project doesn't compile on SunOS 4 or whatever.

* Familiarity: When I'm packaging your project for Fedora, I will not be pleased if you've used some obscure or half-assed build system. If you're using autotools, I may dislike it, but at least I can fix things.

The second one is a difficult one, because until your build system gets popular, it will never be familiar, and us packagers will keep cursing you. Once it's popular, it'll probably be quite crufty because it has had to handle all the horrible corner cases that autotools can do now.

Yeah, I know it's a damned hard problem, but that's why I posted it the way I did. This was mostly an attempt to head off the typical cargo-cult hate against autotools.

The hatred is not really cargo-cult-like - you can understand it and still not be happy with it. That said, thinking about what it would take to replace it helped me come to an admittedly somewhat grudging acceptance of it. The testing alone...

Do you have any opinion on GYP, as used by Node.js and (last time I checked) Chromium? It seems to be particularly good at producing a large binary made up of a complex graph of static libraries.

Some simple things like compiling a single file with specific options quickly get needlessly complex, IIRC.

Even worse, autotools doesn't have the concept of modularization.

Look at the nginx configure scripts. They're hand-written, and modularized. Check for "foo"? There's a shell script. Autotools does the same thing via special-purpose autogenerated shell code.

That's why the configure script is 1000's to 10's of 1000's of lines of shell script. It's all auto-generated crap.

And autotools doesn't have the concept of bootstrapping. Building a C program? Great! Test everything with shell scripts. It might be more efficient to have C programs doing work, but no... it has to be the lowest common denominator.

Because managing multiple pieces is too hard. Why? Because there's no modularization.

I've used autotools for decades because everything else is worse. But I have a long-term deep-seated loathing of it which is hard to put into words.

> I've used autotools for decades because everything else is worse.

I take it you've used CMake - could you explain in which ways you think it's worse than autotools? I agree it has concerning design flaws, but they don't come anywhere close to autotools' and I'd say CMake is overall dozens of times saner.

Does CMake generate a deliverable which builds on a machine that doesn't have CMake installed? With full incremental build support and all, so reasonable development can take place on that system?

Yes to the first question, it generates platform-specific build files (e.g. Makefiles for Unix/Linux), which don't need cmake itself to build the software. Regarding the second question, I believe so, but I'm not sure.

Can we give everyone exactly the same tarball of a project and just have it unpack and build on their platform?

I think so, if you pre-generate the build files for every platform and distribute them all in the tarball.

Haha

Of course once the makefile is generated, there is no need to use cmake (as long as the project is not modified), but you cannot share a makefile generated by cmake, not even redistribute it. You cannot even re/locate it on your hard drive.

The difference is that autotools generates source distributions that do not need autotools to build, while a cmake project always requires cmake when building.

Oh, I wasn't aware it wasn't distributable, my bad.

The problem is most people who end up using autotools and other build/install/package tools are not interested in their internals or interested in working on them. They just want a tool that gets their job done and they get to do something else.

I realized this a while back, when I was advocating git at my company (maybe 5 or so years back...) some developers loved because they like the idea of it, they wanted to know how it works underneath etc., other group didn't care, didn't want to hear about internal they just wanted a list of commands to type to get their code committed.

Neither one is wrong or or bad, it just comes down to personalities or interests. Autotools is a bit in the same area.

A lot of people will complain about it, because they want to use it to get stuff done. But they will never the ones to start writing tools like it, because they do something else -- databases, graphics, numerical simulations etc.

So "start working on a successor if you don't like it" might not have the same effect for this project.

For me it is called cmake.

> as easy for the end user to use as autotools

End-users don't build from source in the first place. At least, if they do, then the developers and/or packagers have failed to do their job.

I recommend a hand-crafted configure script, which provides the same usage interface to package builders, without Autocrap:

E.g.:

http://www.kylheku.com/cgit/txr/tree/configure

That script is 2500 lines long, and appears to have been written by a wizard. If my only other option was producing something like that, I'd just bite the bullet and use autotools. (IDK, that may be your point?)

Do you honestly think that I gradually developed a 2500 line configure script by hand over more than six years for an important project of mine ironically, in order to promote Autocrap?

The script detects just what is necessary for the program and is easy for me to understand, extend, port and debug.

You can fix it right on a given target platform where you're trying to get it working.

You can't do that with an Autoconf configure script (in a way that you can merge back to your stream), because it's generated. You need an installation of Autoconf.

If you want to patch the Autoconf-generated configure script with minimal changes, just reflecting the topic of the change you're making, you need the exact same version of Autoconf with which it was generated, otherwise there will be irrelevant diffs all over the script.

(I learned this when I developed an embedded, cross-compiled LInux distro from scratch. On my development system, I had about half a dozen installations of different versions of Autoconf, so that I could create minimal patches against configure scripts!!!)

And, of course, you first have to work backwards to figure out how to coax the script behavior you want indirectly through the m4 macro files. Forget about "I just want to make this tiny tweak in one command and it will Just Work".

In my script, you just ... open the script and write the necessary code. Then regression-test it on the other targets, do any necessary tweaking and commit.

Once you have written one of these scripts, you can easily re-use it. My above script is BSD-licensed. Anyone can take it and cut out whatever they don't need. If you want help using it or have any questions, you can fire me an e-mail.

Thanks!

The Apache Lucy project has an internal tool called "Charmonizer" which doesn't assume Bourne shell.

Charmonizer needs a supported C compiler command line interface (POSIX cc, MSVC, maybe others in the future) which can run "system()" commands in a supported shell (sh, cmd.exe).

The testing logic is all implemented in C code. Instead of shipping a shell script, there's one big amalgamated C file, plus a tiny "configure" wrapper to present a familiar interface.

Charmonizer hast given Lucy Autoconf-style probing on all our target platforms without adding any prerequisites beyond what Lucy already needs itself. This has made installing Lucy much easier for a wide range of users.

Right now Charmonizer remains a private component supporting only Lucy but if there were to be interest it could be broken out as an independent project at Apache.

http://github.com/apache/lucy-charmonizer

Could anyone speak to why you would use Autotools today in a new project rather than gyp [0] (created by the Chrome team because Autotools sucks) or waf [1]? How many people really have a requirement to support more than just Windows, OS X, and Linux? I can understand not wanting to rip out a working Autotools implementation of a legacy project, but it's greenfield use should decline to zero.

[0] https://gyp.gsrc.io/ [1] https://github.com/waf-project/waf

autotools isn't mainly a build system; it's a system for preparing a program which will self-configure and build in various systems. Those systems themselves do not have any special tools installed. No Autotools, no GNU-anything.

Autotools was intended to solve the problem of how to give the same tarball of code to a user of SunOS 4, Ultrix, HP-UX 8, and so on, such that each of those users could just unpack the tarball, run ./configure and then make. (Using the make that came with their OS, not CMake, not GNU Make, not gyp and so on).

Some of the requirements don't hold as much any more; you can rely on newer POSIX shell features in a configure script, and GNU make is more widely available and so on, but the basic idea is solid: no requirement for any exotic build system just to get the program running.

It's the implementation that sucks in Autotools. The high level ideas are solid.

Historically they have supported a lot of things the others didn't or didn't easily support: cross-compilation, changing the prefix/libdir/datadir, creating tarballs, "distcheck" to automatically check the tarball works, libtool, etc. Also lots of Linux tools are built around autotools, for example it's much much easier to rpm-ify or deb-ify an autotools tarball. Another example, there's gettext integration for autotools, automated tools for translators might rely on autotools, etc.

I'm probably listing the tip of the iceberg as far as network effects. In the Linux/Unix world, autotools is (or at least historically was) what everything and everybody knows how to deal with, and non-autotools is annoyingly nonstandard.

Your gyp link there says for example that its main goal is IDE support; a total non-goal for autotools. Unix/Linux C developers largely do not use IDEs. And the autotools goals probably aren't in gyp's list of priorities. So while they both "build the project" they have pretty different goals.

The autotools goals don't always make sense anymore. For example the historical idea that the shipped tarball would only depend on POSIX sh and not on any other binary made a lot more sense when people were hand-compiling stuff on their HP-UX than it does today where most binaries come from distributions.

If you're building a package mostly for Linux and maybe OS-X-as-Unix-compatible, especially one in C/C++ and open source, autotools probably still has a lot to recommend it.

This looks like a really useful guide through autotool hell, which is a fantastic thing to have, but it's not clear what myths are being busted so the name seems like a bit of a misnomer. Perhaps "Autotools Life Preserver" or something would be more accurate.

GNU make has macros too, but these were added much later than 1991; Interesting if make macros were available earlier, would people have used them instead of autotools ?

Thank you for the link on autotools, i will try to apply it if i am ever forced to do something with autoconf/autotools.

GNU Make macros can't really replace "configure".

They can replace "automake". I've never understood why anyone thought automake was useful / sane. Going back 20 years, a few carefully written GNU Make rules were (a) simple, (b) understandable, and (c) portable.

Automake was more work for me than just writing GNU Make rules.

Trying to do anything non-trivial in GNU make macros is the most bewilderingly awful programming experience it is possible to have. They are horribly broken in every respect, from little stuff like awful syntax (whitespace is insignificant except where it's significant!) and poor error checking (detecting undefined variables? Who cares?) to the really big stuff (can't do arithmetic, despite having core functions which take numbers as arguments!).

A while back I wrote a compiler which targeted GNU Make and made it run Conway's Game of Life (complete with hand-tooled arbitrary precision maths library written in make). I learnt more about this stuff than is healthy to know. The compiler output is... special.

http://cowlark.com/2013-10-19-insane-make

Fun story, I have a set of scripts that extract, decompile and process various sets of files, outputting them into a build directory. At first I wrote shell scripts to automatically do all this, then I figured this is the sort of thing Make was built for (right?). This is what I ended up with after days of work:

https://github.com/HearthSim/extract-scripts/blob/master/Mak...

Turns out, Make kind of just assumes you're building a C project and makes a lot of further assumptions which made writing this Makefile a living hell. In the end, I didn't even manage to do proper checks for built files.

Make seems like a really poorly execution of a good concept. I think a large part of why autotools suck also comes from that.

One thing that make appears completely unable to do cleanly is to build things twice, with different flags. You know, like 'build a release version and a debug version'. I've ended up having to write self-writing makefiles to do that. It's not as nasty as it sounds (but still pretty nasty): https://github.com/davidgiven/wordgrinder/blob/master/Makefi...

Plus, 'set a flag for just this file' is another thing that's appallingly broken. GNU Make has an extension to do that --- but if you use it, it makes your builds non-deterministic. (Flags propagate down the build chain; builds are not in a defined order; each file is built once regardless of the flags for that file; so if a file is reachable via more than one rule with different flags settings, it's pure luck which setting wins...)

If you write make files to build everything in twice, release and debug, you're doing it wrong. The proper way to do this is to write make files so you can do "out-of-tree" builds, i.e. your build directory is outside your source directory and each build directory has its own configuration.

This has the significant advantage that you can not just build 2 or 3 or however many different configurations you would think important enough to hard code in the build system from one source tree, but as many as anyone who builds it happens to need.

Incidentally this is one thing that GNU autotools get right.

It is true that if you use target specific variables you have to ensure yourself that you set them in such a way that no non-deterministic behavior can occur; it would be great if make reported a warning in such cases. Another problem with them is that you can't invoke make with a single object file target any more if that requires a variable set on and inherited from the corresponding link target to compile.

Sorry, I don't follow. I am doing out-of-tree builds?

The problem is that there is no way to generate multiple files from one rule. If you write:

    $(OBJDIR)/foo.o: src/foo.c

...you get one rule, which will build one object file. If you want to generate two object files with two different settings, you have to have two rules.

    $(OBJDIR1)/foo.o: src/foo.c
    $(OBJDIR2)/foo.o: src/foo.c

There is no way round this. Target-specific variables look like they solve this, but they basically don't work for anything useful. There are only three options here: (1) hardcode multiple rules; (2) use make macros to generate multiple rules; (3) invoke the makefile multiple times with different settings. All of these options are horrible.

Your flags should be encoded in the OBJDIR name indeed. It is most likely more correct anyway, as you can't always assume you can mix two objects which were compiled with different flags.

An example is C++, where often standard libraries don't allow mixing debug and release code.

> (3) invoke the makefile multiple times with different settings

This is what I mean, more or less. Your build directory contains a generated configuration state (generated by GNU autoconf, for example), and a wrapper Makefile that reads the configuration and builds everything (maybe by invoking a "real" Makefile in the source directory if you're lucky, maybe by having a bunch of ugly Makefiles generated in the build directory too, as GNU automake does).

And (provided you avoid automake) I don't find it horrible at all.

I don't know, Make is difficult because we are used to think in procedures and not in rules; and the rule and macro syntax is a bit weird; it takes time to master it.

i have a makefile/make system that uses gnu make macros; (here http://mosermichael.github.io/cstuff/all/projects/2011/06/17.... ) this saves you from repeating the same make constructs many times over, in the following example you do a static library and executable.

  1: TOPDIR=../..
  2:
  3: # - declare build targets. (built with make)
  4: TARGETS:=shlib slibuser
  5: 
  6: # - slib target is a static library -
  7: shlib_TYPE=lib
  8: shlib_SRC=slib.c
  9:
  10:
  11: # - slibuser target is a executable using slib -
  12: slibuser_TYPE=exe
  13: slibuser_SRC=slibuser.c slibuser2.c slibuser3.c
  14: slibuser_LIBS=shlib
  15:
  16: include $(TOPDIR)/rules.make

At a previous job they had an even more convoluted make system - it was simulating macros: the makefile was including another generic make file; this included make file was writing hidden files that contained the make rules ; then as the last step these generated files were included.

Your link is wrong and returns 404. Even https://mosermichael.github.io/ returns 404. Can you prove the correct link?

Thanks here is the correct link (can no longer edit the parent post).

http://mosermichael.github.io/cstuff/

http://mosermichael.github.io/cstuff/all/projects/2011/06/17...

tyvm

> A while back I wrote a compiler which targeted GNU Make and made it run Conway's Game of Life (complete with hand-tooled arbitrary precision maths library written in make). I learnt more about this stuff than is healthy to know. The compiler output is... special.

One part of me wants to buy you a drink.

The other part wants to scratch my eyes out with a rusty fork.

Since I have no rusty forks at hand, and we probably won't ever meet, I'm just going to enjoy this as new record in "because-I-can"-land.

Please please please do not use ./configure aka autohell.

Study the Makefiles in http://git.suckless.org/

Straightforward & fast.

Autotools solves the problem of having to write a makefile for every platform and also provides all the expected targets without having to write them all yourself.

There is no need to write a Makefile for every platform nowadays.

I confess that I don't even know how to write a makefile, but reading the comments here, should I understand that cmake is better than autotools? Or should I learn autotools instead of cmake?

Since cmake is more adopted, isn't cmake better ?

If you have a need for this kind of thing, learn cmake. If you don't have a need for this kind of thing, consider yourself lucky.

One of the first lines

"...the language used to write the configure.ac is called M4sh, to make clear that it's based off both sh and the macro language M4. "

Tip: don't use Autotools

The Autotools are better than every other supposedly better solution. I do a lot of distro packaging work, and I can say, without a doubt, that the most problematic software to package are the ones that don't use the autotools.

I don't know Autotools, but I find the CMake workflow to be tolerable. Can someone who knows both provide a comparison?

The golden idea in autotools is test for features not for versions. Allow me to explain.

The problem autotools tries to solve is building your software on a wide array of operating systems and configurations. A very attractive alternative approach taken by most, including CMake[1], is to have a big giant database of systems and what each system can do at which version. Thus, if you want to use a library such as, say, Qt, CMake has a Qt module that looks for Qt libraries, checks their versions, and reports that back to you. These databases of system versions and what is available at each version are necessarily always outdated.

Autotools instead gives you tools to write tests. Do you need this library? Which functions do you need from this library? What should those functions do?

When you're running a ./configure script, that's what autotools is doing. It's checking to see what kind of C library you have, what kind of Unix tools are available, how do those Unix tools behave. It is not going to trust version strings or software names. Perhaps the OS reports that it's Debian but in fact it's some derivative of Debian that changed some things around. If you were relying on CMake's hypothetical database of Debian features, it would fail on this not-quite-Debian system.

Over time, these tests get shared around and become part of the core autotools libraries, which is the other problem: autotools is now checking for the absence of certain features that have not really been missing in any system since 1995.

---

[1] CMake can perform autoconf-like feature checks too, but this is not as common as relying on its databases of systems and features.

That was a nice explanation of what is both good and bad about Autotools. I would love to see the GNU project take that "golden idea" and use a better language (GNU Guile ;) because M4 just sucks.

I'm not sure what you mean by databases of systems and features...

I use CMake everyday and for me it's not a database but a set of modules. Each of this module will look for a particular thing. Some modules are better written than other but they all allow you to bypass them. Of course on obscure systems it will fail. That's why they are called obscure. But on systems following more or less the good practice, there will be no problem at all. On the others, you need to pass the specific path, but it will be true with autoconf too... no magic here...

Also in the doc, it is stated that if you rely on a specific flag/option/function then you need to test for it, and CMake gives you the tool to do it. That's a good practice everyone should follow.

But I agree, it's not always done. Like Autoconf, I guess it really depends on how well the developer understand the tool, and how much time he is ready to spend on it.

There's no magic, it's a complicated problem, with only complicated solutions. CMake is more attractive to me, because it allows me to do simple stuff easily, but more complicated operations are also possible.

> I'm not sure what you mean by databases of systems and features...

I didn't really mean a literal database. I meant stuff like this:

https://github.com/Kitware/CMake/blob/master/Modules/FindQt4...

This is pretty typical CMake. It's encoding that for a particular version of Qt, add a few more compilation flags. It's a "knowledge base" of Qt versions encoded as a CMake module.

The autoconf way would be to try to link a minimal program to Qt, and if it fails, try to see if adding more libraries makes the linking succeed.

Or this, for example:

https://github.com/Kitware/CMake/blob/master/Modules/FindOpe...

It is trying to account for two popular compilers on Windows but not with Cygwin. It's the whole chain of "if Windows, if cygwin, if mingw, if msvc" what I called a "database" above.

Doesn't clang work on Windows these days? What will this knowledge base do now? Shouldn't clang work almost like mingw? But because it misses the "if mingw" branch, code that probably would have worked for clang will be skipped over.

I'd argue that for Qt this approach makes sense because they provide a well defined API that they stick too. The developer can also ask for specific component if it just want them (like QT GUI, QT SVG...).

I agree that this is more problematic with less common, more buggy libs where the API keep changing, but in this case it's quite easy to check for specific files, specific functions, ...

All this assumptions have huge consequences in terms of velocity and ease of use.

For your second example, this does not test for the compiler but for the provider of the SSL library. Clang does not provided it, so it's not a problem.

But I get your point.

My counter-argument is that more flexibility is left to the developer to assume stuff. These assumptions make things easier in most cases. In corner cases, then you need to provide explicit values to the module so he can find what you need.

> The Autotools are better than every other supposedly better solution.

... as long as you don't care about Windows (or iOS, or Android, etc.)

Autotools works for Windows and Android NDK builds.

No idea about iOS.

While I also spend my day compiling packages [0] -- well CROSS COMPILING packages, and the worst problem I have, bar none, is autocrap.

Not only is it a giant pile of poo to try to 'repair' when it invariably does something the wrong way (like trying to fish my host's headers), but when compiling a distro, it's also the single tool that takes the most io/cpu of the host; forget compiling C++ - when you add up the zillion's time autocrap tries to see if I have strtok() in my C headers (hint, I do, since 1991); and then doing it again, over and over and over and over, the /compile/ time becomes rather negligible in comparison to the /autocrap/ time (especially on a modern host with -j12).

The thing is, in MOST CASES you could do it all with a page of gmake. pkg-config and all that are excellent at removing the need for autocrap. gmake has made heaps and bounds at simplifying what used to be the nightmarish Makefiles.

[0]: I'm the author of https://github.com/buserror/minifs

Oh brother... this would have been good when I was developing in LibreOffice!

there is only one myth that needs busting. that these tools solve any real problems that can't be solved better.

Just what I want, a 900KiB configure file that doesn't even work. I'm looking at you lame! And then a make process that doesn't STFU!

http://freecode.com/articles/stop-the-autoconf-insanity-why-...

Reply on news.ycombinator.com