Hacker News Reader

Comment by nuxi7 | Hacker News Reader

The distributed form of a project (especially one based on GNU autotools) is often not merely a tarball of the upstream repository. Often the author has pre-run some stages of the build that she feels would impose unnecessary or esoteric dependencies. Typically this involves pre-generating Makefile.in from Makefile.am and configure from configure.ac. Another common one is for the maintainer to pre-generate the documentation. The user of the distributed tarball is fully capable of re-running these steps, but they have become optional.

This is a fact often forgotten when projects switch to github and just start using the automated release tarballs github will make from the repo.

View on HN

Exactly.

Many web tool chains involve pulling from external repositories such as npm, bower or even github itself. If one of these are down you cannot deploy your application.

Even better many packaging systems such as npm or setuptools allow you and library maintainers to specify a flexible version numbers for your dependencies. If during the course of your build, test and deploy chain one of these dependencies changes your application could break through no fault of your own. You cannot rely on maintainers to not release breaking changes in minor versions, it happens all the time, intentionally or not.

At least for npm there is a solution for that problem: https://docs.npmjs.com/cli/shrinkwrap

This becomes more complex when distributions want to patch configure.ac or Makefile.am -- they need to patch the included Makefile.in files as well.

Trying to balance these two forces is the reasoning behind the AM_MAINTAINER_MODE flag.

These days, I prefer to simply use git archive for a tarball and make users run autogen.sh themselves.

See https://blogs.gnome.org/desrt/2011/09/08/am_maintainer_mode-... and the associated comments.

Yes, incluinding the maintainer-mode option can be important, which is why this line should be included in configure.ac most of the time when using automake:

    AM_MAINTAINER_MODE([enable])

This includes the --disable-maintainer-mode configure option, while leaving maintainer mode on by default which is the traditional behavior you get when you don't include that macro. This shouldn't change any behavior for most projects.

You can change the default to off with the same macro, if that is important; the key point is to include the macro so it can be changed by the automated packaging scripts used in some distributions.

I recommend Autotools Mythbuster[1] for a nice overview of how to use modern versions of autotools, including maintainer mode[2].

[1] https://autotools.io/

[2] https://autotools.io/automake/maintainer.html

To the newcomer, autoconf looks really scary. Are there any good tutorials and are there clear advantages of using autoconf over CMake, waf, scons, ninja or other hipster-language-build-systems?

I'm going to start looking at CMake to build HTML and javascript based projects soon.

I consider myself rather adept with Makefiles for Gnu make, and I also find autoconf scary (I'll be checking out the "autotools mythbuster" linked above though).

I'd guess that autotools and cmake have a lot of complexity to do C/C++/libs related stuff that is inapplicable to html/js projects, and I'd suggest almost any of the others: waf, scons, plain make ...

ninja, as you've probably heard, is designed to be the backend to some other frontend which generates all the explicit dependencies and commands. That could be cmake, it could be a custom script of yours. Then ninja identifies changed files and runs an "incremental" build as fast (parallel and minimal) as possible.

Just btw, I wrote a from-first-principles sort of tutorial on Gnu make: http://www.ploxiln.net/make.html

I found Klemens' _21st_Century_C_ has a decent intro to basic autoconf setup.

CMake has been a huge improvement over autoconf and over visual studio projects/solutions for me. I have a repeatable and multiplatform build system with CMake. Autoconf seems like the ugly and old bastard brother of CMake in comparison, and that's praising autoconf.

However I don't know if CMake is really the best fit to build html projects. In fact, I don't know what should be built about them... my composer scripts barely copy some JavaScript and CSS files to the public/assets folder, nothing fancy like what CMake has to do.

Unrelated, but this reminds me of 10 years ago and the horrors of running

    configure && make;

And watching it implode most of the time. Running a configure script to generate a makefile to build software makes me glad for binary distributions.

I have been very happy with how will this has worked over the last few years. These days it's a rare surprise when I download software and it doesn't compile easily. Of course I much prefer to use an up-to-date binary from the package manager, if one is available.

That's largely because you still have the package manager to resort to in order to take care of most dependencies. Prior to that, you'd still have to install (a lot of) dependencies by hand. Not all of them used autotools, not all of them used standard install locations by default... it was pretty atrocious.

In fact, I remember that one thing which kept me on Debian for a long time was that it had so many available packages. Back then Debian was even slower than today and many of those packages were a little old, but it didn't really matter to someone who was just discovering things.

It's easy to forget about them, but package maintainers are incredibly important in today's Linux (and *nix-inspired in general) world. There's a lot of hard work behind the fact that you can just apt-get whatever new program you found out about.

What I find hardest (as a manual installer) is those packages which were designed with a package manager in mind and end up depending on many third party packages rather than incorporating the needed functionality.

As an example, program X might need to compute a statistic like the skewness of a distribution. A stats package might implement the skewness calculation, so the author of X adds the dependency and calls the right function.

However, building the stats package might require a Fortran compiler and some matrix libraries, since the stats package does much more than compute skewness.

Another approach would be for the author of X to incorporate the few tens of lines needed for the skewness as part of X. This is of course more work for the author X, so I can well understand the desire to simply declare it as a dependency.

Yes! Thank you for explaining this. I don't think a lot of people realize what makes up a good release tarball.

To provide a little more information: the standard Makefile target for generating tarballs is `dist` (which all GNU projects should implement). For those curious, go take a look at some of your favorite projects that implement this target and see what the resulting tarball may result in, versus the contents of the repository.

https://www.gnu.org/prep/standards/standards.html#Standard-T...

To add to this, this is the purpose of the "maintainer" targets generated by automake. To fully clean an automake project so everything is rebuilt, you want "make maintainer-clean".

Running the more common "clean" or "distclean" targets leaves not only the generated Makefile.in files, but also generated source like the generated .c/.h files from a lex/yacc parser. This is important behavior; while C compilers are available everywhere, having bison and flex (or ANTLR, or any other less-common code generation tool) installed isn't as common.

Another issue with GitHub's automated release tarballs is that they do not include submodules. Though, I've also found that submodules are often overlooked when cloning.

I got bitten by this a couple of days ago - I wrongly assumed that tags on GitHub were identical to the tarballs offered on the project's website. Not so much, as it turns out: https://github.com/Tarsnap/spiped/issues/26

Reply on news.ycombinator.com