What autoconf got right
leahneukirchen.org
leahneukirchen.org
This is just a long-winded way of saying it's popular. It's popular, therefore people know it, therefore they view whatever it does as "the standard." Kind of like... MS Windows?
It is based on checking features.
The feature-checking approach is slow and makes cross-compilation unnecessarily difficult. Plus a lot of the "features" that auto-conf checks for are things that haven't been an issue for decades, like "sizeof char"
... autoconf is popular.
Cross-compiling with autotools seems like asking for trouble, but I bet it does do that. It even runs on Windows. I would discourage anyone from using it in general, but as a user it rarely causes problems for me.
[0] https://www.cs.tufts.edu/~nr/cs257/archive/glenn-fowler/iffe...
[1] https://github.com/att/ast/blob/master/src/cmd/INIT/iffe.sh
[2] https://github.com/att/ast/blob/master/src/cmd/3d/features/s...
1) We do not need to support old buggy platforms, we can generally assume that platforms are open source and their bugs can be fixed, so it is less pressure to workaround them in application software. Consequently, we do not need that detailed checks
2) We can assume there is fully working POSIX shell, so configure script and its checks could be made just in shell, no need to have M4-generated shell code.
3) It is probably better not have separate 'release-phase', so end-users could just work with code from git. Users could run Autoconf themselves to generate the configure script, but the Autoconf is not well-suited for this, as there is lack of backwards compatibility.
4) Most developers do not know M4, so they just hack around Autoconf macros without proper understanding.
5) It would be nice if all these directories that are given to configure scripts could be overridden at runtime using environment variables
Ideally, there could be POSIX shell script library that could be used to write portable configure scripts offering all the standard options and checks like ones generated by Autoconf.
Kids those days.
Haven't we learned a long time ago that we shouldn't assume anything ?
Autoconf has set the baseline so low that everyone perpetually pays for the technical debt of ancient buggy platforms. Sometimes it's better to push a buggy platform to fix its own bugs, than to have everyone work around those bugs forever.
Then guess what, I'm not building software for you
From the article:
> Of course, the checks should be reasonable (and in practice, often are excessive). If your code assumes a C99 environment, you don’t need to check whether all C99 functions you use are available.
Which makes sense to me (including the check based approach).
This assumption cannot be made, when ~75% of user machines and ~60% of developer machines are running Windows[1].
[1]: https://survey.stackoverflow.co/2023/#section-most-popular-t...
The moment I see 'build.sh' in a library's or application's source tree, I am certain the build system is not portable and I look for something else.
> Then guess what, I'm not building software for you
I think people on HN really like not capturing significant chunks of the user population, in a bid for an unrealistic and impractical ideal.
Autotools certainly won't give you that. Maybe inside CYGWIN or WSL and even then
> The moment I see 'build.sh' I am certain the build system is not portable
Obviously. They usually ship a build.sh and build.bat or something more modern.
The moment I see a configure script I know the developers are still in the 90s
"but the buildsystem is not portable" well, I have bad news for you regarding the rest of the fsking owl.
One additional point: autotools are bootstrappable without going all the way up to C++ (CMake, ninja) or Python (meson).
And yes, muon and samurai are written in C99, but as soon as I wanted to both support -fvisibility=hidden and unit tests in my project, I was forced to build both static and shared libraries. This forced me back onto libtool, and once I was there I figured I may as well stick with the good ol' autotools.
GCC is written in C++.
So on a modern system you are going to have to use C++ anyway.
But also, I still don't have an answer to "I want to build with -fvisibility=hidden and also have unit tests" that isn't "libtool".
You do need Perl though.
As is the fact that 99% of Linux software defaults to hard-codes absolute paths and spewing its files all over the system, rather than installing to a single directory that you can freely relocate.
They turn out to be bringing fairly little benefit (minuscule disk space saving) and giant drawbacks (the vaunted "your software will be upgraded / bugfixed behing your back" which instead turned out to be a configuration nightmare and an exponential multiplication of the attack surface for would be attackers)
To this day, I dread to try an run an old binary that was linked against shared libraries say 5 years ago on a modern system.
Chances of it working are quasi zero, and good luck trying to reconstitute the tree of shared libraries that are recursively required to get that binary to run, glibc often being the worst offender of the lot.
I don't think many people have problems with the idea that core system libraries could be dynamically linked. But Linux distros take this to a ridiculous extreme. I absolutely do not care if a bug in libgmp means my package manager updates the 2 apps I have installed that use it instead of 1 library.
Would be really interesting to see some actual stats about this though. How many library projects do people have installed that are depended on by more than say 3 apps? I'm guessing it's under 100.
I think that's the approach Flatpak takes with it's "runtimes" and it seems like a very sensible one.
I think that if the library is independent upstream project, then there is no reason why it should not be dynamically-linked independent package in distributions.
Surely that's automated?
Doesn't matter. You'd rather update X separate packages (of unknown size & potentially different maintainers or update policies) to fix 1 bug in a library they all happen to use?
Shared libraries are a good thing, period. Implemented poorly? Fix that instead, rather than include everything & the kitchen sink in every single app.
Drew DeVault did some tests on this subject and published the results here:
Of course these days storage and RAM are both massively cheaper, and there are a plethora of variants even for what might previously have been the core version of something, so the benefits are minimised.
We've seen similar with JS libs over more recent years: first everyone had their own copy, costing bandwidth and meaning every site/app dev had to update their version when there was a security issue. Then CDNs became a thing for libraries & such, reducing the bandwidth cost and (unless you pinned at a specific minor version) making security/efficiency updates easier. Now as shared sources for JS and the like that are potentially a privacy or security issue, and treeshaking & such can reduce the payload for most uses, we've swung back and (almost) everyone has their own version, as-is or crunched into a bundle.
Package management, the likes of which wasn't available in the 90s, deals with the updating of JS libs even if everyone has local versions (they get updated on next build, unless holding back to full test a new release), but that in itself is a security concern due to supply-chain attacks.
Or in other words, the perceive simplicity of static linking comes form pretending that something is now a stable dependency you can freeze that does not need to update or which you can update at your own schedule. This is not true because of security issues.
The advantage that you can run the same binary independent from the shared library context even in ten years is only true if you do not care that security issues in the binary remain unfixed for 10 years.
- flat namespace, which you can sort of work around in some cases with dlmopen() but...
- ...glibc's subordinate libraries (pthreads) don't like being in the same process as different versions of glibc, and loading two different libcs in the same process creates problems
- There isn't a strong check for ABI compatibility at startup, the best you can hope for is soname being set and symbol versioning
Symbol versioning (especially auto-versioning) with proper sonames and ABI stability help a lot, but there is still the problem that everything needs to use the same glibc within the same process.
There's also static linking, but it has its own problems, and at that point you're just doing shared linking (plus some optimizations) and copying all your dependent libraries that all still had to be compiled with the same system library, hopefully in a way that integrates with the whole system (but probably not).
I took a different approach with AppFS [0] than traditional package managers:
- no install step for packages - since there's no install step, you can have as many concurrent versions available as you want
However, the hard problems remain. A concrete example is my library CACKey [1], which is a PKCS#11 module. PKCS#11 dictates shared objects (so no getting around it with static linking) and that the library should be thread-safe, so to deal with threading it was written to use pthreads primitives (mutexes). However, if you try to load pthreads in a process with a different version of glibc, pthreads aborts. I tried to fix this with dlmopen() [2], but this refuses to load a different version of glibc (IIRC). So, your best bet is a package manager that compiled everything in the system with the system libraries.
[1] https://CACKey.rkeene.org/
[2] https://cackey.rkeene.org/fossil/file?name=build/libcackey_w...
My other 2c are that languages now have their own package managers is far bigger indication of the huge failing (of the overall ecosystem). apt-get is a blessing compared to npm, pip, cargo, and all that horrible garbage,.
1. They aren't cross-platform. NPM, Pip and Cargo all work identically on Mac, Windows and every Linux distro.
2. They are too heavily gate-kept. I can upload a package to PyPI, NPM or Crates.io without having to convince anyone that my project is worthy. This is a very good thing!
If the Linux community had made a packaging standard with similar properties maybe those languages would have used it.
It's a good thing for me because I can upload my ransomware disguised as a file manager and people won't know it's ransomware until they run it. It's not such a good thing for my victims because they need to do their due diligence for each project instead of having a trustworthy maintainer vet it for them.
1. Maintainers don't thoroughly vet packages. You should know this from a certain recent event that was all over the news!
2. Projects that don't want to deal with the main apt repo just set up PPAs and tell you the commands to copy/paste to add their keys and the PPA.
Maintainers are a first line of defense. They are users of the software just like us, and share the same interests. This means that most obvious misbehavior (not necessarily malware but just anything anti-user or just general bugginess) is caught before it has broader impacts. The fact xz made it into the news is because it was an anomaly, and even then its impact was reduced to only the unstable versions of debian and such.
As for PPAs, you're right, adding a new repo means trusting a new set of maintainers. Thankfully only few software needs this so the amount you need to verify is relatively small.
PPAs at least require the user to explicitly trust the source of packages. If a distro releases a package, that is a tad more trustworthy than a random PPA, because at least the distro maintainers (very experienced people) thought it was good enough and not a virus. The distinction between "official" and "unofficial" is not there when it comes to all those language-specific repos. Also, typo-squatting is not a thing with official Linux repos, because maintainers know to not let it happen.
2. This is not a good thing in terms of security, long-term stability, and quality in general. But there should be a two-level system where you can upload easily so that other people who are careful can explore but then there should also be a quality controlled system like apt.
So yes, both points are true but I feel cross-platform package manager should be based on apt or similar with multi-level quality control, and npm, pip, and cargo and other low-quality attempts at package management should die IMHO.
Generally you can install programs into arbitrary directories, but I think those directories aren't meant to be moved. That is not a build system or packager issue. To make a program or library load from an arbitrary location, it must be linked the right way, and your environment has to be right. The only semi-practical way out is to statically link everything, which wastes a ton of space and creates other issues related to updating software.
I used C/C++ for years, and it was always a nightmare to build something, unless it was completely self contained. When your "packaging" is so bad that everything has to be vendored, that should be a red flag.