Large Unix programs were historically not all that portable between Unixes | Hacker News Reader

Large Unix programs were historically not all that portable between Unixes

utcc.utoronto.ca

74 pointszdw43 comments

Reading this reminds me of the days in the 90s when programs could target 10+ different Unix systems and shipped as giant tar balls of code. The C code consisted of giant #ifdef <UNIX VENDOR> blocks to accommodate how different Unix systems might handle a couple system calls.

It wasn't uncommon for the wrong block of code to get included so you had to figure out how the compile target's system call worked, how your system's system call worked and modify the code accordingly to get it to compile (and hope it worked). This was the 1990s so getting code to compile on Linux wasn't a high priority (since Linux was still in its infancy).

Fun times.

> in the 90s when programs could target 10+ different Unix systems

Fast forward to present: programs can target 10+ different Linux systems.

Fast forward 30 years to the future: programs can target 10+ different Ubuntu systems.

What if I told you that today we can build executable files that run on five different UNIX systems and we don't need #ifdefs?

Any 5 different or especially selected 5 different ?

In the 90’s I worked for a company whose C/C++ code needed to run on at least 7 different Unix versions: SunOS, Solaris, two different AIX versions, HP-UX, DGUX, and whatever Motorola’s Unix was. The slogan around the office was, “There is no such thing as portable code, just ported code.”

In the 2000s, I worked for a company that shipped an HPC NIC (think precursor to infiniband). We had userspace software AND DEVICE DRVIVERS that supported Linux, FreeBSD, Solaris, DEC OSF/1 (Tru64), AIX, Windows and Mac OSX.

The device driver code had a shared component that supported the hardware, via an OS abstraction layer over top of the various OS specific ways to do things. The driver did fun things, like pin userspace memory for RDMA, which was often outside of supported driver interfaces.

The Windows, and the closed source commercial UNIXes were a PITA to bring up initially (AIX was the WORST). But once supported, they were easy to maintain because the interfaces never changed. Linux was fairly easy to write, but a PITA to maintain because the kernel interfaces changed so much. We had a configure style shell script to probe the kernel headers and detect how many args certain functions took on this particular linux version, etc. That script by itself was larger than the entire FreeBSD driver. (and FreeBSD was as easy as linux to write, and as easy as a commercial UNIX to maintain, since the interfaces we used didn't change).

One interesting thing that fell out of it is that we used ioctl interfaces everywhere (even Windows and MacOS). Since this was an HPC device, latency was critical and the Mac ioctl routines were painfully slow (like 4x slower than Linux/FreeBSD on the same hardware). So I tried switching to the Mach based IOKit IPC that Apple wants you to use, and that was twice as slow as ioctls!

A bit more than a decade ago, I was on the receiving end of a C level (CEO/CTO that is) decision to target Linux, BSD, Solaris and ... Windows, for a C++ application.

I think I can imagine your pain.

I built rn on a number of different systems at the time, and I was fascinated by the configure/build's messages while it tried to figure out what os & hardware it was running on.

Also "This program posts news to thousands of machines throughout the entire civilized world. You message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing."

I was kinda excited when I first saw autoconf. In the "good old days" porting software that academics wanted on the local Unix box was a genuine exercise. When you see all the differences that the "configure" command comes up with it gives a little twinge of nostalgia for when even things like the version of libc available wasn't in the expected place. Trying to get stuff running on an Apollo or a DEC 3100 or a Pyramid was fascinating and frustrating at the same time.

I wasn't. I have always felt autoconf was the wrong solution because it gives in to the problem.

Uname -a having a build hash, and an index of hash to settings would have done better. Think of all the wasted CPU cycles computing how this chip orders a long long and testing POSIX compliance one function at a time.

Autoconf only fixes the apparent problem. Not the underlying disease.

At least X11 tried encoding the known patterns.

> uname -a

so whenever you upgrade the kernel or change CPU all your programs need manual patching? you may answer, "well uname -a was a joke, you would actually check the libc version". this worked alright in a very specific time period with a couple of big vendors with nigh-unmodifiable toolchains with well-defined release versions. it became increasingly awful when people started making their own toolchains with free software and had to patch every single piece of software to properly detect their new libc. sitting through an hour of autoconf sucks, but slogging through ten hours of patching poorly written bespoke configure scripts is a hundred times worse.

edit: not to mention cross-compiling where you can't even run uname -a, and which was almost universally broken before autoconf, which, despite being painful, made cross-compiling mostly work everywhere.

That would move a lot of work onto whoever first stumbled against a new combination. The magic of autoconf, when it works correctly, is that you don't have to have a configuration that has been seen before.

I can't count how many times I end trying to run autoconf only for it to tell me I have the wrong version of autoconf.

This is a general problem with modern SW. You always need the right version.

Hindsight is 20/20.

It's not that it can't be done, it's that the community went in a different direction, and there's a strong cultural bias. Bell Labs internally had a "new make" written by Glenn Fowler, that went full bore into tracking dependencies, probing local and global resources, recording configuration state at build time, that we adopted in Graphviz. It was powerful and concise. It was also a heroic effort for Glenn to maintain and develop, though others were assigned. https://ieeexplore.ieee.org/document/6771042

At one point in our project we had a lot of arguments about build systems and software repositories (there were times when we had to stop talking about it), then at some point John Ellson said, look, everyone else is using autotools, so do you want them to use your software? We realized he was right.

Individuals in the software community are capable of building a lot of amazing tools. The difficulty is persuading the community and teaching non-specialists to use them. They already know how to run make, or ./autogen.sh; ./configure --prefix=$HOME

Despite this Graphviz still has 3 or 4 build systems: autotools, Visual Studio (probably 90% of our audience is on Windows though we try not to think about it), Xcode (needed for an under-maintained MacOS app), and Magnus Jacobsson and Matthew Fernandez have a lot of a Cmake build working, though not yet complete.

Apologies for rambling a little off topic.

If there's one thing you never have to apologise for on HN it's going off topic into trivia like this.

That's true, now we can go back and simplify, I was more speaking to the origination of all of these horrible things. Nobody knew when they picked 2048 instead of 1024 it was going to matter, but then suddenly it did and nobody wrote down a good way to probe that API.

I remember this too. First year of university in 1991, me and my friends who'd all gone to different places used to write actual letters on paper and post them to each other. I was on the internet for the first time, and was excited to be able to download pictures! I could print them out and include them in the letters. If I remember correctly I needed to compile some kind of image viewer/manipulator on my account on the DEC Mips machine we CS students had access to. Anyway, as you say it was amazing watching the autoconf script feeling its way around and discovering its environment.

The message if you were running Eunice:

https://github.com/Perl/perl5/blob/perl-4.0.36/Configure#L10...

A couple decades ago I ran an NSF-funded project to develop and run a public, open-source automated built-and-test platform for ~50 distinct versions of a dozen or so unix OSes (plus a token Windows version or two), long before modern CI platforms made this commonplace. We needed it for our own software and "productized" it for the wider scientific community. It was a ton of fun, and a beast. The diverse hardware in our test lab was like a zoo. Linux consolidated the industry and made the whole thing unnecessary shortly after I left.

I remember one C book that described a way to write portable code. There were three phases in the header files: (1) The vendor, (2) The characteristics, (3) a portable layer on the system calls. So the first would define VAX, SUN, etc. The second would use the first section to define LITTLE_ENDIAN, BIG_ENDIAN, HAS_IOCTL, etc. And the third would implement a portable layer on top of the system calls. It was nice to compile because section 1 would often be defined automatically by the pre-processor, and the rest would just be defined properly. No config was needed to compile - you just types "make".

Alas - this was extra work on the developer, and at the time there were a zillion variations, which a developer had to deal with.

Of course not, POSIX portability leaves lot undefined across UNIX platforms, it was mostly the C runtime that ANSI/ISO C didn't want as part of the standard than anything else.

Back in 1999 - 2003 I had lots of fun porting code between AIX, Solaris, HP-UX and Red-Hat Linux.

A decade earlier the fun was with Xenix, DG/UX, and early GNU/Linux flavours.

And it wasn't only the OS specific APIs, each OS SDK had their own glitches and deviations from C standard.

In my first job out of college was supporting monitoring software in big financials. They often didn't even know what they were running.

More than once got sat down and told to connect to their HP-UX, AIX, Solaris, Linux, etc box and setup things up.

Would have to go back and explain that particular server was actually something different and I'd put the appropriate build on it.

Of course, every system had to have it's own build, and own scripts to accommodate all the differences mentioned here. University CS courses had definitely not covered things like GNU vs BSD tools, and various other differences.

I remember being advised early on in that job to learn vi, not because it's the best editor, but just because it's the only one you can rely on being everywhere, on every server. There were other discussions in the office about which editor for actual dev work, but for client visits on random servers - vi is always there.

The old commercial Unix vendors killed themselves by repeatedly creating forks whenever unity threatened to break out, letting Microsoft eat their lunch. Every attempt at unification was fought off.

The word "portable" refers to the distinction of using assembly language programming versus a language like C, which with little modification can compile clean on the target OS...

I agree that to take a program from one unix and make it work on another is a tedious task, but the difference is minimal when you compare portability of architecture specific assembly language code. It is the reason "portable" languages like C were created.

Historically? What changed? Everything is Javascript now? There's only one Unix now?

Portability libraries like glib are better at hiding the differences, at least for simpler C/C++ programs. Also (sorry BSDs), there is kind of only one "Unix", which is Linux. Even if you consider Linux + BSDs + macOS, that's a lot fewer combinations than in the good old days when you might have considered a port to Solaris, HP-UX, SCO, AIX, OSF/1, ... all of which were very different.

Porting programs to Windows still remains a challenge.

>Solaris, HP-UX, SCO, AIX, OSF/1

Just to make the problem more clear, add:

Irix, Ultrix, SunOS, DG/UX, Pyramid OSX, NCR Unix, Xenix, A/UX, Bull DPX, EP/IX, ISC, MachTen, NeXT, Dynix, UnixWare, ConvexOS, Coherent, DomainOS and more...

See the supported platforms for something like Kermit or Pine to see this isn't just piling on. Some real and popular software had to support huge lists like this. Perl's CPANTS testing network was neat for stuff like this, you could write a module (including C code), and watch all the various ways your code would bomb on other platforms without having direct access to them.

http://www.columbia.edu/kermit/unix.html

http://www.mit.edu/afs.new/athena/astaff/source/src-9.0/thir...

>There's only one Unix now?

Not quite that, but there are certainly a lot fewer with notable amounts of use. Which means, in turn, fewer compilers with unique flags...you can get away with supporting just gcc/clang.

And there are fewer machines that are big endian, or have unusual byte alignment, or have a buggy built-in malloc, or don't have off_t defined, or have unusual flags to link to pthreads. And so on...all the stuff things like Configure test for.

Autoconfig exists for a reason!

it works pretty well if you stick to C and target Linux and want to support a multitude of distributions. As soon as you throw something else into the mix (C++, or OSX or ...) you're heading for a world of pain.

Sounds like you're saying it doesn't work?

But as the essay notes:

> they gave you the tools that you could use to sort out how to achieve that, and to automatically detect various things you needed to do to adopt to the local Unix

autoconf doesn't do much on its own, it's more of a toolbox, if you don't leverage the tools it gives you, you won't achieve any sort of significant portability.

No. It works, but it's painful. Let's say m4 and sh are not the most comfortable tools to work with. These days, there are better ways. (CMake or Scons fe)

I would say the not popular opinion around here that applies to most tools on UNIX, and I have used plenty of flavours since 1993, when I got introduced to it via Xenix.

Isn't it true for all programs? Not just _unix_ programs?

Firefox is portable and it's a large program, but a LOT of effort went into that.

> Isn't it true for all programs? Not just _unix_ programs?

That's irrelevant whataboutism.

TFA is very specifically in response to one pining about older unix ecosystems and seeing them as havens of portability, and explains that that was not the case at all.

> seeing them as havens of portability

Was that in absolute, or relative terms? Since portability between OS/360 and TOPS-10, for example, was non-existent.

Not only. The PDP-11 had 5 (6? 7?) different and incompatible OS options available for the same instruction set and hardware.

Which - of course - varied somewhat different across different models.

I don't know who ever thought Unixes were portable? None of them worked the same... If you were lucky maybe they'd be running the same architecture.

Are commenters on this site functionally unable to click links anymore or something? Literally the first link in TFA's body text is the take they're replying to.

POSIX programs were portable enough that Microsoft had to become POSIX compliant before being granted the operating system monopoly rent by the free market.

"Portable" until you rely on signals behaviour, file systems, threading, file locking and other stuff left for the implementations.

Windows NT POSIX subsystem was for ticking check box on DoD contracts.

Had Microsoft been half serious about POSIX in NT like they are nowadays with WSL, Linux would never had taken off.

Reply on news.ycombinator.com