Hacker News Reader

Comment by lutorm | Hacker News Reader

I must admit to being a bit puzzled as to what his point is.

Yeah, there are a lot of dependencies if you want to compile an entire linux distro from source. But it's just exposed because everything is right there. If you actually tried to figure out how to compile all the software on your Windows machine, would it be any better?

And to say that libtool, bloated and complex as it is, came about because someone was incompetent seems quite a bit insulting. It seems that confronted with a compatibility problem between different Unix-like systems, one has two choices:

(1) Coordinate changes to all Unix-like systems so the compatibility problem is removed, or

(2) Work around the problem with some sort of wrapper.

Now, in a perfect world, (1) is obviously the better alternative (but even that wouldn't help existing installations.) But the world is not perfect, and my chances of accomplishing (1) are practically zero, even if it's "just a single flag to the ld(1) command". Hence, anyone who actually wants to get something working on all those systems would have to do whatever it takes.

View on HN

His point is someone has to stand up and say "This is how it will be done." Without that you just get tons of wasted effort.

Why do so many developers use OS X instead of Linux? One benefit of OS X is that there is one entity responsible for it, not 250. There is a person (or a few) making the decisions, proving stability and removing things that don't work. It's not perfect (recent transitions in OS X as leadership shuffled) but the different parts of OS X usually feel like part of the same whole.

As much as people 'like' the bazaar I see the same things in open source. Current init systems give you tons of choice and people keep screaming because it's being removed. But a lot of people seem to like having it organized under one umbrella so that it can all coordinate properly. Doesn't matter if you don't like the choices being made (many people don't like Apple's choices) but there is someone in charge.

There are other examples. Ubuntu seemed to get popular saying "This is what it will work like" as opposed to what used to be common of "Here are 6 GUIs, choose one!". Ruby (with Matz), Python (with GvR), Clojure (Rich Hickey), the Linux kernel all seem to be doing pretty well with someone nominally in charge to make hard decisions.

I think his example of build systems was perfect. No one would set out to design a system to generate a file to probe a system for 26 programs that you don't even need to choose build options for a program that only runs on a handful of nearly identical systems.

But the point of libtool or autoconf is not to make it easier to build various programs on Ubuntu, it is to make it easier to build the same program not just on Ubuntu and RedHat, but also on FreeBSD, OpenBSD, Darwin, Solaris, SunOS, Tru64, HP-UX, IRIX, AIX, and god knows what other systems. Even if Linux turned into a perfectly polished and neatly organized cathedral, you'd still need to get it to work on all those other systems. And most of those systems predate "the cathedral and the bazaar" anyway, don't they?

libtool and autoconf are messy partial solutions to problems created by writing non-portable code. This is an aspect of the bazaar model, where reasonable standards (e.g. POSIX) are sometimes ignored in favor of novelty or minimal adequacy.

The bazaar model embraces the mess, believing that something spectacular will emerge. The cathedral model argues that coherence produces better results.

Both can point to successes. phk is arguing that the open source community has lost more than it has gained from accepting the bazaar as the one true way.

Even in the last ten years, where has the innovation in operating systems come from?

The CatB essay predates the book, and the idea predates UNIX.

Any non-trivial code is non-portable between Unix systems in practice. No system implements all of POSIX, even when they do they often disagree on how parts of it should be implemented, and some software needs features that simply aren't in POSIX. (For example, high-performance webservers need a better way of polling for events than select/poll, and that's platform-specific.)

As for libtool, anything that deals with shared libraries is inherently non-portable. The file extension they're created with and the compiler and linker flags to create them are entirely platform specific - POSIX won't help you there. (Also, a lot of libtool's hairyness exists because some older proprietary Unix systems don't support shared library dependencies.) If you're only planning on supporting open source OSes you don't actually need libtool though - their native toolchains are either the GNU one or something compatible with it, and include most of its functionality natively.

And this is because there is nobody with authority to force the right solution.

Compare POSIX to the Java spec: without Sun forcing coherence, "write once, run anywhere" would have died very early on, and we'd be saying the same thing about Java as you've just said about POSIX.

Curious though, what do you use: maven or ant?

Neither.

> webservers need a better way of polling for events than select/poll, and that's platform-specific.

POSIX was just an example, and it is possible to write portable code using #ifdef. But you have to have a goal of writing portable code.

Re: libtool, most of the reason it's ever necessary is because of the historical rush and the incompatibilities created. Again, a single command line switch could handle that.

Still, plenty of software which has clearly only ever been built for Linux requires libtool. It's just the way things are done, just like autoconf. It works, poorly, inefficiently, fragile-ly -- and to most, mysteriously.

Appreciation for autoconf and libtool depends on one's position on the defeatism-pragmatism-idealism spectrum.

"POSIX was just an example, and it is possible to write portable code using #ifdef.."

Yes, two ways. One way uses autoconf (and tests each individual option to see which branch of the ifdef to use) and the other uses Imake (and has a big, continuously maintained database of which option to use on each system).

How much easier does autoconf make things, really?

I think the key phenomenon here is that software is non-portable by default, and portable through effort. If your problem is that the Linux code should use epoll() and the BSD code should use kqueue(), but OS X has quirks in its kqueue() implementation, and you need to fall back to poll() otherwise, autoconf will help... slightly. Then you need Sys V init scripts, or rc scripts, or launchd scripts, or... who even knows, unless you actually convince someone to port your project to that system.

So, you already have people testing and fixing bugs on other platforms, but now you tell them that they need to also learn M4 so they can write an Autoconf macro to test for some... thing... that you need which doesn't have a standard test. I'm not convinced that this makes my life any better.

Whether or not you like Autoconf, however, you have to agree that libtool has serious problems.

"Build systems", like the GNU one (autoconf/automake/libtool/etc), cmake, or others, theoretically "solve" the need of your project build in a newer/different platforms. However, in practice, in a newer OS "version" or newer platform, you have to "fix" things, again.

In my opinion, when possible, it is better to stuck to standards, and use just a Makefile for POSIX targets (it allows context detection) and a custom scripts or Makefiles for exotic platforms.

In the end, the reason for e.g. Autoconf, is the mess of different non-standard crazy stuff. Why not to agree a build standard base, at least for the C language and library handling?

It's a nice, principled idea, but what happens when someone says, "We'll give you lunch money if you'll make your software work on ..."?

Also many of these bugs are somewhat transient, and it is better to not support a buggy platform, or insist on manual patching until it is fixed, rather than build complex tests and workarounds in.

"How much easier does autoconf make things, really?"

Speaking as someone who was a system administrator responsible for compiling a large number of packages across a wide variety of platforms (let's see, SunOS, Solaris, AIX, Irix, Linux, and HP-UX, all at different versions at different times) through the '90s and early 2000s, autoconf made things much, much easier.

It doesn't help the developers any; each individual difference has to be identified by the developer. (My biggest complaint isn't that it's written in m4, but that there's no comprehensive list of what I need to do with it. That information makes up the m4 tests, but there was no documentation.)

Don't like libtool? http://www.fortran-2000.com/ArnaudRecipes/sharedlib.html

> Ubuntu and RedHat, but also on FreeBSD, OpenBSD, Darwin, Solaris, SunOS, Tru64, HP-UX, IRIX, AIX,

So if you dropped SunOS, Tru64, HP-UX, IRIX, and AIX how many users would you use? How much of that configuration garbage would go away?

Yeah, that may be true today, but it certainly wasn't when these tools were conceived. (According to wikipedia, autoconf was first released in 1991.)

But you don't have to go on using it...

Oracle's SunOS (now Solaris), HP-UX and IBM's AIX, plus the community-supported *BSDs and Darwin would be the only ones you'd see in the wild today, from that list. IRIX and Tru64 are retired or history, according to Wikipedia. ZDNet in April 2014 reports that Unix server share as a whole is 13.6% according to IDC, Oracle has 4.7% leaving 8.9% to competitors. Since many of these are RISC or Itanium servers, leaving them out means fewer compatibility headaches on multiple fronts. That all said, unless there's a testing service out there for this kind of enterprise hardware, I can't see how anyone writing open source code outside of large enterprises would be expected to support this configuration without patches from the community... which is probably why "standard checks" remain today. Nobody really knows if they're still needed or not, since the original patch might go back to the mid-90s.

"Ruby (with Matz), Python (with GvR), Clojure (Rich Hickey), the Linux kernel all seem to be doing pretty well with someone nominally in charge to make hard decisions."

So many programming languages. So much confusing choice and wasted effort.

What we need is for someone to be put in charge and consolidate all those programming languages in to one language.

I think you are getting down voted because you voiced a very controversial opinion without much explanation to to back it. Still, I think there is some merit to it (although I wouldn't go as far as saying we're ready for the "one language to rule them all" yet).

The paradox of choice and duplication of effort are very real problems. Let's take web app development for example. We have PHP, Python, Ruby, Java, C#, etc. Each language has their own set of libraries/frameworks and it takes a lot of time and effort for a programmer to learn all of them. I've been developing web apps for 10 years and yet, I couldn't contribute to a web app written in Java. What if instead of having X libraries in Y languages, we had X*Y libraries for a single language or X libraries of Y higher quality for a single language?

Standardisation has lots of benefits.

The web standardised on HTTP/HTML/CSS/Javascript and it enabled the creation of a massive ecosystem of tools, libraries and pool of developers. Front-end developers can easily switch jobs and contribute to other projects. I personally dislike languages that compile to HTML/CSS/Javascript because I think standardisation trumps their benefits. C is a de facto standard in the embedded world.

In my opinion, sometimes the hammers are so similar (e.g. Python/Ruby) that we should just collectively agree to only use one.

Parent's post sounds like sarcasm to me.

I know a very good developer who does both Unix and Windows development. He knows both very well.

He points out that with Linux you officially have the ability to customize everything perfectly, but spend a lot of effort just deciding what your stack will be. And if you integrate 4 or 5 things together, you get a lot of redundant crap because they did things differently.

In the C# world, by contrast, Microsoft declares technology winners and everyone adopts them. Those winners may be in principle not ideal, but the fact that everyone actually uses them saves a lot of wasted duplication of effort.

That's a very idealised view of Microsoft's platform. In particular, they have a habit of periodically declaring new technology winners that aren't compatible with the old ones, which then become legacy. Remember Windows Forms and Windows Presentation Foundation, both of which were Microsoft's officially-sanctioned ways of developing GUI apps with C# and both of which are now legacy APIs in maintenance-only mode?

Silverlight, anyone? Microsoft is excellent at leaving folks high and dry.

This is true.

However the cost is no worse than the cost in the open source world of jumping on technology decisions that then wind up fading away and dying. And the fragmentation in the open source world makes that more likely.

Yes, in principle you can support open source software yourself. But you'd be amazed at the long term costs of doing so when you are locked in to a platform with diminishing market share.

Perhaps a better example of the Microsoft strategy would be Windows itself. For many years and several major versions, they went to extraordinary lengths to ensure backward compatibility and stability, so software that ran on Windows from the mid-1990s would still have a reasonable chance of running acceptably on Windows in the late-2000s.

We had almost reached the point where, notwithstanding any commercial licensing barriers, you could either still run application software natively, or it was so old that you could host a virtual machine containing the old OS within the new one. That's a great situation to be in if you rely on the software, and it's also a very nice situation if you write software, because it basically means that once you've got software that works it works forever. A firm foundation keeps everyone safe.

It's regrettable that under Ballmer and now Nadella they seem to have been pushing hard in IMHO the wrong direction with increasingly rapid updates and more reliance on inherently unstable on-line services, instead of playing to those traditional strengths. This makes Windows just as unlikely a prospect for long term stability as anything from Apple or the Linux world these days.

Personally, all I want is a system that I can buy today to do useful work and reasonably expect to still be doing useful work more than a year or two later. It's disturbingly difficult to find software that meets that apparently simple requirement any more.

If you want backwards compatibility, you really should look at IBM.

Their mainframe family still runs most serious financial platforms around the world, and it is routine to run code written decades ago on the latest generation of hardware.

No, it is not cool. It is not sexy. It is not cheap.

But it is backwards compatible. With a vengeance.

That's missing the forest for the trees.

There is a difference between being able to choose between 15 tools and being able to choose between 15 hammers. The point of the article is that we just don't need 15 hammers -- especially as 10 of those are obviously inferior -- and the existence of 15 hammers is incredibly unhelpful.

http://en.wikipedia.org/wiki/Hammer#Hand-powered_hammers

You're not going to need 15 hammers on a specific job (... for one person), but there are a lot of kinds of jobs in the world with particular needs.

That's kind of a poor metaphor, as there are at least a dozen distinct and useful types of hammer.

Didn't God put Larry Wall in charge of that decades ago? How did that work out?

so your in 98-99% of cases not going to be using OSX in production so why introduce more risk by developing on a different system to the end product will run on.

> Ruby (with Matz), Python (with GvR), Clojure (Rich Hickey)

Many would argue that Haskell ends up more consistent and elegant than those, despite having been designed by committee.

> the Linux kernel

Ends up with messier code and design than that of FreeBSD; perhaps an inevitable side effect of popularity, but perhaps exacerbated by the person-oriented command hierarchy.

> No one would set out to design a system to generate a file to probe a system for 26 programs that you don't even need to choose build options for a program that only runs on a handful of nearly identical systems.

Actually almost all of libtool's code was written by one person, Gary Vaughan.

Bazaars can produce elegant code, and cathedrals can produce monstrosities; a handful of anecdotal examples prove very little.

> Many would argue that Haskell ends up more consistent...

I would argue that Haskell is an outlier in more than one dimension.

Along with other outliers like Smalltalk and Scheme, neatly grouped together under "experimental languages". Cf. SQL, a language that from the beginning had its industry in mind.

Common Lisp would probably be the best counter-example to "language design by committee doesn't work." Despite being developed by a committee over an almost 15 year period and the standard document being over 1000 pages long and full of functions that are there just for backwards compatibility, the language manages to be both more consistent and feature rich than Ruby or Python.

The language most people know as C has been developed by a committee since 1983 (try reading pre-ANSI C). C++ has been developed by an ISO committee since 1993(?) and has expanded hugely in scope (standard library, namespaces, RTTI).

So successful programming language design by committee seems to be the rule and not the exception.

Common Lisp, C and C++ did not contribute to the field in the enormous way that Scheme, Haskell and Smalltalk did. They're sheeple languages (made with the worker in mind) to control employees, and not languages to empower and liberate the programmer (like Forth, another great example of an experimental language by Chuck Moore).

My point stands.

> They're sheeple languages (made with the worker in mind) to control employees,

Total nonsense.

> and not languages to empower and liberate the programmer

Also nonsense.

If you want to refute you will have to do better than just show up; you could start by explaining why do we have so many shops using Java and .Net and now Rails (which makes it possible for Ruby programmers to be fungible for managers) and so few using Scheme and Smalltalk etc.

Ask your manager why you can't switch to a better language. The answer is it's harder to find talent for it. Bingo.

> Ask your manager why you can't switch to a better language. The answer is it's harder to find talent for it.

That's a really good point.

IDK if I would say that about Scheme though. There are a lot more people around that know Scheme than know Common Lisp. I learned to program Lisp starting with Scheme; going back to Scheme from Common Lisp feels really disempowering.

You've just made his point: you're so used to living in the bazaar that it seems an inevitable fact of life.

Through the lens of the Peter Principle, command is itself a competency. If you have sufficient authority, (1) is easy. His point is that unless you have someone with that authority, (2) is inevitable. People have risen to a position where they need the solution libtool provides, without the competency to force the right solution.

This wasn't always the case.

But this is what I don't understand. When was there someone with authority to force all the Unices to adopt (1)? Autoconf came out in 1991, so clearly the problem existed at that point. That's coincidentally also the year Linux was first released, so it can't be blamed.

It seems to me that Unix was never a cathedral. Each specific operating system may have been one, but they clearly weren't more compatible back in 1991 than a Protestant and a Catholic cathedral are, since someone felt the need to write autoconf.

Please give me an example of this shining 20-th century "cathedral development", because I don't see one.

In the essay, phk specifically mentions when AT&T decided to commercialize UNIX.

Whether that makes sense or not is something I leave for the graybeards to fight out.

I believe that his point is that the world would be a better place without those 22,198 tools rather than with the gigantic mess that having them creates.

I myself am somewhat amused by the paragraph,

"...Later the configure scripts became more ambitious, and as an almost predictable application of the Peter Principle, rather than standardize Unix to eliminate the need for them, somebody wrote a program, autoconf, to write the configure scripts."

The key phrase being "rather than standardize Unix". Anyone want to count how many Unix standards there are?

He goes on about libtool with,

"...yet the 31,085 lines of configure for libtool still check if <sys/stat.h> and <stdlib.h> exist, even though the Unixen, which lacked them, had neither sufficient memory to execute libtool nor disks big enough for its 16-MB source code."

I don't know if AIX had sys/stat.h or stdlib.h (actually, I believe it did), but I do know those machines I worked on certainly had enough resources to run libtool; I did it. Because without libtool, building shared libraries on AIX was an exercise in crazy. (Almost as crazy as m4.)

http://stromberg.dnsalias.org/~strombrg/AIX-shared-libs.html

I don't know about Poul-Henning Kamp, but I didn't have the luxury of spending those 15 or so years using a single flavor of BSD.

You pretty much just reiterated his point for him.

His point is that the entire Unix ecosystem should be maintained as a single cathedral?

Windows is maintained as a single... (I won't say "cathedral")

"Windows-the-operating-system" is, but "Windows-the-distro" is not. He's specifically mentioning Firefox, so his argument is not restricted to the core operating system.

Windows is like a decent catholic cathedral in the middle of Mogadishu.

*nix is like 50 chapels using two customized holy books in mos eisley.

Linus shot first.

Actually, it only really looks like that on the outside. The internal organising principles are a mess, and that mess shows in the design.

Windows also breaks existing workflows and software with every new release, and then cuts support in less than a decade.

Windows still runs software from the 1980s and 1990s without much complaint. DOS software, Windows 3.1 software...

Amusingly, you can run that software under Linux also.

Not really. The decision to require applications to open ports, as opposed to just bind them, broke pretty much all networked software on Windows in the 00s. That's just one example where software may run but doesn't work. It's complicated.

True, but sometimes those "breaks" are good things (Vista to 7 was nothing but improvement). And generally, Microsoft will support their releases for longer than a decade. XP had over 13 years of extended support, and Windows 7 will have 11 years of extended support. Even Vista, one of the worst OS releases from Microsoft, is still in extended support for another two years.[1]

Contrast that to major (LTS) releases from Ubuntu, which can be from 3 to 5 years, and only 9 to 18 months for non-LTS releases.[2] And Ubuntu is one of the kings of bleeding-edge breakage in the Linux-based OS world, surpassed only by Arch in my experience.

All that said, it's not all rosy in the Windows world, and not all thorns in the Ubuntu world. I just wanted to point out some facts to counter your misinformation.

[1]: http://windows.microsoft.com/en-us/windows/lifecycle

[2]: https://wiki.ubuntu.com/Releases

You mean either 'eco system' or 'echo chamber', both of which describe the unix community.

yeah, typo. i meant ecosystem.

I think I'll start using the term "echo system"

It's not the people who wrote all that bloated software who are incompetent. It's the person who wrote that ridiculous book "The Cathedral and the Bazaar" who is incompetent.

Reply on news.ycombinator.com