Comment by tmhedberg | Hacker News Reader

Comment by tmhedberg | Hacker News Reader

The FHS (Filesystem Hierarchy Standard) [1] is the go-to reference for this sort of thing. It explains that `/bin` is for binaries that are essential before other file systems are mounted (e.g. in single user mode), and `/usr/bin` is for "most user commands" (all others). This allows you to keep a minimal local filesystem containing only the binaries needed for init to get the system running, and then `/usr` can be mounted, say, from a network share. This is useful because then network admins can install software to the common `/usr` share and make it immediately available to all machines which mount that share.

The `/sbin` and `/usr/sbin` directories are for commands needed only by administrators, which will not normally be used by regular users.

Most systems don't really require this separation, but it does make sense. Perhaps the historical reason for doing it is no longer a factor, but that doesn't mean it's perpetuated merely because of tradition.

[1] http://www.pathname.com/fhs/pub/fhs-2.3.html

The central thing about all these conventions is they developed in the absence of good union mounts. In Plan 9 there is no $PATH: all such things are bound into /bin. This works both because Plan 9 has very good union mounts, but also because those union mounts can be localized to a specific process and as a result are user accessible. I seem to recall some work was done to make this possible on Linux but It didn't get accepted into the main kernel because it is at odds with how we see how a Unix system should behave.

It would be interesting to see a Linux distro that embraced this.

I wonder why rc still has $path in Plan9.

   term% echo $path
   . /bin

What happen with Ape ports whom expect the path environment variable?

rc has $path because rc predates Plan9, it first appeared in tenth edition Unix.

Inferno's shell doesn't use a $path variable.

Ape's execlp() tries first the program name passed to it (be it relative or absolute), and if that fails it prepends "/bin/" and tries again. It ignores environment variables entirely.†

Rc's path variable allows you to easily tell rc to check the current directory when looking for programs to exec. Doing that with bind after each cd would be clumsy. And if your working directory is a remote server, you can set path to just /bin so that you aren't statting the remote directory before each exec. Inferno's sh does use a path variable, but it is typically left unset and the default (/dis .) is used.††

† See http://plan9.bell-labs.com/sources/plan9/sys/src/ape/lib/ap/...

†† See /usr/inferno/appl/cmd/sh/sh.b:/^runexternal

I know about FHS and the rationale FHS uses for the continued existence of /usr, I just do not agree with it (nor with the existence of /usr/share).

I'm curious, what is your opposition to `/usr/share`? It seems quite logical to me as a place to store non-executable, read-only, shareable data like man pages and other documentation, fonts, keymaps, color profiles, game data, etc. Where else would you propose to store this sort of thing? I can't really think of a different location in the hierarchy that would make sense.

Because everyone's idea of what "makes sense" is different. Thus:

/bin

/usr/bin

/usr/share/bin

/usr/share/local/bin

/usr/local/bin

/usr/local/share/bin

/opt/bin

/opt/some/clever/path/system/bin

Followed by hacky symlinks so that programs can find what they're looking for in /usr/local/share/lib/x, which really resides in /usr/share/lib/x or maybe /var/lib or maybe /var/local/lib or maybe just /lib because it's considered "essential" on some systems but not others.

It reminds me of this: http://xkcd.com/927/

Perhaps you have some misbehaving software, but I've never seen /usr/share/bin, or /usr/local/share/bin or a lib directory in either of those locations. I just checked FreeBSD boxes (which, admittedly, follow their own standards closely), and a Redhat box.

var is a directory which may be found under /usr or /usr/local, but won't contain a local directory.

Think of it as 3 roots: / is required for boot, /usr is stuff maintained by an administrator (say, company wide), /usr/local is where files installed by local users go. Within each of these 3 roots, you'll have some subset of bin, etc, lib, libexec, sbin, and var. The root directory has some additional singleton directories which don't make sense in the other directories: dev, root, mnt, boot, proc, rescue.

/usr is maintained by the OS, and contains stuff that isn't required for boot but to round out the system.

/usr/local is where local modifications to the OS go, these can be site-wide (company wide) and can be hosted on NFS for example for network boots. User installed stuff belongs in /usr/home/<name>/*. Users shouldn't have root access to install stuff in /usr/local.

Side point: it really bothers me that on freebsd the ports system takes over /usr/local. I expect /usr/local to consist of things that I as admin have installed by hand, and nothing else.

Agree totally.

NetBSD uses the more sensical /usr/pkg for package-managed software, leaving /usr/local for the admin.

You can change where packages are installed using some make.conf flags. You can then install the software anywhere you want. Your users will need to add the new path to their PATH and you'll need to set some flags in rc.conf to pick up the new rc.d path as well (so software installed through ports gets auto-started).

Then again, when you install ports you as the sysadmin are installing those by hand ...

This is a little bit disingenuous. I can certainly see the point that all of this separation is unnecessary on a typical modern desktop system, but several of the paths you listed in support of your argument are not actually specified by the FHS, nor do they even make sense.

Of the paths you mentioned, there is no such thing as:

    /usr/share/bin
    /usr/share/local/bin
    /usr/local/share/bin
    /usr/local/share/lib
    /usr/share/lib
    /opt/bin

All the paths that involve `share` are for non-executable data, so `bin` and `lib` subdirectories of these don't make sense. That's not to categorically say that there isn't any OS that provides them anyway--I've seen plenty of Linux distros with disorganized file systems--but that's a problem with the distro, not the FHS as a whole.

The `/opt` hierarchy is pretty much the wild west, I'll give you that. It's basically like `Program Files` on Windows; packages get an entire hierarchy to themselves. Most of the software I've seen that installs to `/opt` is Linux ports of Windows software, where the authors were either ignorant of the Unix way of organizing things, or simply didn't want to bother conforming with the norms of a different platform.

Incidentally, one of the reasons I prefer Arch Linux over some other distros is that the organization of the file system follows the standard and actually makes sense. Things are always where I expect them to be.

Your overall point may have some merit, but it feels a little like you're reaching for support for your position by making up wacky, confusing paths that don't actually exist.

Every one of those paths I mentioned are paths I have encountered during my use of various unix distributions since 1991 (Solaris was the worst offender). And that's not even the complete list. I gave up trying to predict where software would install to a long time ago.

My point is that what "makes sense" is subjective. Each developer/distro manager who made one of those paths thought to himself "it makes perfect sense to do it this way". FHS does go a long way towards cutting back on the craziness (by arbitrarily dictating "do it this way"), but it's still clunky.

Your encountering them doesn't make them part of FHS.

Their being included in FHS does.

At no point did I ever say they were part of the FHS.

OP did, and by inference (with my pedant bit set) you were supporting his statements. http://news.ycombinator.com/item?id=3520178

Hmm that was not my intent.

Most of us are more concerned about the real world than a "standard" that is routinely ignored.

I put Java in /opt/java/, and embedded toolchains in /opt/[platform]/. Not exactly Windows ports, but they're the sort of thing that can get out of hand easily. :)

You forget my favorites: /usr/libexec and /usr/lib/$progname

Actually it reminded me of http://xkcd.com/981/

But i do remember seeing /usr/share/bin and /usr/share/local/bin and thinking wtf... But then i play around too much with my OS to reliably blame it on the distro...

I really like the way FreeBSD separates system and third party software, the latter typically being installed to /usr/local.

Linux' failure to differentiate between system and non-system binaries is IMO one of it's worst features.

Neither FreeBSD nor Linux accommodate network mounted NFS mounted applications as well as SunOS did so successfully by the late '80s. This is probably due to the corrupting influence of Unix and Linux engineers who came from MS Windows backgrounds. Windows admins waste countless hours having to install software locally on every machine. Sad that only Solaris provides for a standard network-based application filesystem. Odd too that Solaris is home to the single most ill-thought-out non-system directory, /opt.

BSD is in a position to differentiate in a way that Linux just isn't. In Linux all* software is third party, except for the kernel and util-linux (and maybe fileutuls or some other set of GNU software).

To me the BSD distinction doesn't make much sense anyway. You still have to just know which bits are "local" and which are not, plus anything I pull in that isn't a port gets lumped in with them anyway.

Well, first let me extend your explanation of the rationale for /usr/share: the reason for having a place to store not-executable shareable data is that in some network environments, it helps to have to maintain only one copy of this data, on some file server somewhere, which then gets mounted over the network to every machine. We need to keep /usr/share separate from /usr/bin, the rationale continues, because our network might be heterogenous in executable formats. In other words, some networks cannot maintain only one centrally-maintained copy of /usr/bin because some machines need, e.g., the x86 version of /usr/bin and some need the ARM version. In contrast, every machine can mount the same version of /usr/share.

Now suppose you are an admin for a network in which the numerical software foocode is a big deal. the software contains a large read-only database of numbers, which according to the logic of the FHS goes into /usr/share/foocode/. Now suppose that some machines upgrade to a new version of foocode which for performance reasons stores integers in some specialized format, not the ones-complement format that has become something of a standard. So now, if we continue to apply the logic that motivated the creation of /usr/share in the first place, we need a place to put numbers of the new format so that they are kept separate from the numbers in the old format. Thus /usr/share/ones_complement/foocode and /usr/share/new_format/foocode are created. Suppose further that yet another version of foocode is released, and again that some machines are upgraded to it and some are not. This new version introduces a new, more performant format for the database in which the numbers are stored. Well, if the old format is called the 'hyperbolic' format and the new format is called the 'elliptical' format, then the logic that led us to create /usr/share leads us to create /usr/share/ones_complement/hyperbolic/foocode, /usr/share/ones_complement/elliptical/foocode, /usr/share/new_format/hyperbolic/foocode and /usr/share/new_format/elliptical/foocode. My point is that at some point you need to move to some way of assigning 'attributes' (and "read-only, shareable, non-executable" would be an example of an attribute) to files in some way other than putting those attributes in the name of the file. It would have been better for that point to have arrived before the FHS caused programmers around the world to have to type /usr/share/emacs/23.3/lisp 500 million times when /emacs/23.3/lisp would have done.

What would this other way of assigning attributes to files be? A detailed explanation would be too many words for a HN comment. For the specific problems mentioned in this comment, namely, centrally-adminned file servers, you probably make your net-mount command more complicated like they did in Plan 9.

ADDED. div and mod are well know numeric functions, right? The mere fact that some group of people would like some convenient way to refer to all the numeric functions does not justify requiring every programmer to write numeric/div and numeric/mod (or numeric :: div and numeric :: mod) every time they want to use or refer to one of those functions. I say the same argument applies to /usr and /usr/share.

I very much appreciate your added comment. I feel like banging my head against a wall every time I see a new programming language that boasts a 'hello, world' along the lines of:

  System.Console.WriteLine

or

  System.out.println

or

  Ada.Text_Io.Put_Line

just to get a little text output.

FYI, the linked post completely contradicts what you wrote.

`quote'

Reply on news.ycombinator.com