Hacker News Reader

/bin, /usr/bin, /usr/local/bin, /opt... is a mess on unices. There is no profound idea behind it. Someone needed a folder that was not interfered with, then someone started interfering so a new folder was born, and so forth. However, they are mostly just prioritized copies of eachother. / comes first, then /usr, then /usr/local, then /opt. The structures in them are identical.

On Windows, however, the only organized region would be that applications get a folder, but there's multiple potential locations (Program Files, Program Files (x86)), and they might also dump a bunch of things in various user-local folders, as well as the giant landfill that is C:\Windows. It lacks structure, to the point where there is even clones of UNIX structures in there (C:\Windows\System32\drivers\etc\hosts?!?).

First you got /bin on boot, enough to mount an external drive to mount /usr and you're set. That's what I heard was the case back then. These days it's non sensical and I heard about an effort to symlink either of it to the other and consolidate the binaries.

/opt is alright. It's where you dump your tar ball and compile your custom stuff and place it under not to mix stuff up with package managed software. Not sure how people differentiate against /usr/local though.

/usr/local vs /opt, afaik, is mostly a case of 3rd-party commercial software ending up under /opt, whereas your more personal scripts and hacks are in /usr/local.

I'm not particularly bothered about /bin vs /usr/bin, these days everything is in /usr/bin -- unless you are in an embedded scenario, and at that point everything is custom anyway.

The one location that doesn't make sense is /usr/sbin, which is hopefully being phased out.

Tbh the only thing I really miss is having ~/bin in PATH by default (and ~/lib, ~/share etc similarly picked up where necessary), since installing software in personal userspace is still pretty common in real multiuser environments; standardizing that would help, imho.

> Tbh the only thing I really miss is having ~/bin in PATH by default (and ~/lib, ~/share etc similarly picked up where necessary), since installing software in personal userspace is still pretty common in real multiuser environments; standardizing that would help, imho.

Not sure that will become standardized, it's a big security risk. Usually you want ~/bin to be at the front of your PATH so it takes priority, but the downside is... well that it takes priority. It's easy for someone to subvert the stuff in PATH because they only need your permissions.

Homebrew use /usr/local/bin, macports use /opt, so if you want to be "safe", you'd need to make yet another location...

Arch undid this, but did it the silly way by letting /bin be a symlink to /usr/bin. I would have preferred it the other way around.

"These days it's non sensical and I heard about an effort to symlink either of it to the other and consolidate the binaries."

Yeah, Fedora wants to move everything to /usr, last I heard. I personally think it's a stupid idea (/usr is just a vestige of Unix's early development and was never meant to be a permanent part of the Unix filesystem layout) and would prefer everything to be under /, but whatever.

"Not sure how people differentiate against /usr/local though."

/usr/local is typically reserved for stuff built from source. OpenBSD also uses it for things installed via the package manager.

/opt is a lot more freeform. Yeah, you can create an /opt/bin or /opt/lib or what have you, but the way I've seen it used (and the way I personally use it) is something closer to Program Files (so I'll have /opt/firefox or /opt/julia or whatever). A lot of commercial software packages will install to /opt in a similar manner.

Fun fact: /usr exists because Thompson and Ritchie ran out of disk space on the root disk of their PDP-11 [1] in the early 1970s.

/usr was originally where the user's files (everything but the OS) lived, but when they ran out of space they decided to just move OS stuff into it.

The whole "/bin is available on boot until /usr is mounted" story, which a classic form of post-rationalization, though it was presumably a useful distinction for a while.

[1] http://lists.busybox.net/pipermail/busybox/2010-December/074...

That's one-sided and misleading. In fact, the Windows model of "/Program Files/%COMPANY_OR_PERSON%/%PACKAGE%/", "%USERPROFILE%/AppData/Local/%COMPANY_OR_PERSON%/%PACKAGE%/", and suchlike is akin to NeXTSTEP's ~/Apps, /LocalLibrary, /LocalApps, and so forth; both are from the early 1990s.

Similarly, the Windows organization of %USERPROFILE%/AppData/Local/ , %USERPROFILE%/AppData/Local/Temp/ , and so forth has a parallel on Freedesktop operating systems.

* https://unix.stackexchange.com/a/313001/5132

And of course as indicated on this very page at https://news.ycombinator.com/item?id=14927512 and https://news.ycombinator.com/item?id=14926999, interference was not the reason for the creation of /usr .

People have also observed on this very page that far from being new-fangled, as it has been mischaracterized on this page, GoboLinux has been around and doing things like this for almost a decade and a half. (Of course, as NextSTEP and Windows NT 3.5 show, the idea is almost a decade and a half older than that, even.) It is interesting to note that slightly earlier than that, Daniel J. Bernstein, around the turn of the century, proposed a /package hierarchy (for package management without the need for conflict resolution) and a /command directory.

* http://cr.yp.to/slashpackage.html

* http://cr.yp.to/slashcommand.html

* https://news.ycombinator.com/item?id=10356933

* https://news.ycombinator.com/item?id=11647304

It has been a long time since I read those email correspondences. They are quite hilarious. However, despite the pedantry, /usr/local/bin and /opt/bin are all about not interfering with /usr/bin and /bin. /usr was, of course, made to be a home folder, and converted for the silly reasons mentioned in those old emails (although this was changed back in Plan9 and Inferno), and /usr/bin has maintained a similar idea of being non-essential binaries in many current distros (/bin often belonging to one package, and /usr/bin containing many independently managed packages).

You seem to have missed that I did not complain about "Program Files" being nicely organized folders containing all relevant application data (although I am highly allergic to unnecessary capitalization and poor naming), nor say that Windows was the only platform with a mess, but rather complain that there are multiple such program locations (Program Files, Program Files (x86), and some applications install to user folders), that Program Files contain a bunch of entries for things aren't programs (helpers, things that installed elsewhere for stupid reasons, etc.), that AppData is a hidden mess, and that the Windows folder is a ghastly landfill that contains everything under the sky, including things that should have gone in the other locations (system applications, libraries, ...), things that didn't fit anywhere (most of whats in here), and even configuration files in UNIX directory structures (/c/windows/system32/drivers/etc/hosts). It's a disgusting mess in there that cannot by any meaning of the word be considered organized.

Note that I have not used Windows for at most 60 contiguous minutes for the past many years, so my memory may fail me.

Repeating the same thing does not make it any less one-sided and misleading. In fact, Microsoft discouraged applications from putting anything into the the windows directory or the system directory from the middle 1990s onwards.

* https://msdn.microsoft.com/en-gb/library/ms997548.aspx

* https://blogs.msdn.microsoft.com/oldnewthing/20110620-00/?p=...

* https://blogs.msdn.microsoft.com/oldnewthing/20120307-00/?p=...

And of course the windows directory being a subtree with everything that is part of the operating system, is exactly the same model that people have been adopting with the "/usr merge" from AT&T Unix System 5 Release 4 onwards, and only a "landfill" if one thinks having the operating system in its own single subtree off the root is a "ghastly" idea. The range of operating systems that have done this very thing over the years, with the likes of C:\DRDOS, C:\WINDOWS, C:\OS2, /System, /boot/system, and /usr, are really not for you in that case.

By the way, your description of priorities with / overriding /usr overriding /usr/local was as misleading as your idea about interference rather than the far more mundane and ordinary having a second disc. In fact, on systemd operating systems the default path is to search in the order /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin which is the opposite order to your description. FreeBSD nowadays uses a similar order, with _PATH_DEFPATH being /usr/bin:/bin , although _PATH_STDPATH is /usr/bin:/bin:/usr/sbin:/sbin . OpenBSD goes with /usr/bin:/bin:/usr/sbin:/sbin:/usr/X11R6/bin:/usr/local/bin for both.

The misleadingly simplistic and wrongly ordered /, /usr, /usr/local wasn't even the historical case, moreover. Back in the 1980s people had the likes of /usr/ucb, /usr/lbin, /usr/bin, /usr/5bin, and sometimes even /etc and /usr/etc on the search path. There were actually quite a lot of these. AT&T Unix System 5 Release 4 had them all under /usr in 1988.

Repeating the same thing does not make an opinion accepted or popular.

Just in case you weren't aware, on most distros you can enter "man hier" to get a full description of the filesystem hierarchy.

wow

the more you know

That is an incredible little tidbit. I love finding out the answer to most of my questions is under my nose in the docs baked right in

How have i never heard of this!?! Thank you so much for this tidbit; very handy!!!

The content of hier(7) looks a bit out of date, e.g. it's missing "/run" but still documents the whole archaic "/usr/X11R6" hierarchy that went away together with imake a decade ago.

file-hierarchy(7) appears to be a more modern alternative.

(Cue complaints from users of a real UNIX about how GNU/Linux is such a cobbled together poorly documented hodgepodge...)

On your last point about installing software: I've used Ubuntu and Fedora, and finally settled on Arch ~4 years ago. Interestingly, although Arch has a reputation of being a "DIY" distribution (in the sense that it targets power users), I've had way more trouble installing software on Ubuntu and Fedora than on Arch.

Arch has what is called the Arch User Repository (AUR). It's a repository of recipes for building all kinds of software, and it's submitted and maintained by the users, not maintainers of the distro itself. Whenever there's a package missing in the official repositories, I almost always find it in the AUR. AUR has no binary builds---instead, you download files and scripts needed to build the software from source. You then run the makepkg utility, and you get a package you can install. (There are also helper programs which automate this whole process for you.) The result is that the package manager tracks the files properly, and you can keep track of which version you have, can remove the software cleanly, and update it whenever you like.

Today, when I'm at a Ubuntu box, it's a pain in the ass to install some less-known software. I can download the sources and then try to build, and I usually succeed, but not before I've figured out the right flags, prefixes, build-time and run-time dependencies, etc. I also get questions pertaining to this from my friends that have recently switched to Linux, and I don't really have a solution to offer. Yes, there are PPAs, but more often than not, PPAs are outdated and are binary-only, meaning you can't really review how the binary packages came about.

AUR really solves the problem nicely. I'd like to see such a concept in another, more beginner-friendly distribution, that I can then recommend to people around me who are considering making the switch.

> AUR really solves the problem nicely. I'd like to see such a concept in another, more beginner-friendly distribution, that I can then recommend to people around me who are considering making the switch.

Yeah, I wish there were AUR for Ubuntu

How does Arch's AUR compare to Gentoo's portage?

Manjaro or another Arch-based distro might be the answer there?

I tried arch a couple of years ago (in a VM), and while I liked it and the reasoning behind it, I had certain misgivings that led me away from it being considered for any future use.

The main one being that whole AUR thing: What happens if there isn't a recipe in there? For example - you want to install something which only has a binary, no source, and can't be (legally) redistributed; what then?

Well - to "install" the software, you basically have to go thru all of the steps to build your own recipe, noting down what you do manually, and eventually getting to the point where you have a script (or whatever) to install the software. Then, you undo all of your manual work, and run your arch package to do the real install. Tada - you now have a working package for that piece of software, and have spent potentially hours to get to that point.

As a bonus, you can't redistribute that recipe, because it is based around a binary blob that was purchased and you don't have the rights to redistribute it.

At least, that's how I interpreted it. I honestly don't recall what piece of software I wanted to use, but I think it may have been EagleCAD - which has a Linux port, but is binary only. I think it only has RPM and DEB packages, so to get it on arch, you'd have to build your own package.

It wasn't so much the fact that you had to do this, just that it wasn't an easy and straightforward process (from what I recall), and if you only need to do it once or twice, it would be pretty arduous. Maybe by now (haven't looked) they have some kind of package conversion tool, which would be a great addition.

Switching operating systems and tools is always a messy experience.

Coming from Linux and macOS to Windows this year, the Windows filesystem has been... well, one of the very confusing things. Although I understand the Linux fs is messy (and macOS improves a bit on that), I'm used to it and it makes sense.

OT, the only good bit I found is the Outlook directory, mail, calendar, Office experience (but that will not make me stick to Windows either).

(I work mostly with rpm based distros so ..)

> I generally have no idea where my config files are stored

rpm -qc <package_name>

> and can barely find where programs keep their data

rpm -ql <package_name>

>And then I download some software that doesn't install itself and I have to figure out where to put it...

Not sure if you mean source to be compiled or a package?

>The fact that this is all only sorta-kinda standardized between distributions

Distributions exist because people have their own opinions on the best way to do things. Pick one you like and learn it.

If your complaint is that this isn't all obvious then I wholeheartedly agree with you. But this is genuinely a simple case of you not RTFM.

I don't want to RTFM to understand a file arrangement. It must be intuitive, like what I see in Finder in macOS.

UX design failures like this is the main reason why Linux can't get more than 3% on desktop.

Personally, I'm glad that "Linux on the Desktop" has such a low figure (whatever it really is, but 3% sounds reasonable).

It would be like when AOL allowed their users on the internet.

With viruses and other crap too.

So your point is that I need to RTFM to know something that should be intuitive? That kind of corroborates the OP's point.