Gobolinux : Redefining Linux filesystem hierarchy
gobolinux.org
gobolinux.org
Installing each package into it's own directory, and maintaining symlinks into that directory structure, was an approach explored for large system installations in the 90s and 00s.
NIST depot, Xheir, CMU, and Nix all did this in varying ways. The advantage of this approach was that packages could be installed once on an NFS, scoped by version/configuration, and mounted into the filesystem of every work station on a campus. The workstations then just needed to maintain references to the files on the NFS for their particular software configurations. There is also some interesting exploration of composable configuration templates that allow workstations to inherit and extend a particular configuration (i.e. the physics department might have a base template).
Nix is easily the most successful in this space. It even uses something similar to the NFS approach! It can maintain a remote cache of observed configurations to pull from instead of redoing work.
Reading these LISA papers is a lot of fun. If you work in system administration (i.e. maintaining an Artifactory instance for your company) and have a day to spare - I'd very much recommend reading them, starting with the Nix paper!
To get you started:
Nix - https://www.usenix.org/conference/lisa-04/nix-safe-and-polic...
Depot Lite - https://www.usenix.org/conference/lisa-viii/depot-lite-mecha...
Packages bootstrap filesystems for installations. We’ve parted out system administration at scale - multiple paths you can take.
Interestingly, GoboLinux Runners are basically the same concept as NixOS' FHSUserEnvs, where user namespaces are used to mount a subset of the existing shared libraries and other dependencies to make them visible (and the only ones visible) to a running program in a compatible way. This spares them some of the filesystem overhead of running a full OS container with its own package manager. So there's a bit of further convergence there, too.
I love the cleanliness and legibility of the GoboLinux file hierarchy, and I think the simple, Mac-ish app bundling approach it takes is something many users might like and prefer over the rough edges of Nix or the complexity of Flatpak.
A GoboLinux-like presentation could be great for a stable, impure NixOS downstream some day.
Thanks for the link to the Depot Lite paper! I'd never heard of it.
If you do want to dive into this stuff, I’d recommend reading both the Nix and Depot Lite papers of course, but their references section is also a gold mine of fascinating systems.
I've read through a few if the Nix papers before, but they're probably worth revisiting by now after using Nix for a few years and experimenting with Guix a bit.
> I can geek out about package management all day.
Same! I've been fascinated with package management probably since I was about 12 or 13, when I first tried desktop Linux. Package managers are incredibly powerful systems tackling an equally incredibly tough problem space. Multi-language package managers (i.e., ports systems and Linux distro package managers) have always especially impressed me. (Imo even the 'bad' Linux distro package managers are pretty good, as much as I can be harsh on them in certain aspects when comparing them.)
Getting into Nix has made me think more deeply about the designs of the many language-specific package managers, since each one needs/gets a different treatment in Nixpkgs according to its unique properties. It's similarly provided an illuminating alternative to the container approach to traditional deployment troubles— one that conserves more of what is good about the traditional package management paradigms on Linux and elsewhere.
Have you looked at Michael Stapelberg's work on package management? I think it's the most exciting work on package management outside the Nix and Guix worlds: https://michael.stapelberg.ch/posts/2019-08-17-introducing-d...
Luet also looks interesting, but I've not really played with it yet. Its approach seems to be to add more package-like granularity to existing container systems: https://github.com/mudler/luet
What they do is store packages in the winsxs folder. They then do an NTFS hard link to where the component is installed, usually system32.
If you ever used Vista and Windows 7 and started to run out of disk space over time, that’s because Microsoft never used to remove the old components after they were updated. In Windows 7, the initial plan after SP1 was to allow people to use a special switch on DISM.exe to remove old service pack files.
Microsoft removed the idea of service packs and instead did continuous patching with rollups. It seems to taken them quite some time before they twigged to the fact that the old updates were using up a lot of disk space, because a long time later (almost at the end of the Windows 7 lifecycle) they added a Windows update to the disk cleanup wizard. Even worse, it would only remove the components after you rebooted but before you could login to Windows. If you ran it, it could sometimes take literally hours to get the computer to finish removing files from the component store.
https://www.usenix.org/conferences/upcoming
Note the non-existence of a 2022 or 2023 LISA.
A lot of the directory structure of POSIX systems is completely obsolete in the 21st century. The difference between sbin and bin? Historical, for the most part, and irrelevant in the age of rescue systems.
One question that is nagging at my mind with this: What if two programs require the same lib, how is this resolved?
Because, when I install progA, the lib is symlinked into the index, so when I now install progB, the lib is installed again if I understood correctly. Storage is irrelevant, the pkg manager sees there is a symlink, so it stays in place, all is well.
But now I remove progA, so what happens? progB is still present, still requires the lib, it is also still installed, but now the symlink, which refered to /Programs/progA/libs/thelib.so is broken. How is this situation handled?
Take journals in systemd for example, they are better than plaintext log files in some cases, but now you are stuck with a format that needs to be interpreted and parsed before it is useful.
That doesn't mean improvements shouldn't be done, but every step that is a big change often also makes it impossible to do some basic things that used to be possible for decades. This is similar to analog radio (i.e. only requiring a diode, tunable capacitor and a coil to receive audio! - no power supply needed!) vs. DAB. DAB has a lot of improvements over the base feature, as well as a whole lot of new features. But it is no longer possible to simply receive the radio waves and get audio.
In a way you lose simplicity to gain sophistication. Sometimes we get sophisticated simplicity where a complex problem was solved in a very neat and elegant way that can be reasoned about with ease, but most times it isn't the case and that makes me (and a whole lot of other people it seems) wonder how we can get the improvement without losing the simplicity.
For the very simple "the filesystem is the database" concept, this might actually be possible, all we really get is a well-known structure and versioning with symlinks, similar to the alternatives system used in Debian. But then we get /Programs which is capitalised which looks nice but mixed capitalisation seems too much of a "they are doing it, thus so are we" thing, while on a filesystem level there isn't any gain. Same with spaces in names or dashes. For visual representation that is nice, but everything else it sucks.
I would take out the capitalisation and add some sort of unionfs/aufs/overlayfs hooking (symlink into a bundled filesystem would be one way) so from the system's perspective it's still a single filesystem tree but you add the option to get all those newfangled distribution methods in there as well. The problem we still haven't solved is versioning, but since it is arbitrary at this point /programs (or just call it /apps) could just contain <shortname>/<versionstring> for the application name and then version of the application. If you then dump in a version that comes from a docker image, flatpak, some weird weine-dxvk-steam construction or something else you symlink it in as a <versionstring> and then it's no longer the problem of the distro.
There is still the issue of partitioning off parts of the system you don't want to be touched or want to have safe; a RO-snapshot plus user-overrides could work, but you might end up with a huge RO-snapshot that needs to be entirely replaced, and then a billion override mounts on top of that...
As simple as Gobolinux presents it, this only seems to be useful for a single distro with a single distribution point, which is no longer where we are. Even FreeBSD is past that and that's not even Linux.
If you shouldn't have to care, if there should be a layer of Unicode normalization happening, why is that okay but case normalisation is not okay? If you do have to care, then you no longer "know exactly what you get".
You do know you get exactly what you put in, whether that is é (U+00E9) or é (U+0065 U+0301). When you reference files created by yourself, this is not a problem as realistically speaking, your input method will only have a convenient way of forming one of these, and it will consistently generate the same one every time. When you reference files created by someone else, this may be a problem but no more than e.g. the distinction between file.txt (lowercase L) and fiIe.txt (uppercase i): from the user's perspective, the problem is pretty much avoided by selecting the file using tab completion, TUIs, GUIs, whatever you use.
Just want to remind a little but extremely important thing: There's absolutely no guarantees that upper("i") is I. There's at least one language breaks that convention, and you won't believe how a big headache is that.
Windows NTFS is sometimes described as case-preserving, case-insensitive; i.e. if you name a file "Test" it will stay in that case, but if you ask for "test" it will find the file "Test". I don't know whether that happens in Win32 or NTFS, but it seems like best of both worlds; I don't want "case insensitive" where it could show me the name in a different case than I entered.
I don't override the font, either HN or FireFox or Windows is picking a serif font. I assume you didn't explicitly choose a font where different things look the same, but if it rendered 'a' and 'Z' the same glyph, you wouldn't say that was a problem with the English alphabet or with case sensitivity, or anything other than bad font design, right?
Apple's APFS will create the folder as "Programs" if you use "mkdir Programs", and will give you an error when you try to create "programs". You can navigate with any case combination, but it just plays along like the capitalization you entered is present. It feels like a hack tbh.
Another thing I noticed that I never notice that I'm pressing shift while capitalizing. It's not a deal-breaker issue for me (while this might sound like "it works for me", I'm sure that I'm not alone in that).
Case insensitive systems created a lot of problems down the road. Esp. due to popularity of Windows. Program porting is a pain, using macOS with real case-sensitive mode is again a pain for some scenarios. It also encourages sloppy programming practices and "I don't case as long as it works" ism.
What is this? It's not capitalised so it can't be an acronym or initialism. And your username doesn't start with a capital like a Proper Noun, so it's unreadable. This is my problem with case-sensitivity, it's not how humans use writing, think about writing, design fonts, etc. 'a' and 'A' are the same letter, they're the same letter in the alphabet, they are pronounced the same, they make the same word "apple" or "Apple" which is also pronounced the same and means the same, they could be rendered the same in a given font without making words unreadable or confusing (e.g. small caps fonts).
If the computer isn't letting me use "apple" or "Apple" interchangably as I (the user, supposed to be in control here) wish, that isn't helping me, it's nitpickingly hindering me and making me implicitly care about character encoding values I can't even see. This is dreadful UX.
Domain names aren't case sensitive, email addresses could be but generally aren't, usernames could be but generally aren't, phone numbers aren't, Twitter handles aren't, my bank cards have my name in ALL CAPS but it's still me, you wouldn't join an IRC channel and think it okay to use a nick which matches someone else's but for capitalisation differences, you know which book I mean when I say "the martian" and so does Google search and my email search, PostgreSQL table and column names aren't case sensitive.
It's nice, convenient, helpful when identifiers aren't case sensitive.
It's a case insensitive "to be honest". Non capitalizing it wasn't intentional, but it has shown to be a prime example how case also conveys meaning.
> And your username doesn't start with a capital like a Proper Noun, so it's unreadable.
No, it's a combination of my name and surname, and a play on C/C++ header convention, more specifically "bayindir.h", which is based on a joke one of my university friends. See? Again case conveys meaning. In this case, my username is case sensitive and has a meaning when it's all lowercase. Bayindirh won't do in this case.
> it's not how humans use writing, think...
Case carries information, conveys meaning. "apple" and Apple are not the same word, doesn't point to the same "thing". Computer is the ultimate data storage and manipulation device we have, and I don't want ambiguity in any parts of it.
If humans didn't use cases, and this is not how we use writing, i can just write will all lowercase letters and make distinctions between apple the fruit and apple the company, or turkey the animal and turkey the country some other, more convoluted way. Cases matter. They carry information. Try to tell a mathematician or any scientist that their formulas are moot because X and x are the same variable.
> If the computer isn't letting me use "apple" or "Apple" interchangably as I (the user, supposed to be in control here) wish, that isn't helping me, it's nitpickingly hindering me and making me implicitly care about character encoding values I can't even see. This is dreadful UX.
I'm the opposite. I consider apple being equal to Apple "Dreadful UX", so it's subjective. IMHO, every user have preferences and a computer can adapt to that. Given the choice I'll prefer case sensitive, you'll prefer case insensitive. That's OK. What's not OK is pointing a finger to a property and calling "This is useless!"
On the technicalities side, there are some capitalization rules which make case insensitive systems shiver and break. For you probably i and I are same. For me it's not. We have i, İ, ı, I. How to make this case insensitive? A lot of programs break because of this. This is fun. This is called Turkish.
So, no. Neither a and A, nor i and I are the same letters. They don't have to sound the same either.
> Domain names aren't case sensitive, email addresses could be but generally aren't, usernames...
Case insensitive passwords and usernames are big security nightmares, including the technicalities I've described above. Names on credit cards are built upon the assumption that capitalization works the same everywhere (it isn't). Just because Google wants to match something case-insensitive doesn't make it right.
Some things are designed to be case insensitive against the norms. In normal human language case conveys meaning. Some conventions decide to ditch that meaning. Many programming languages are case sensitive. Are they wrong? No they're not. SQL statements use cases to denote keywords, even if they're case insensitive.
Case conveys meaning. Ditching is a choice.
> It's nice, convenient, helpful when identifiers aren't case sensitive.
In some cases yes, in most cases no. Try to lower() this comment, and try to read it. It may not be nice, convenient and helpful.
I have occasionally complained that I find math notation frustrating to read because of the use of the same letter with a tiny variation, maybe an unpronouncable one. Mathematicians are not particularly interested in what outsiders find difficult (why would they be?).
I have occasionally enthused about PowerShell, a shell/scripting language where string comparison is case insensitive (by default, sensitive if you want it), where variable names, method names, object properties, function names, are all case insensitive, and how wonderfully convenient, liberating, easing and brilliant I think that is for an interactive shell experience. And about other simplified systems such as APL and Dr Iverson's work to simplify math precedence rules and how its builtins try to work the same over scalars, vectors, and multidimensional arrays. About Simplified Music Notation and their work to reduce cognitive load by in-lining sharp and flat information into the shape of the note instead of having to remember it from the beginning of the bar; the Esperanto language and the 'clockwork-morphology' where all the patterns about past/present/future apply to every verb with no exceptions for strong verbs or loan-words, and how much easier that makes it to learn.
Humans use case flexibly, it adds meaning but it can be overridden. News headline "Apple Announces New Computer" means something different from "Fruit Grower Announces New Apple". Humans now have to deal with more and more different systems, the fewer nitpicking rules humans have to learn for each one to be able to do that, the more the systems adapt to humans, the better.
> "Computer is the ultimate data storage and manipulation device we have, and I don't want ambiguity in any parts of it."
I don't want ambiguity, mapping "a = A" for information retrieval isn't ambiguous, it's completely specified. I don't know what it would do in Turkish, but your own example tells me you can see those four things as variants of 'i' differing in capitalisation and you don't confuse any of them with K or Z. If you can, computers should be able to. I would expect datetime and number formatting to be different in Turkey, and capitalisation too. If i, İ, ı, I are different, then they are different and need to be carefully specified each time, that Turkish case shouldn't mean English a-z and A-Z must be considered different.
> "I'm the opposite. I consider apple being equal to Apple "Dreadful UX", so it's subjective."
You consider it "a dreadful experience" that someone searching Google for 'apple computer' gets results instead of nothing? That "December 2021" and "december 2021" are considered the same month? Someone who doesn't remember the correct capitalisation for XF86Config, what should they search for to find it in your world? All possible case variants? One way they could find out is to list a directory which contains it and read the listing, and the only reason that works is because their brain can recognise 'the same thing' without knowing the case(!). Humans do it, machines should do it (unless explicitly overridden).
> "Case insensitive passwords and usernames are big security nightmares"
Passwords yes. Usernames, no. Case sensitive names are a security nightmare, they lead to problems like phishing. It's bad enough getting an email from @bankofamerlca.com but if the official one was bankofamerica.com and BankofAmerica.com was a scam domain, what chance would anyone have?
Like, apple and APPLE are the same company name in the UK! Search for 'apple' on this government site https://find-and-update.company-information.service.gov.uk/c... - and it says "The company name "APPLE" is considered to be the 'same as' the existing names shown below: APPLE.COM LTD". If that's ambiguous, a computer system should represent the ambiguity of real life.
> "In normal human language case conveys meaning."
But the case difference is not pronouncable, not memorable. I don't have to learn apple and Apple as two different words, I change the case on-the-fly as I need it. Same way I would not want to condiser apple and apple as different words where I have to get the italics correct before the computer will find them, or underlines, or bolding, or colour, I don't want to have to specify the way the news headline was written before I can find it - not font size or emphasis or capitalization. "README.TXT" is still at the core saying "read me" not anything fundamentally different.
> "Try to lower() this comment, and try to read it. It may not be nice, convenient and helpful."
Try to transcribe a talk and write the case they are speaking. They aren't speaking cases, you have to guess from your knowledge of language and work out from context which things are names and where sentences begin. Case isn't present in speech but it still carries meaning.
They are case-sensitive. The thing is however that is that PostgreSQL will automatically lowercase names that aren't explicitly double quoted.
Also imagine having two files in the same directory which differ only by case. What would be the benefit of being able to have two or more such files?
Are there any use-cases for which case-sensitive file-system is better than case-insensitive one?
Actually, not having the exact file may not provide a utility for you, or most of us, but searching for a file in a case sensitive manner, and getting ENOTFOUND just because it doesn't present in the way I searched is valuable from my point of view.
Also, having a case-sensitive system is simpler from technical point of view, and prone to less bugs. This is especially true after UTF-8 become ubiquitous. Searching for an exact byte stream is much more simpler than a country dependent, exceptions filled search/match algorithm.
Most shells can be configured to behave that way, and Fish works that way by default.
Remember that some systems may have multiple users (some may even have multiple simultaneous users) who use different locales. Or a single user might run different programs using different locales, e.g. if a program has a bad translation in their native language, they may prefer to run it with a different language that they also speak.
A lot of computer systems only got rid of the problem of storing filenames in locale-specific character encodings (e.g. ISO-8859-*) relatively recently, by switching to a locale-neutral encoding (UTF-*/UCS-*). It would be nice if they could not repeat some of those problems with locale-specific filename matching rules.
You could remove the problem of "requiring you to type the name with exact case" by having shells' autocomplete feature fix case issues. So you could type "vi foo<TAB>" and the autocomplete change that to "vi Foo.txt". That could have the advantage of also fixing combining character normalisation issues. And the question of "which locale's rules get used?" is easy - the locale of the shell doing the autocomplete.
You install ProgA and the lib is symlinked into the index You install ProgB and the index's symlink to ProgA's lib gets some information added to it along the lines of "if the symlink to ProgA/lib is broken, try ProgB/lib"
Then when you uninstall ProbA the fallback symlink is activated.
At some point you would want to clean up the dangling primary link but maybe the best time to do it is immedately. i.e. the first time the primary fails you make the backup the new primary and now there's no backup, i.e. it looks the way it would look if you had only ever installed ProgB.
Shared libraries actually being shared is the exception and not the rule
For all the hate of npm and node_modules it’s wonderful to have this isolation
But since there can be only ne symlink to the library (because the lib is searched by name in the index) the question remains: How is the package manager handling this for Gobolinux?
Looks like you could use
find /System/Index | RemoveBroken
In a way this is inside-out from what Nix and Guix do, where their equivalent of the System Index (the Nix/Guix store) is the build target, and everything all symlinks point to that. That makes this kind of thing a little easier to deal with, at some other costs.
From the answers There is no package comes with a dependency, there is a pkg depends on a well dependency
Either you are asking a deeper question or comes with a dependency is an unusual terminology
~] ls -l /System/Index/lib
libgtk.so -> /Programs/GTK+/1.2.10/lib/libgtk-1.2.so.0.9.1
libgtk-1.2.so.0 -> /Programs/GTK+/1.2.10/lib/libgtk-1.2.so.0.9.1
libgtk-1.2.so.0.9.1 -> /Programs/GTK+/1.2.10/lib/libgtk-1.2.so.0.9.1
~] cat /etc/ld.so.conf
/System/Index/lib
Personally I think it would have been interesting to version the system index itself, like /System/Index/Current|1.0|myappenv|etc/lib|bin|etc, and each app is chrooted into their own system index by default.
The question is, when 2 packages, pkgA & pkgB bring in the same library, call it "thelib.so" into the system, logically, only 1 of them can be refered to by the symlink in System/Index/lib.
So we have both
/Programs/pkgA/1.0/lib/thelib.so
/Programs/pkgB/1.0/lib/thelib.so
/System/Index/lib/thelib.so -> /Programs/pkgA/1.0/lib/thelib.so
My question is: how is the pkg management system resolving this?
https://github.com/gobolinux/Documentation/wiki/Removing-pro...
https://github.com/gobolinux/Documentation/wiki/Understandin...
/System/Index/lib/thelib.so -> /Programs/thelib/1.0/thelib.so
But Gobolinux can actually handle this scenario better than most distros since package-specific libraries don't need to go into the global index in the first place. So neither pkgA nor pkgB's 'thelib.so' would end up in the index, it'd just be local to the package.
So it would be better if either there were no `/sbin` or if it was always in the PATH.
[1] https://github.com/blueman-project/blueman/blob/fcef83a01c80...
Other projects need to sit still so one project doesn’t have to change one line of code?
Nonsense. Keep up or get forked (that’s not a euphemism for vulgarity). It’s not esoteric magic like it used to be.
Or probe for its location Rather than hardcore one or the other
Everyone happy
Hrdcoded paths for things a program doesn't bring with itself, should be avoided in general. There is a reason $PATH exists and is easily extended. It is the users/admins responsibility to make sure envvars are set correctly, and it's the programs responsibility to give a useful error message when it cannot locate the programs it expects in $PATH
My favourite one is how people try to come up with reasons for the /usr dir and why /bin and /usr/bin both exist. The original reason was that one of the original Unix devs ran out of space on their hard drive and needed a second bin folder for the new drive.
EDIT: didn't realize you were talking about Gobo specifically; not sure how they deal with this, but theoretically a similar strategy seems like it could work
The historical difference is that in /sbin there are binaries meant to be executed by root, and thus they were not be put in the path of non root users (Debian still do that), so you don't get binaries that you can't run in your path (these days it's stupid since you may want to use sudo and thus have them in path only for the shell autocomplete to work properly).
By the way I prefer how things are nowadays, otherwise you get the same confusion that I have every time I use Windows or macOS that you either have an infinite PATH variable or have to specify the full path every time.
Yes, the package manager can make symlinks, but at that point what is the difference?
Also having everything in /usr/bin has the good side effect of avoiding that by mistakes someone creates a program that has the same name of one of the thousands of programs that are in the repository of a distribution, to avoid confusion to the user that can launch a program or another depending on who comes first in the path.
Could it be that docker is missing something, maybe the Dockerfile isn't easily reproducible? Maybe the ephemeral status of all containers is quite some effort to work with?
Maybe it's just not the one true answer to the question of software packaging.
I love Docker for the simplicity it offers, but the gigabytes of storage wasted on yet another slightly different version of Debian every time I pull a container irks me to no end. Sadly, it's the only way some software is distributed.
Where are you seeing distros using docker for containerization? If anything, I'd say Flatpak is the one you're most likely to find installed in a 2021 workstation linux.
Docker's killing features are the same as they always were: usability and ecosystem. 'docker run thingamajig' is simple and DockerHub is very likely to have thingamajig in its repos.
Of the ones you listed, I'd say flatpak is the only one that compares favorably to Docker on both aspects ('flatpak install thingamajig'). And since flatpak is more designed for desktop applications, it's actually the one gaining ground. Docker or other containerd-based solutions should remain the preferred choice for servers, since they're designed for that.
Nix is vastly less user friendly. Snaps, last I checked, still don't have an open-source repository implementation, meaning that Canonical has absolute control over them. I don't know much about AppImage, but AFAICT it's just a file format and not a full package manager.
Another problem is that you want to run an application on differen platforms, not only Linux. Again the proper solution would be to write a cross-paltform application and install it using a cross-platform package manager.
Docker doesn't offer a proper solution; instead it suggests that you create a virtual machine, install a whole operating system (without a kernel) there and then write a bash script that will download and build dependencies. As docker image configuration file is an executable script, it means that it cannot be processed with automated tools; it can only be read by a human.
Also, docker requires a daemon running as a root. This increases potential attack surface.
Docker is not good for desktop applications because it doesn't have portals which can restrict access to DBus, audio daemons, file system etc.
me too. I broke my system a bunch of times, and eventually just gave up. I remembered the other day that I have a lot of programs installed with it that I don't even want to think about updating. rofi changed it's configuration file format recently, and that was tied up in nix. I am too lazy to fight with it for now, so I've just disabled my config file. It bothers me every time I use it.
If the system evaluates it'll most likely work, if it doesn't you roll back a generation.
I have an Ubuntu container around if I quickly wanna mess with something.
But the downside is that debugging is impossble.
Guix uses guile, which is a Lisp so we could accomplish the same with other languages.
Reproducibility. You need a pure (side effect-free) language to ensure that a package expression always evaluates to the same value [1]. If you have a non-pure language, external factors (e.g. environment variables or a server returning a different response) can influence/change what a package expression evaluates to.
[1] That said, Nix was not completely free of side-effects either. Though this is one of the issues that Nix Flakes attempt to solve.
Pick any programming language. Now, remove access to I/O (except standard input and output), networking, time, random number generators, threads, OS syscalls, etc.
Given an input, the program will always generate the same output, i.e. it is deterministic. However, the program is still allowed to have global mutable state; the state is just encapsulated to the program's memory and execution time.
A trivial example is Brainfuck. It is most certainly not a functional programming language, but it is deterministic.
The devil is in the details, and if you miss even one case then the problems are just as bad as if you hadn't bothered at all. For example, comparing two URLs in Java is not deterministic. For another example, iterating through a set in Python will give you the elements in a different order on 32- versus 64-bit systems.
In practice it's just not practical to retrofit deterministic behaviour onto a language that was not designed for it. Even your "trivial example" isn't; Brainfuck does not standardize overflow behaviour and so the same program may behave differently on different systems.
Well, it does network access, so it would excluded.
> For another example, iterating through a set in Python will give you the elements in a different order on 32- versus 64-bit systems.
This is a good counter-point.
> Brainfuck does not standardize overflow behaviour and so the same program may behave differently on different systems.
Different implementations of Nix might have the same issue in some places. In this case, whether the language is fully defined is orthogonal to whether its behavior is deterministic; here, a particular implementation will continue to be as such.
As far as I know, Nix doesn't have a formal specification; it is defined by its reference implementation.
After all you need to be non-constant over architecture, since you need to download the correct binaries for 32-bit and 64-bit architectures.
The problem is Nixpkgs, the largest Nix program. Nix-the-language doesn't lend itself too well to programming in the large. It's a bit like writing a large JavaScript SPA without frameworks or strict coding standards.
The biggest difficulty comes from the fact that it is a dynamic, weakly-typed language, which can lead error messages in configuration to point you toward library code rather than your config file. This is a pitfall shared by many of the languages that are commonly suggested as replacements or alternatives.
There's been work to solve this by adding gradual typing (like in Typescript) to Nix, whose most promising iteration atm is (imo) a Nix-like language called Nickel, which is almost ready for preview by the community.
The other, more minor, issue is tooling, which is being addressed by emerging LSP work for Nix.
(Scripting is done in Nixpkgs, mostly via Bash.)
The beauty of Nixlang, imo, is that for simple use cases it really feels like a dead simple configuration language, which is only possible with a declarative language.
The functional character and Nix's laziness were important in the early design of Nix. The latter may still give Nix some nice performance characteristics. But since the initial Nix design, Nix has become eager in some places, and Guix is implemented as a lazy DSL embedded in an eager language, so I guess we could implement something like Nix on top of an eager scripting language.
Imperative scripting languages might not be as well suited for the kinds of overrides that Nixpkgs and NixOS use, afaict. But imo aside from its novelty, the Nix language is nice to use because it's really simple and totally declarative, which is what you expect from a configuration language.
> You don't need all values of your language to have lazy semantics
Once upon a time (no longer, I think), Nix was 'maximally lazy', so I think historically at least, there has been a question of how essential laziness is to the design of package managers in the functional paradigm. Since then I guess we've seen what you describe (some targeted laziness) is all that's required in both DSLs.
I've edited my comment above to better reflect Guix's design and stop spreading the 'Guix is eager' meme. :)
There’s an open RFC/PR to bring NixOS style modules to Nixpkgs for package management (with derivations being the primitive).
Here's the paper from Eelco Dolstra (founder of the Nix project) https://edolstra.github.io/pubs/nspfssd-lisa2004-final.pdf
I don't see him mentioning configuration an awful lot in there, it seems to be mostly about dependencies and multiple versions of things.
> > How does GoboLinux compare to Nix & Guix these days? Why would one use the former vs the latter?
> This blogpost has a comparison of Gobolinux and Nix:
systemd compliant?
./configure && make
And such a downstream system could benefit from mimicking GoboLinux in a few ways, imo.
{ pkgs ? import <nixpkgs> {} }:
pkgs.stdenv.mkDerivation {
name = "foo-1.0";
src = ./.;
buildInputs = [ pkgs.libfoo pkgs.libbar ];
}
But I think impurity could be a different kind of on-ramp. See things like envfs or the reverted PR for the LSB module in NixOS.
I'd rather look into how to make the Nix-native workflow be as appealing and welcoming as the impure one, than compromise and end up with a system ultimately just that has the downsides of both.
That's why I say this would be a downstream distro, because then
> and the unreliability of the produced frankenbuilds [can] scare newcomers away
to NixOS proper, which can remain pure, principled, and a little difficult. :)
The idea is a workstation/desktop-centric downstream where people can
* use NixOS modules to configure the base system (which is super convenient)
* learn the Nix language ‘by immersion’ and tooling without *immediately* having to choose between packaging and giving up any proprietary/oddball software they might already be using that isn't already in Nixpkgs
* try Nix in its most attractive form (fully declarative OOTB and in control of a full operating system, unlike Nix on macOS or foreign Linux distros)
* start out with all of the commonsense community goodies (flakes, flake-utils-plus, Home-Manager, NUR, and extra binary caches enabled when relevant) pre-enabled and configured in the simplest possible way
* comes with a reset button (like a slightly gentler NIXOS_LUSTRATE) that users can easily use to remove any impurely installed software
There's no reason that Nix itself couldn't be used to create the same kind of stable, impurely accessible, compatible (with traditional tools), kind of base system. I think it'll happen some time in the next few years. There's much wider interest in NixOS than their used to be, and with that has come increased interest in making it even more accessible to new users.
Now admittedly actually doing this is pretty rare in these days, but I still like having the option. I believe he does address this talking about "union mounts" and "overlay filesystems". I'm really not too familiar with either or how production ready they are, but it may address my concerns.
It’s pretty frequent! When Atlassian launched their Cloud offerings in 2013, they installed Confluence and Jira on their own servers (1.5GB each), one per customer, and set /bin and /etc as read-only.
Then they mounted /etc and /bin from the network. 1.5GB saved per instance!
So it makes sense not because you can set them as read-only, but because you can mount them separately.
Only beef I have is the CamelCase of GoboLinux CLIs.
We all can do without shifting the shift keys as much as possible.
Unless the ultimate goal is for 100% GUI mouse-click operation, which it does not appear to be.
I sense a tinge of hostility toward typists in general.
Caps leading command names is not great with case-sensitive filenames imo.
And it shouldn't be possible to create two or more separate files in the same directory which differ only by case.
But they were right on case-insensitive.
Most end users rarely interact directly with the filesystem, so a different hierarchy won't impact many people. Installing up-to-date packages that do not threaten the stability of your system is easy with flatpaks/snaps; it is even possible to install GNOME Software add-ons to handle them. If you want to install "system-software" that is not possible with snaps or flatpaks, there's always Nix, Guix or homebrew. All that without considering AppImages, of course.
So... gobolinux is a great idea. But it fixes problems that are much smaller today.
It's not for me I think (nor is Nix), but that's the nice thing about Linux, everyone can do things their own way. I mostly use FreeBSD anyway.
It's theoretically closer than ever, thanks to stuff like nix-processmgmt and Initware for the BSDs, but it feels like something that keeps coming close only to fade away :'(
But Nix can absolutely do it, and it could be so extremely good!
(There's lots of progress in the Nix world to enjoy and celebrate, as usual, but it's just that type of project that makes you think of all kinds of crazy new things it could do)
I am wondering if we should do some sort of course source to get some basic smoke test cross-compiled VMs for multiple kernels up and running.
Just get over that hump, and I think it's all down hill and will work with our usual nixpkgs crowd-sourcing.
What do you think?
What do you want to start with? FreeBSD + Initware (so that we can reuse the existing NixOS systemd service file generation stuff?)
I think I'll try messing around with them a little on their own to get a feel for what they need in order to run
I do want to finish my FreeBSD cross PR, but is is hard. We already have NetBSD working, including building the kernel, so I think it is better to start there.
I opened https://github.com/cleverca22/not-os/issues/16 because I think that would be the absolute easiest first step, with the fewest moving parts. But I don't know how kernels, even Linux, are packaged into bootable thingies at all.
I do think crowdfunding is a decent idea, and I'd be willing to support someone who knows what they're doing :)
* NixOS supporting kernels other than Linux
* NixOS supporting init systems other than systemd
* NixOS supporting usermode services (i.e., code sharing between Home Manager modulea and NixOS modules)
* NixOS supporting a 'guest mode' for configuring system services (i.e., NixOS foldinf in Nix-Darwin and supporting similar systemwide service configuration on other foreign operating systems).
Some people are asking how it compares to some cross-distro package managers.
The answer is, it doesn't – but it does seem to me that it would be very cool to see Gobo contain support for Flatpak, Snap and AppImage.
This would alleviate its shortage of apps somewhat. Keep the distro simple, and just don't package apps: outsource that to the cross-distro package formats.
Secondly, ISTM that Gobo would be a natural fit for a desktop that already treats certain specially-structured directories _as_ apps. I only know of 2 of these: GNUstep, which uses essentially the same `.app` bundles as macOS, and the ROX Desktop, which has !AppDirs that are closely modelled on Acorn's RISC OS. This, incidentally, is the internal format of AppImage.
https://www.nayuki.io/page/designing-better-file-organizatio...
Everything remains in a hierarchical structure (or a few, if you consider the symlinks).
The notion of tags does away with a hierarchy.
Their documentation gives examples of primary files stored at locations like:
/Programs/Bash/4.4/bin/bash
/Programs/Netkit-Base/0.17/bin/ping
/Programs/LibPNG/1.2.5/lib/libpng.so.3
/Programs/Glibc/2.24/include/stdio.h
/System/Index/bin/bash
/System/Index/bin/ping
/System/Index/lib/libpng.so.3
/System/Index/include/stdio.h
Generally speaking, tagging can be half-simulated using hierarchies and (hard/soft) links. But a tag-based system would be cleaner and not prioritize one retrieval method over another. Here is an illustration of how I might rephrase GoboLinux's package storage and retrieval system in terms of tags:
Blob("... binary data of bash executable ..."), with hash = 9e19e455.
Blob("... binary data of ping executable ..."), with hash = 28224f5b.
Blob("... binary data of libpng.so library ..."), with hash = 6243c115.
Blob("... binary data of stdio.h C code ..."), with hash = d0335126.
Tag(packageName="Bash", version="4.4", target=9e19e455).
Tag(packageName="Netkit-Base", Version="0.17", target=28224f5b).
Tag(packageName="LibPNG", version="1.2.5", target=6243c115).
Tag(packageName="Glibc", version="2.24", target=d0335126).
Tag(traditionalDirectory="bin", fileName="bash", target=9e19e455).
Tag(traditionalDirectory="bin", fileName="ping", target=28224f5b).
Tag(traditionalDirectory="lib", fileName="libpng.so.3", target=6243c115).
Tag(traditionalDirectory="include", fileName="stdio.h", target=d0335126).It is convenient in case-insensitive filesystems like those used in Windows or Mac, but not in Linux. By the way it would be better if Linux also switched to case-insensitive filesystem, or even a filesystem where similar looking characters from different alphabets are considered the same. So that A is always A no matter what alphabet is used.
Also I don't like the way they hide classic directories like /bin. Instead, they should just not create these directories; well-written programs should not require their presence.
I cannot imagine a situation where you need two files whose names differ only in case.
"The true horrors of HFS+ are not in how it’s not a great filesystem, but in how it’s actively designed to be a bad filesystem by people who thought they had good ideas.
The case insensitivity is just a horribly bad idea, and Apple could have pushed fixing it. They didn’t. Instead, they doubled down on a bad idea, and actively extended it – very very badly – to unicode. And it’s not even UTF-8, it’s UCS2 I think.
Ok, so NTFS did some of the same. But apple really took it to the next level with HFS+.
There’s some excuse for case insensitivity in a legacy model (“We didn’t know better”). But people who think unicode equivalency comparisons are a good idea in a filesystem shouldn’t be allowed to play in that space. Give them some paste, and let them sit in a corner eating it. They’ll be happy, and they won’t be messing up your system.
And then picking NFD normalization – and making it visible, and actively converting correct unicode into that absolutely horrible format, that’s just inexcusable. Even the people who think normalization is a good thing admit that NFD is a bad format, and certainly not for data exchange. It’s not even “paste-eater” quality thinking. It’s actually actively corrupting user data. By design. Christ.
And Apple let these monkeys work on their filesystem? Seriously?"Inside the module you'd prefer to follow conventions like name classes title-case and name functions camel-case.
Thus you might have one module exporting a class named "Camel", and another module exporting a function named "camel". You could then have two files, Camel.js, and camel.js in the same directory, containing two separate programs.
There's no such thing. What's the lowercase of ‘I’ — ‘i’ or ‘ı’?
Pick something.
Also you don't need to have an opinion on how to lowercase, you can just say that İIiı are equivalent and it shouldn't make people that upset.
Also, please, no whitespace in filenames. Allowing linebreaks is just insane.
Also, my native language does not use the Latin alphabet. Attempts by American companies to be more inclusive and internationalized are often misguided. MS is especially nasty with its translated formula functions in Excel, but having differently named filesystem paths in different alphabets depending on the phase of the Moon or something is also very inconvenient.
Filenames with terminal control escape sequences are lots of fun.
Limiting possibilities to just lowercase Latin, numbers, periods and dashes (in non-initial positions) decreases chances people see symbols they do not understand by a lot.
By the same logic the whitespace sensitivity of Python is a bad idea. :-)
The response to your second problem is that not all programs are well written, and they cannot afford to patch every poorly-written program.
I'm personally still on the fence about the more verbose style. I think I like it for directory names but not really for executable names.
Fri Jun 7 12:13:33 1996 Bob Glickstein <bobg@hiro.zanshin.com>
* Created stow, formerly "depot."They call it "rootless". It runs inside your home directory.
Flatpak is an app packaging system, distro-neutral.
GoboLinux is a distro and a new, different way of organizing the Unix filesystem.
Flatpak contains conventional (or at least similar) filesystem layouts inside app packages. Gobo replaces the contents of apps as well as the whole filesystem.
Flatpak requires runtime infrastructure on the distro you wish to use. Gobo is a distro.
Flatpak is distro-neutral; Gobo replaces other distros.
Flatpack puts all apps in its own hierarchy deep inside /var and manages it itself. Gobo does not have a /var, or a /bin or /usr or anything else.
Flatpak could be run on Gobo. Gobo could not be run on Flatpak, because Flatpak is for apps.
So this is a very broad request and could take tens of pages.
Flatpak installs apps on any distro which has or has had Flatpak support installed on it. It does this by unpacking apps into its own directory trees. Apps go into:
`/var/lib/flatpak/exports/share/applications`
Inside there are more complex trees, so for instance, to reply, I just installed Sublime Text, and it is in:
`/var/lib/flatpak/app/com.sublimetext.three/current/active`
Inside that is a `files` folder. Inside that we see an encapsulated version of a traditional xNix filesystem:
total 40
drwxr-xr-x 8 root root 4096 Jan 5 12:30 .
drwxr-xr-x 4 root root 4096 Jan 5 12:30 ..
drwxr-xr-x 2 root root 4096 Jan 1 1970 bin
drwxr-xr-x 4 root root 4096 Jan 5 12:30 extra
drwxr-xr-x 2 root root 4096 Jan 1 1970 lib
-rw-r--r-- 2 root root 4817 Jan 5 12:30 manifest.json
-rw-r--r-- 1 root root 0 Jan 5 12:30 .ref
drwxr-xr-x 6 root root 4096 Jan 1 1970 share
drwxr-xr-x 2 root root 4096 Jan 1 1970 sublime_merge
drwxr-xr-x 2 root root 4096 Jan 1 1970 utils
Flatpak is a wrapper around traditional xNix directory layout. It encapsulates them inside a layout it manages, allowing apps to be transported between different distros by having their own layout inside a private tree, but it's a fairly conventional xNix tree.
With me so far?
The conventional xNix layout has globally-shared folders, segregated by functional roles: this subtree is for sysadmin stuff, this subtree is for all users; this subtree is on the main disk and is needed for boot, this one is not essential for boot and is only used in multiuser mode, this subtree is auxiliary additions later, this one is local to this network node only, and so on.
This was needed in the 1970s with very small disks that could not hold the entire OS. `/usr` was originally for home directories but some of the people using the OS were also simulatenously writing the OS. Stuff in their home directory became essential to the system. So it got fossilised into place meaning that a new `/home` tree was added later.
The Mac and Windows conceptual model is different.
It says, each app lives in a separate tree of its own, and we will put only files expected to be shared with other apps into system-global locations.
So the OS itself is in `\OS\` and all the components of the app Bar by Foo Corp get put into `\apps\foo\bar`. Any shared libraries get copied into `\OS\libs\` so that other apps could use them. Later this was found to be a mistake and so `\app\Foo\bar\libs` was added for a known good copy.
Gobo discards the traditional xNix model of global shared folders containing binaries from multiple sources, and segregates them in a Mac and DOS-like model. There is no real `/bin` or `/lib` folders. Instead, there are new trees: Programs, Users, System, Data, Mount etc.
Inside Programs is a tree for each app, and inside that, for each version of that app.
xNix: the files containing the components of each apps are split up and put in the system-global dirs for that type of component. Apps do not have their own dir. Config is in the config tree, libraries are in the libs tree, binaries are in the binaries tree, etc.
Flatpak manages its own version of the global OS layout, one per app, inside the Flatpak tree. Apps are not put into the OS global dirs but each has its own private version of the OS-global tree inside its own dir tree inside the Flatpak tree.
DOS, Windows: the OS lives in its own tree. Each app has its own subtree in the Apps tree and copies only shared components into OS-managed shared folders.
Mac OS X: the OS lives in simplified xNix tree. Apps live entirely in their own private shared tree, one per app, with an OS-mandated layout.
Gobo: the Mac OS X concept is extended to the OS itself. The OS' components are in private, versioned folders inside an OS-mandated tree, with the traditional xNix layout hidden. For legacy apps that expect trad xNix dirs, they are there but populated only with symlinks and the OS hides them from the user.
Flatpak adapts the xNix model and encapsulates private versions of it.
OS X keeps some of the xNix model for the OS only and enforces a new private-dir-tree model upon apps.
Gobo takes the app model of OS X and applies it to the entire OS, and only fakes the trad xNix model to make it easier to port legacy code.
I think you're missing the point here.
It's not "apps are packaged with all their dependencies inside a private directory hierarchy". That is true of both, yes. But it is also true of Canonical's Snap format, and 0install, and of AppImage, and of NeXTstep & Mac OS X's .app bundles. In other words, it is far from unique.
The real difference with Gobo is twofold:
• Most of the others preserve significant elements of the traditional xNix directory layout. macOS in the main OS if not inside the bundles; Flatpak and Snap inside the apps; and so on. Gobo gets rid of it everywhere, top to bottom, from kernel to text editor.
• Flatpak, Snap etc. are just app-packaging systems. Even on OSes which use Flatpak exclusively – and PopOS is a poor example here; better ones are Fedora SilverBlue or EndlessOS, with read-only root filesystems, no package manager at all, and OS updates using `ostree` – the directory layout is conventional. Like macOS, the app directory structure is entirely separate from the OS directory structure. In Gobo they are one and the same.
I wrote about that here, and you may find it worth a read: https://www.theregister.com/2021/12/03/nixos_linux_os_design...
I believe it used to default to installing Windows NT to C:\WINNT... but that was almost certainly longer ago than you're suggesting. To be fair, the Desktop also used to live at C:\WINDOWS\Desktop, but there are some pretty good reasons why it doesn't any longer.
The closest thing I can find is this https://docs.microsoft.com/en-us/windows/win32/shell/knownfo...
Just my personal opinion.
https://techcommunity.microsoft.com/t5/windows-blog-archive/...
(My personal recollection is that it is much earlier, at least back to XP, but I could be wrong, so I’ll go with what I can find documented.)
On some versions it was also "C:\Documents and Settings\...", it is so convoluted that even I don't remember all the specifics, but yes, it was a total mess, especially on non-english installations