Unix Delenda Est (2015)
housejeffries.com
housejeffries.com
What I like about this post is that it's trying to point out some specific things that are perhaps the most constraining, though I'm not sure the solutions or conclusions are correct.
The question is the right one though.
As other people have pointed out in this thread pretty much everything I talk about has been implemented by one program or another. It hasn't all been collected into one user environment though, and without doing that it's hard to envision the details of how things in that environment should fit together.
>Tagged Filesystems
While not in common use, these exist now. And you can run them on your unix machines. Look up BeFS.
>Typed Data
...I'd rather have 1000 ops on 1 crappy data structure than 10 ops one 100 good data structures. And that's what you usually end up with, regardless of what you're trying for.
>Immutable Data:
This is actually something I'd like to see. OH WAIT, I have, it's called git. Now we just need to integrate it into an FS. It's also called Nix, for packages specifically.
>Server First:
Plan9 did it, It's been emulated in userspace, It should have caught on.
>Decent Shell Lang:
Python and Ruby anybody?
We really should make those defaults.
>Reasonably Secure.
Yeah, we need to fix this. We need less C.
>Better Security Model
We'll implement it if somebody can design one that Fking works, and isn't Fing SELinux.
>Browsable Executables.
Yeah.
>Sandboxes
THE DOCKER REVOLUTION HAS ALREADY COME.
You get the best of both worlds, good package management within containers, reproducibility, etc.
With Docker or apt you only get half baked solutions.
In a strict sense, almost all software accessing standard hierarchical file systems is even just broken -- it breaks for certain concurrent operations, and there is no way to forbid them.
But still hierarchical filesystems are the best usable-by-default model I know. It's a humane approach from a cognitive standpoint: the real world of "things that exist only once, in one place" is inherently hierarchical.
For different needs, it's often easy enough to flatten data a bit and use tools like filename globs or regular expressions, to get a performant, good enough solution that is also maintainable.
For very specific needs, you can always use a relational, or whatever, DB.
The linked to "worse is better" article I have read before, years ago. The criticism of the failure mode of Unix is invalid. If you hit an error condition, then unless you have have kept your systems state for an indeterminate period of time, it's often not feasible to back out of the program to where the error occurred.
In terms of tagging, that's never taken off. Why? Because it turns out tagging things is harder than you might think, because you can call the same thing by different names, which is somewhat ironically a major problem with hierarchical file systems. You basically end up with needing to be able to locate all the tags that are synonyms of each other.
At the very least, a file system comes with a convention. I know where configuration files are on Unix - I go the the /etc directory. If I want to find my executables, I go to either /usr/bin or /bin. For my files, I'm under /home/chris.
If I was to tag stuff, I'd also need to stick to a convention, or chaos reigns. I know, I've tried this before. I basically ended up with a hierarchical tagging structure.
That's not too say tagging is awful, only that it's not a panacea to all perceived ills.
As for version control - git is honestly the best example of a versioning system that does everything you describe - content addressing, versions control, snapshots... And guess what? It's all in a DAG.
Oh, and one more thing: all that "untyped text" that is a "bad persistence model" firstly ignores the fact that it's not untyped, that every file system contains metadata, and that a file name is no different to a tag that is a label.
I guess for software packages some sort of hierarchy seems essential. At least for documents I always imagined that tags would be perfect. It seems that documents always fit in multiple hierarchies...
On the other hand, if you really want no hierarchy in software packages, some standard tags are necessary. Like 'ConfigFile', 'Active', ... Also one should keep in mind that data in software is also in most cases not hierarchical. Hierarchical databases are a weird animal that nobody uses. Maybe hierarchical filesystems have constraint our minds so much that we cannot easily imagine some other way.
Regarding the rest... I think as well that Unix has some problems, mostly due to old design. Having to use regexes to match file names is so error-prone, I think only for document file names that would be okay. I really like the innovative approach of Windows Power Shell eventhough the syntax is still too weird.
The reason it is ubiquitous is that the human mind needs order, and a filesystem imposes this. It ain't perfect, but it works.
Of course the whole desktop computer is a metaphor of the desk and the first commercially successful graphical interfaces resemble a desk.
And GUIs are what most people use most of the time, even most devs tend to start their terminal from the desktop in a window, which appears a bit like paper, laying on the desk.
It's interesting that only few graphical operating environments break with this convention. It seems there hasn't been much conceptual innovation in the space of "desktop computing". Even the tablet seems to represent either a piece laying on the table or the table itself, depending on the size.
It should be more efficient to design systems based on the available digital infrastructure, instead of designing them based on partially obsolete office table infrastructure.
If I know I left a file in a folder, I know it will always be there. If I tagged a file with some sort of mnemonic, then it's on me to remember what burst of inspiration told me to tag it as such.
You know, this is all said very earnestly
but the problem is that it tends to ignore
the fact that many, many great things have
been achieved with current technology.
Obviously, we're crushing it as (1). Linux, PostgreSQL, V8, GHC and so on are just disgustingly amazing feats of engineering. Nobody here is going to argue that Netflix, Google, or even your local dev shop don't do incredible work.
Now let's look at (2). Normal people have a super sad relationship with computers. They can't compose one program with another (webapps aren't very good at this). They can't run a command repeatedly in an automated fashion (hard to do with a mouse). Their data is smeared across a dozen different servers, one for each webapp they use. Companies regularly do tricky things with export formats to make it hard for them to get their data back. In general, non-technical people just have an awful time. It's them we could do better by.
Inventing good ontologies is a problem that's orders of magnitude harder than a problem of making a filesystem hierarchy searchable. Look at Wikipedia -- despite all this time and all this truly immense planet-size effort, they still haven't figured out a good categorization system!
Web pages being saved as blobs and not providing raw data is not a problem technologists can fix unfortunately.
But if web browsers are so bad, why do they keep spreading?
[...] mainly because the alternative is defective.
The startup, academic and open source communities all
have their weight behind Unix, but Unix is a dead end.
I wanted other people to share this experience, but couldn't make any converts. It turned out that it was only working for me because I was willing to plow hours into fixing problems. For most non-programmers, command line Unix isn't a good enough user environment to be a competitor to the web. And we really need that competitor.
I am also a programmer, know UNIX before Linux was a thing, as my introduction was with Xenix. But I was already versed in systems like the Amiga.
The CLI might be interesting, but a REPL environment is so much more powerful.
POSIX is stuck with UNIX System V. It hardly changed anything in terms of OS architecture and they don't seem to be adding anything new, in terms of modern systems.
I agree that a unix command line is not a good interface for non-programmers, but it is good for people who have to work with large quantities of textual information, and source code IS text.
I think that the main problem is that there is not much (commercial or academic) interest in empowering non-technical users. There are some specialty tools for non-programmers like excel macros, or task automation in photoshop, but general purpose system that let the average user program the computer without actually programming it never took of.
I still think it is a very big stretch to go from web
browsers (which suck on many levels) to unix bashing.
I just know from personal experience when I try to get people to switch away from webapps and they ask "what should I use instead?" I don't have a good answer.
I could say "use desktop GUI apps", but that's not really convincing. What would be convincing would be "use this awesome command line/REPL environment!", but I have trouble honestly suggesting that (even though I love my setup) because I know they won't have a good experience with current command line environments.
Fixing problems, or finding solutions for your problems? Every advanced system has a learning curve, and Unix is an advanced system without a well-defined domain. It's great that you invested the time to make it work for you, but that you feel that it is inappropriate for "most non-programmers" is not necessarily a failure of the system.
As a sibling post said, "there is not much interest in empowering non-technical users". I think that's close, but I would add that it's a problem of scale: non-technical users require domain-specific solutions. If there were one single solution to empower non-technical users, there would surely be commercial interest.
I'm sorry, but I can't help feeling that the article has a high dose of "lo! I've seen the light and my light will solve every problem on the planet". It ignores the many good things Unix has enabled, and is very light on the details how a better system would look (or how it would be secured -- befriend, really?).
(You could implement this different ways, but I'm imagining that once you friend someone they can see a select subset of posts on your computer. For me that would be status posts, some photos, some WIP code, etc. This would be a huge deal. Think about how many webapps it would make unnecessary.)
EDIT: Less flowery sentences.
For example, your code as shown implies that you can reach your friend's computer through an Internet name (through dns even?). That already assumes that these non-technical friends can register their own Internet names, and care enough to do so. But that assumption is well outside the bounds of "unix sucks".
Perhaps a charitable reading of it would be that instead of reading "unix sucks" (sadly I use almost these exact words in the article) instead read "the mismatch between the tools provided by unix and the needs of a normal user, combined with decisions made with good intentions at the time and kept for backwards compatibility, make the unix user environment a bad user environment for normal people today".
But of course I didn't write that, so I'm in no position to complain too much.
The classic analogy is you unleash genealogists on a village cemetery and have them produce a family tree and the output is actually useful information. Unleash the same team and ask them to digitally tag tombstones, and you just get a pile of useless, error filled data, packed with duplicates and weird capitalization issues. This is a classic "information vs data" misunderstanding.
Another problem with hierarchical vs tagged is we have somewhat obtuse yet usable set of tools to handle truly immense spammy hierarchical filesystems, but strategies for handling and scaling tagged filesystems usually are hand waved away with "it'll never happen" or "we'll sprinkle magic AI dust on it, then it'll evaporate away". A classic "make the simple easier, while making the more difficult impossible". No thanks.
Much as there's nothing really new under the sun in IT, a generation or two ago this was "sure sendmail configuration is complicated, but all it needs is a GUI because in the small subset of problem domains that are a match for GUIs and people with a problem simple enough to be handled in a GUI, it works well, therefore its perfect for everyone to do everything in"
Also data typing is just a subset of tagging. Or rephrased a type is a very poorly implemented limited single tag.
Ditto the above with a "decent shell language" again we're dealing with people who can't handle paper calendars without a book telling them how (GTD, 42 folders, etc). They're just going to mess things up quicker with automation, while people who can handle the cognitive challenge will likewise jump into org mode or crontab without any serious effort on their part... You "could" write a noob gui for a nuclear reactor, or a UI for even newborn babies to use matches, but you probably shouldn't.
As a meta observation this is a classic "future is already here just unevenly distributed" problem, I use databases and git all the time. However see the comment about the GUI for sendmail... Much like some people are unqualified to be handed an acetylene cutting torch or a beryllium-Pu neutron source, some people should not be handed relational DBs or KV stores or version control, and if they had the background to understand and safely use those tools, the existing tools aren't really all that hard to use for folks of adequate cognitive level.
It is stupid. The only way you can make someone an expert of a new tool in 15 seconds is by making the tool so simple that it's completely useless. There's a reason you get a few dozen hours of training before you are allowed to drive a car around. Or fly a plane. If you want to have a simplified car UI, then one exists too - it's called a taxi.
The reason of course is the competitive market pressure. The primary interest of a company is to sell the product, not for the product to be useful. And so most of current design effort is about making software easier to sell, not about making it actually useful.
It's absurd to expect people to figure out by themselves how to use their file systems, databases or - yes - even their paper calendars. All of those are tools that require some learning to use effectively. We need to make people read manuals again, and stop expecting everything to be usable out of the box by an untrained monkey.
If you've ever read a technical manual, you're probably in the 1% of the population. People don't even read two-lines popups, and you expect them to read complicated manuals?
What you really need is adaptive interfaces that grow in complexity as requirements expand, reliably detecting the newbie / power user / hacker progression. That's what works best. Hackers ridicule UIs that bury stuff under "Advanced" modes, but they're actually the best approach.
Doing three (or four, or five) times the work, though, is no fun for developers, so we go from "you're a dumb user, be happy with limited options" to "you're a clever user, surely you can read a 1000-pages bible before launching a command", because building multiple interfaces for the same routines is boring.
What I would call number 1 of user experience: The human is never at fault. Systems are there FOR people and systems should be designed to cope with the flaws of humans. If your system can't handle normal users then you are at fault and you're just pointing fingers.
> And likewise the problem with tagged data is likewise the people who can't organize tags
Extremely many attributes (which IMO is a similar but much better model than tags) can be set automatically. Date and location (and people in them by using face recognition) for photos, artist, track title for music files. It's almost comical that document names almost never match up with their filenames. File endings are another extremely silly concept.
> The classic analogy is you unleash genealogists on a village cemetery and have them produce a family tree and the output is actually useful information. Unleash the same team and ask them to digitally tag tombstones, and you just get a pile of useless, error filled data, packed with duplicates and weird capitalization issues.
The attribute scheme seems to work out fine for Wikidata. Filesystems today also have adopted attributes but are fairly limited in using them. On OS X Photos and iTunes organize files through databases that connect files with attributes. On Linux when using Gnome you have Zeitgeist that automatically indexes all your files and keeps attributes on them, which is used for instance by the Gnome Music application.
> You "could" write a noob gui for a nuclear reactor, or a UI for even newborn babies to use matches, but you probably shouldn't.
The complete opposite: You actually need to put very much consideration into the user interfaces of critical systems to prevent mishandling. Chernobyl was partly caused by a bad UI.
I beg to differ. Chernobyl was caused by flagrantly violating the simple principle of "if it ain't broke, don't fix it". It wasn't broke (well, not any more than any reactor of that type is broken, by design) and they tried to fix it.[1]
[1] https://en.wikipedia.org/wiki/Chernobyl_disaster#Steam_turbi...
Is not only sudo, but sudo AND polkit. Are you sure you know what are you talking about?
And about your sensitive data, AppArmor is more than capable. For the other sensitive data, create a folder out of /home and "chattr +i" your content recursively, giving proper permissions first.
And you rely in Plan9 for your speech, which is more UNIXy than UNIX and everything IS a filesystem in a more hardcore way (something you look as obsolete), altough factotum and namespaces are a complete different beast.
The question is if we can do better, and the answer is obviously yes because research OSes have existed for many years, like EROS, KeyKOS etc.
However rewriting all our software is not going to happen, so the question is how to get from where we are now to better systems.
The fewest unix systems out there have several users connecting to it with each user having a different unix user account. But exactly this is what the unix user system was designed - nothing else. Everything else like root or users for services is just a dirty hack, because unix wasn't designed to offer real isolation of applications.
AppArmor doesn't work because users creating profiles about what syscalls are allowed is out of touch with reality.
The only unix systems partially offering the isolation actually needed today are Android and iOS. You could still improve them massively though.
Security wouldn't be possible without this. You can also think of cgroups/lxc/docker as a direct extension of this.
I maybe didn't express myself clearly enough. The unix user system wasn't designed for the program isolation it is used for today, but for multi-user systems. The security model was to protect the system from users and users from each other.
Cgroups are certainly a move into the correct direction, because it's based on the assumption that programs are evil (either by design, or because they get exploited). When you correctly sandbox programs you won't need to run them as a different user anymore.
I think we don't really need them. We only need 3 operating "modes" in 90% of the cases: normal, administrator, guest. And after that various per-application permissions a la Android or iOS.
We don't need multi-users systems.
I think your mostly correct, but in most cases I would prefer individual authorization rules, like apparmor. There's always cases when you don't want to escalate the entire app to root level, but need something more than what's a normal mode.
As if with "modern" languages you would be safer. Poor new delusional kids.
That's a rather amazing quip. How would I, in my mid-thirties, accidentally write a buffer overflow in Haskell or Rust?
http://stackoverflow.com/questions/498234/vulnerability-in-t...
All software has bugs, it's true, and language runtimes included. But that doesn't mean some languages can't be safer.