This is a bad design. Process is not a file because you cannot send signals to a file, or cannot debug a file. Network socket is not a file because you cannot get file's peer address. Shared memory is not a file. And so on.
Debugging is hard to imagine, yes. One could look at /proc/12345/mem, write breakpoints in there, but I'm not sure about how to do the more exotic things.
npm install -g styled-linux
The serialized streams make it easier to think of these ioctl-alikes as something you can easily access over a network (9p).
And that’s how resource sharing is done in plan 9.
The question isn't "will it be just as bad" but "would some things become easier, and if so, how many things would become easier?".
Because if the latter, then it's a worthwhile topic to think about even if we never use it in any commercial sense of of the word.
It's far too easy to pretend reality is not complex and that "elegant" solution somehow will fit everything
The situation there is exactly the opposite of a syscall, it is rather that the kernel calls into userspace to perform a helper function.
Communication with the socket is still read()/write()/..., so there are still syscalls. The userspace program will do a read() to get the next struct+packet out of the socket.
The modern, syscall-less interface for stuff would be io_uring. There, you do not need to read(), you can just get your data written into a userspace buffer that you can mwait or poll on.
Outside of lots of non-obvious problems with synchronization, this is your example that best fits the idea. This interface is probably a good one.
> And of course you can get a peer address from /dev/tcp
You will have lots and lots of problems with access controls if this is your only interface.
> Shared memory can be a file
Coercing random access memory into a serial file just to go and emulate a random access over that file is... not a great way to deal with a high-performance primitive.
The file doesn't need to have an on-disk representation. I don't see why an mmap-ed file should behave any different from a SHM segment. They are basically the same thing, the SHM segment even has a file descriptor. It just doesn't have a name somewhere in the filesystem hierarchy. https://man7.org/linux/man-pages/man7/shm_overview.7.html
Because if you just add some high-level interface without any concern for performance, yeah, you get what Linux does today. But if you make them a core concern of your shared memory interface, you will certainly lose performance on the cases it's mapped as memory. And "everything is a file, but this one here is actually all about random access" doesn't give you much abstraction.
As somebody already said on the comments, the nice (maybe IMO, I'm not sure) thing about Plan9 is that every resource is named somewhere in a tree. The fact that those things are "files" only detracts from the value and makes the system less fit for modern usage.
But well, if the proposal is to unify everything, you will have to unstream network connections too.
This interface is a bad one because if you want to filter syscalls then it will be difficult to distinguish write to a file from sending a signal.
Try `strace|grep open` on any program, and you will be spammed with a number of shared libraries. You need to filter them out anyway.
In Plan 9, you send signals and messages to a file by writing to it. And read messages by reading from it.
It's in essence no different than OOP or actor model or what have you.
It’s really “Everything has a File Descriptor”, nothing about the concept of a file has changed.
It's in essence no different than OOP or actor model or what have you.
I’m sure there’s an isomorphism that could be drawn but it does nothing to show that it’s an equally good paradigm to write software in. IMO, it’s a tortured abstraction.
This is akin to how shell piping in Unix is also just... text and bytes. This is limiting and produces many ad-hoc protocols.
But take what Plan 9 was trying to do, and add to it what Microsoft's PowerShell tried to do, where you stream objects, structured information, instead of just bytes, on the way in and out of commands and files...
And suddenly... we got ourselves an Erlang.
Many perfect and awesome systems have been built that never saw significant adoption.
I agree with ESR's observation: "Plan 9 failed simply because it fell short of being a compelling enough improvement on Unix to displace its ancestor. Compared to Plan 9, Unix creaks and clanks and has obvious rust spots, but it gets the job done well enough to hold its position. There is a lesson here for ambitious system architects: the most dangerous enemy of a better solution is an existing codebase that is just good enough." Source: https://www.catb.org/~esr/writings/taoup/html/plan9.html
Another amazing OS that never caught on was BeOS: https://en.wikipedia.org/wiki/BeOS Neal Stephenson's description of BeOS as "fully operational Batmobiles" is accurate and it is sad that BeOS did not become more mainstream. See https://people.cs.georgetown.edu/~clay/classes/spring2010/os...
interface UniversalInterface {
fun open(…)
fun read(…)
fun write(…)
fun close(…)
}
Ultimately, when someone does end up implementing something like the above as the sole interface for some major component, it’ll result in libraries that return a wrapping object with a more useable and pleasing interface that abstracts away the ugliness of using UniversalInterface to interact with said component.
I see two ways forward with this. Either: (a) present the option of using UniversalInterface to interact with X object, in addition to interface that’s much more idiomatic and closer to how X object actually behaves.
Or, (b) come up with an alternative universal (or flexible) object interaction interface that’s much more flexible than UniversalInterface.
> Or, (b) come up with an alternative universal (or flexible) object interaction interface that’s much more flexible than UniversalInterface.
That’s, basically, what COM/Corba/… are. There, UniversalInterface has an additional call “If you are a Foo, give me your Foo interface”, with Foo as an argument to that call.
That's really good. The fact that the file interface just gives you a string of bytes, with no concept of structure or type safety in the interface itself is a major flow of it. Type safety is immensely valuable.
You're describing a meeting in which people seem unaware of the purpose of uniform interfaces. Not sure we could call such a meeting "engineering design" meeting, because engineers tend to know better.
If I went up and proposed the same interface for all cables, displays, mice, keyboards, speakers, hard drives, phones... and called it USB, would I also be laughed out of the room?
There's no benefit to reinventing open/close/read/write in 50 different ways. The goal is to do it once, and then build more complex interfaces on top of it. That is, unless you're paid by the number of lines of code you write.
There is an universal interface, it is called "system call". Rather than build unnecessary layer on top of it, improve the syscalls if you don't like them, and get rid of ioctls, /proc, /sys and other pseudo-syscall abstractions.
Hence open is not part of your UniversalInterface, but it is a way to obtain a reference to it. It seems reasonable to have a generic way to dispose of an UniversalInterface (hence close). read/write are simply a generic ways to send and receive messages from UniversalInterface, not unlike a dynamically typed object. Ideally you would do a checked down cast to your actual interface [3], but this was designed to work with C so you have to make do.
[1] I don't subscribe to the Everything is an Object in the OO sense, but having a common base class for most OS resources seem a reasonable solution.
[2] or at the very least there isn't always a cross-resource-type namespace.
[3] Not unlike COM QueryInterface, and in fact UniversalInterface is equivalent to IUnknown
This is very obviously an acceptable solution because the whole world runs on TCP.
The metadata around files - ownership, permissions, creation time - apply to a lot more than files.
I don't really see it being laughed out of the room.
Doubtful. Why are the main paradigms?
Everything is an object
Everything is a function
Everything is a resource (REST)
Uniform interfaces are common for good reason: you gain a lot of flexibility for redirection, introspection and policy control.
Yes? All interfaces do that, especially system interfaces. It's the whole purpose of interface - to provide implementation logic in a usable form. Try to call the kernel by hand and see the difference.
1. ioctls can make any "syscall" on a file.
2. a process does not have to be a singular file. All processes have most if not all their attributes exposed as files /proc/$PID/ as files, and can have this arbitrarily extended. In plan9, passing a signal (technically a note) is done by writing to /proc/$PID/note, and there is no technical reason for not allowing the same on Linux.
3. the entire concept of memory mapping is based around files, with anonymous memory - i.e., non-disk-backed memory - just being a subset of this. POSIX shared memory (shm_open) is provided through /dev/shm, which is a tmpfs folder and is indeed just files.
4. sockets are file descriptors, and file descriptors is what makes a file, and as such you can get a peers address of a file descriptor when such is present. Ways to expose creating sockets in the filesystem also exist, and not just for plan. The special socket-bits could easily be made less special, with the only justification for the current BSD socket API being that it became dominant and so everyone copied it.
A file descriptor is exactly nothing like a file. You cannot write to a pidfd. You cannot waitid an eventfd. You cannot getsockopt on a regular file. The only operations that all file descriptors have in common is close, dup, poll and some other basic operations.
So "file descriptor" basically just means "kernel interface object" and the available operations depend on the type of object.
A file is a container for arbitrary data that I can read from, write to, and reposition the read/write cursor in. If you call anything else a "file" then you haven't made everything a file, you've just redefined "file" to mean "thing."
What business does a socket have in a physical, on-disk filesystem? (Let alone a clunky hack to invoke the much simpler `signal` syscall in a roundabout way?) The socket "file" is completely meaningless unless the process that opened it is currently alive and still listening on it. So why the fuck should it get written to a persistent storage device?
How do I specify the socket type, which is a meaningless concept for an actual file, when I open a socket "file"? Oh that's right, I don't. Because I don't open a socket. I bind or connect. I don't use "file" APIs because they're not applicable. I use a dedicated socket API that's fit for the purpose.
Pidfd was not introduced to treat processes as "files," it was introduced so they could share the operations that they do meaningfully share with other kernel objects (e.g. poll).
The overloaded ioctl syscall is bad design. The proc "filesystem" is bad design. /dev/shm is ridiculous design. So I have to mock a fake filesystem in memory so I can create a fake file in that "filesystem" just so I can get the same memory pages mapped into my virtual address space as some other process, all of which has absolutely nothing to do with files or a filesystem (and is much lower level than that). lolwat?
If not being able to write makes it not a file, then files stop existing when a disk is full, and means that /dev/zero and /dev/null are not files - despite being at the heart of the whole "everything is a file" paradigm.
Pidfd was not made to make processes behave like files - that is what /proc is - but to solve problems with process related syscalls and PIDs, which are flawed and racey. The solution to that was to make APIs that treat processes as files, which gives you the ability to poll it like a file.
A streaming socket is exactly like a normal file. You read, write and poll. The only thing that is special is how to create it, but that is a design decision, not a technical limitation - see the plan9 file based API for making TCP sockets, which is trivially implementable in Linux.
Domain sockets are a bit different because of their side channel and would require more ctl files, but Linux's API is 99% magic files and ioctls so this is not that weird.
ioctls are not themselves bad design. In fact, scoping kernel functionality onto file handles is a great design and why that's almost the entirety of the kernel (device driver calls dwarf syscalls). The problem is not the design itself, but the fact that ioctl was not originally meant for it and got overloaded through several design iterations. This is what happens when you do organic design through more than 3 decades.
If you start out by defining a way to do file-scoped syscalls - and file does and always will mean "an fd" to a kernel - then you wouldn't have that awkwardness. That is what plan9 did: Take the learnings, and implement them clean instead of on legacy.
Which is what I said. It's a "file" in name only.
> If not being able to write makes it not a file, then files stop existing when a disk is full, and means that /dev/zero and /dev/null are not files - despite being at the heart of the whole "everything is a file" paradigm.
Great example that showcases the idiocy of Everything Is A File. /dev/null and /dev/zero are basic parts of the Unix API, so the basic OS API is broken-by-default at boot until one mounts a file system that had these dummy "device" nodes at a specific path that's hardcoded everywhere.
Instead of providing a sensible API like memfd_create or timerfd_create, for example.
> A streaming socket is exactly like a normal file. You read, write and poll.
That doesn't make it a file, that makes it an object that shares common traits with file objects. Datagram sockets do not read/write because they're not bound to a fixed remote address. And that's perfectly fine. They're not files.
> The only thing that is special is how to create it, but that is a design decision, not a technical limitation
And it's a good design decision. The socket API is pretty decent except for the dumb "file" nodes it creates when listening on standard unix sockets.
> see the plan9 file based API for making TCP sockets, which is trivially implementable in Linux.
But thank God it's not implemented in Linux.
> ioctls are not themselves bad design. In fact, scoping kernel functionality onto file handles is a great design
Agreed, the only bad part is that too much was shoehorned into the same syscall. It's certainly better than magic "files" that pretend to be "files" by having you read and write structs from, but you're only allowed to read and write whole structs per syscall, which has no resemblance whatsoever to how reading from and writing to a file work. (Maybe Plan9 doesn't have this limitation and tries harder to keep up the charade, I wouldn't know.)
No, it is the very definition of a file from the OS perspective. There is no other applicable definition to the OS. You seem to conflate files with disk storage, in which case not just plan9, not just Linux but the entirety of UNIX history seems to have flown past you.
That files are a nothing but abstract handles that implements the VFS interface to serve every conceivable function - where regular file is treated no differently than a device driver - is the entire point of modern UNIX. If this is the part you are stuck on it is not a surprise that both the existing Linux kernel APIs and trivial (and quite frankly, perfectly ergonomic and efficient) alternatives like the plan9 API seem so foreign to you.
"read and write structs" is the most normal thing for an application to do, whether you are reading JSON from disk, communicating over UNIX domain sockets with raw C structs, or sending protobuf over the network.
"But thank God it's not implemented in Linux" - Linux has many of these APIs already and it constantly grows, see for example all of /dev, /sys and /proc, not to mention FUSE and support for the 9P protocol to use all of plan9's services as-is.
You don't get one object == one file node, you get one object == a subdirectory with lots of file nodes that you have to open and close independently. That should tell you that your pattern doesn't work. (With enough effort, you can of course always shoehorn everything into an ill-conceived concept (and you did), but if you have to bend over backwards to make it fit, you should just admit it doesn't fit.)
It works for /dev, but definitely not for the mess that is /sys and /proc.
"Reading and writing structs" is the most natural thing to do when you're forced to serialize your data. You then design wrappers around that serialization that expose a sane, type-safe API on each end.
Wouldn't the same be applicable to files, processes, devices and so on?
And like it or not OOP shows that it's possible for one idiom to describe all the things when it's flexible enough.
I've got a lot of experience working in systems that model everything as a series of resources passing messages and it works very well. The entire QNX operating system right down to its POSIX support uses this underlying primitive and, while you would never see it in your own code their system-wide profiler leans on this design to make it easy to see how control flow moves between isolated threads and processes within the software system.
Both "special" and "device" files are still "files".
https://man7.org/linux/man-pages/man2/pidfd_send_signal.2.ht...
Also sockets have the concept of ancillary channels used to deliver out of band data (see recvmsg). If every system resource had such channel, it could be used for control similar to how sockets do it.