Comment by MyrtleTook | Hacker News Reader

Comment by MyrtleTook | Hacker News Reader

Parent Full thread

I don't understand how the "everything is a file" could apply for everything in practice. For example, Linux has ioctl(), which in practice are like a side channel. Granted, Linux doesn't apply this API philosophy too thoroughly.

I guess the "everything is a file" might have multiple meanings. For example:

(1) Everything is represented by a (file) descriptor

(2) Same as (1) and the descriptor has a file-like API (think read(), seek(), write(), etc.)

(3) Everything is an "byte addressable blob of bytes"

Meaning (1) is OK. But it doesn't tell nothing about the API the (file) descriptor itself would use. It could be a fixed set (like meaning (2)), or be variable depending on something else (like the (file) descriptor type).

Meaning (2) looks like too restrictive and inefficient to me and is the one I really have trouble accepting as a general OS primitive.

Meaning (3) surely can't be used for everything in practice, right? It's just to generic like "every computer architecture can be emulated by a universal turing machine." And it also seems too inefficient. But it could be very useful if the blob of bytes had an API like (2) or any other, including having an API depending of the "file type".

Is option (3) that folks are meaning when talking about "everything is/should be a file"?

EDIT: formatted the meaning list correctly.

It's more like #2, but without ioctl() and similar brain-damaged abortions. If you open /dev/mouse, for example, you get a stream of events (encoded as blobs of bytes), which you can get by read(), not a byte-addressable blob of bytes.

But lots of things in Plan9 present an interface that isn't just a single file. The 8½ window system, for example, presents not only /dev/mouse but also /dev/cons (character-oriented I/O), /dev/bitblt (to which you write requests for 2-D accelerated drawing operations), /dev/rcons (keystroke-oriented character I/O), and /dev/screen (the contents of the window—which is just a byte-addressable blob of bytes). http://doc.cat-v.org/plan_9/4th_edition/papers/812/ explains in more detail.

And, of course, file storage servers similarly provide an arbitrary tree of files, and when you remotely log into a CPU server, you mount your local filesystem on the CPU server so that processes running on the CPU server can access your local files transparently—including /dev/screen, /dev/mouse, and /dev/bitblt, if you so choose.

That's actually really cool.. This democratization of useful information probably opens the door for lots of interesting interactions between distributed systems.

Even on my local Linux system I wouldn't know how to get hold of the mouse data without using an X Windows API (or SDL on a console only app before X is run).

I agree! I think that kind of transparency and openness to hacking is very important.

You may be interested in https://gitlab.com/kragen/bubbleos/blob/master/spikes/intell..., which reads from /dev/input/mice, although I haven't yet implemented support in https://gitlab.com/kragen/bubbleos/blob/master/yeso/yeso-fb.....

There are devices under /dev/input which are quite easy to decode for raw mouse data.

And what's the advantage vs an API via function calls? It's the same thing no? Calling 0x42 with a given ABI vs interpreting bytes as an ABI seems oddly similar.

I'm not overly familiar with Plan 9, but a significant advantage of the filesystem as an API is that it composes well. You can put any service inbetween that consumes the existing file and provides a new file with the same API contract, but different behavior (e.g. a /dev/screen that's cropped to a certain area, or that inverts colors on everything drawn into it). Then you can make a new mount namespace where the new file is put in the place of the old file, so that the actual process consuming it has no idea that someone is sitting in the middle. With a traditional shared object, you'd have to engage in LD_PRELOAD shenanigans to achieve the same, or if the library is statically linked and does not have any hooks, you're straight out of luck.

In other words, it gives you a system wide way to override any APIs. As long as you are compatible with the ABI of course.

At the cost of probably making all APIs a bit less efficient. Now I wonder if any Plan 9 derivative offers a 3D rendering API?

No, and even for 2D it lacks acceleration.

Not true:

http://www.cs.cmu.edu/~412/lectures/2009-10-23_radeon.pdf

>>Provide 2D acceleration via the 3D engine.

I stand corrected, however 2009 is already past the time I was paying attention to Plan 9.

As explained in the abstract of the paper I linked above, 8½, the Plan 9 window system, has supported 2-D acceleration since at least 01992. I also mentioned this in my comment, which additionally outlines how this acceleration support works across networks.

[From memory, so take with a grain of salt]

a) tooling and familiarity

b) it's leveraged for modularity, composition and access control. Normal unix basically has no modularity and little composition beyond piping to stdin (which is of limited applicability). And of course no useful access control mechanism to speak of. The last 5 decades mostly just added layers of shitty hacks that don't work and that no one understands anymore (by contrast e.g. the original unix ownership and permissions model was misdesigned but at least possible to grasp).

Plan9 does substantially better in both regards via clever use of a proper union file system (which incidentally makes another horrible hack superfluous: symlinks). If you want your shell, or some other executable, to find some executable, you mount it into its /bin -- there is no $PATH. If you want some process or user to have access to a resource (file, device, server in the network, ...) you mount it into their filesystem. This gets rid of a lot of problems and special purpose solutions.

Isn't the same possible now with mount spaces in Linux?

You mean union mounts? Typically doing a union mount is a privileged operation, while setting $PATH is not, so you cannot really replace the latter with the former. A bigger issue is that, even if mounting various things on /bin were unprivileged, removing $PATH from Linux would break an enormous amount of existing software. Similarly with file permissions. So union mounts don't permit you to simplify Linux in that way, even though they did permit you to simplify Plan9. (Incidentally, Plan9 did have the traditional user/group mode bits.)

An advantage is, you don't need any API or function calls. You can inspect or manipulate anything from the shell ad lib using cat, echo, and the usual commands. Plan 9 doesn't have ioctl() calls because it's not meaningful to send pointers across the network. Instead each device and service provides a file called cntrl (or something similar) to which you can read or write text - using cat, echo etc. So it's easy to write small test scripts or even whole applications, in the shell or any language you like.

Functionally it's the same. The functions called could live in the kernel (a system call), in the process itself or in a shared lib loaded into the process address space.

Interpreting bytes is message passing and this communication happens through the handle/descriptor to the file/object/whatever. As above, this handle could have come from the kernel, another process or the executing process itself (the process opened some file or whatever).

I think the message passing version happening through a handle can be more versatile as you could send this handle over to another process. Could copy it and distribute it to other processes in a easier way than injecting/changing a (presumably C ABI) into a process address space.

Well, as I said, one advantage is that processes running on the CPU server can access your local files transparently—including /dev/screen, /dev/mouse, and /dev/bitblt, if you so choose.

That means you can run graphical applications on the CPU server.

I think "everything is a file" also means: everything is addressable by a file path. Per-process namespaces are part of what makes this possible.

In a way it's similar to HTTP REST, which is also organized by file paths, except instead of the HTTP verbs GET, POST etc. you get open, read, write as your verbs.

A side channel is then just a directory entry.

It may be useful to see Russ Cox's "Tour of ACME" video [1]. ACME is a Plan-9 text editor, which applies the "everything is a file" philosophy pretty deeply. It doesn't answer your ioctl question (that I remember), but maybe it'll give you a better example of how other things can be accessed as files.

[1] https://www.youtube.com/watch?v=dP1xVpMPn8M

It's more or less 3. Not everything is a blob but everything is a stream of bytes. It think the confusion for most of us (me) initially is that a file means a blob that you read in, change and then write out more or less atomically. But the Unix originaters understood files as streams. So, for example, the input stream from your mouse is a file. Everything is a file really means everything is a stream.

It's basically your second option. In Plan 9, everything is a file server that presents a file system interface that speaks the 9P protocol.

I agree with you. A type amounts to the sum of operations that are valid on an object conforming to that type.

A file object is a very basic, general type, that allows open, read bytes, write bytes, close, maybe seek, maybe some ops are restricted (read-only, write-only) etc.

I don’t think it is generally appreciated how far it gets you to have a unifying simple interface. You can always add a complex one, you know?

It's interesting both of you had different answers. I realize that the streams interface was a Sys V thing but would the Unix's "everything is a file" generally be option (2) in the OPs comment then? I feel like I've heard the phrase so much and just always assumed it was (2.)

If Unix is "everything is a file", then Plan 9 is "everything is a network filesystem".

If you have a laptop and a desktop, and your desktop has a printer attached, can your laptop just print to it? In Linux, you have to set up CUPS, open network ports, download drivers, and generally set up both machines to be able to "talk printers". In Plan 9, your laptop just opens the desktop's printer file over the network, and prints.

Plan 9 provides a generic framework for writing server-client programs based on the 9p message-oriented protocol; nothing more nothing less! In Linux, there is no such widely generic equivalent. Every server-client application does that in its own way. Actually Plan 9 forces everybody providing a service to understand 9p and to be able to translate 9p message requests to whatever it means for that service (what does write(bytes) means for a printer? What does write(bytes) means for a screen? what does write(bytes) means for a GPU? Everyone of those should understand a 9p write() message and translate it to whatever service it means) then Plan 9 uses that power to forward requests over the network to remote servers, making easy to use resources in a distributed environment.

Google's Fuchsia is remarkably similar to this.

There is per process namespaces/"filesystems". Their 9P protocol is FIDL and the main system API is defined by FIDL protocols/APIs.

Linux had CORBA, DCOP and now DBUS, among other UNIX variants.

The latter does not prevent the laptop from the need of a "printer driver" to actually render something. But the driver can be unified (TTY or PostScript), and the printer object (not exactly like a file) has, well, methods to report options and set options, like the paper tray to use.

Or so I understand.

If we could imagine a network filesystem (spanning many hosts) full of Erlang objects which can receive and send data, it would be somehow similar.

I think, in Plan9 philosophy, if the desktop has a driver and it is started in the current namespace, it is represented as a file too (like /dev/my-printer/print-pdf). But it quickly becomes confusing for me:

1. Nowadays we can represent the essence of this approach with FUSE and SSH/SSHFS. Sadly, nobody does: local servers (i3, dbus, ..., vscode) use domain sockets and client executables, probably due to the lack of private namespace support by default.

2. The difference between hypothetical "/dev/my-printer/print-pdf" and almost-like-real-world "my-printer-print-pdf /dev/my-printer" looks similar to binding the first argument in OOP-style vs the explicit C-like call syntax.

>1. Nowadays we can represent the essence of this approach with FUSE and SSH/SSHFS.

These suck hard against 9p and factotum. Not even close, Linux and BSD's are jokes against what you can achieve with plan9/9front on networking whole componentes. You can run remote processes seamlessly.

Who are both of us?

The important point is that "everything is a file" is just a short way to say "everything is a file system". Your interface is not a file descriptor to which you read and write, but a whole tree where there are different files on which you can perform the operations defined in the 9P protocol (create/read/write/...). For example, in the windowing system, you open a file to create a new window, and the new window will have associated a directory with files that represent the screen, mouse and keyboard (and the process running in that window will work with those files exactly as it does with native devices).

You can have a look at the man pages (sections 3 and 4) to see how these filesystems work.

"Everything is a file but with different semantics"

Reply on news.ycombinator.com