Projected File System | Hacker News Reader

Projected File System

scorpiosoftware.net

70 pointsthunderbong24 comments

Reminds me of "fuse" - userspace filesystems for Linux

https://www.kernel.org/doc/html/latest/filesystems/fuse.html

fuse is surprisingly easy to use once you get familiar with it: there is a number of callbacks to file operations you need to provide, and that's it! Very similar to making an HTTP API.

This isn't FUSE. It's a thin middle layer to hydrate content into the real filesystem.

This would let you implement the UNIX "everything is a file" philosophy on windows.

Although I feel guilty because that's probably more the Plan 9 philosophy. On Linux you get a lot of nice things by having /procfs for example but lots of things aren't files that might be. I've often wondered if a file based interface to dbus wouldn't be very liberating from the scripting point of view for example.

Plan 9 even does sockets via a file based interface and to my eyes it's no more clunky than BSD sockets - but presto, any bash script would be able to open a socket without netcat.

Indeed, bash already has socket access via /dev/tcp/HOST/PORT and /dev/udp/HOST/PORT, which allow opening a socket and reading from/writing to it as if it were an ordinary file.

Though this is implemented in bash itself and only works in bash, not in other programs.

The number of unix environments w/o bash available is surprisingly small thou

It means you can't just call open('/dev/tcp/blah') from a program and get a TCP socket though, unfortunately

IIRC that only works for clients and can't be used for listening.

Also IIRC there is a bash loadable module that implements server sockets but that can't be relied upon if you're just shipping bash scripts meant to be run without the end user preparing the ground before running it.

I mean, file handlers exist in Linux. That would be so cool if you could add a file/directory watch to an http endpoint/resource

I don't think trying to shove "everything is a file" everywhere is going to work well but let's see what the future holds.

IMO we actually should gradually start moving away from that. It's a leaky abstraction (though granted, our current tooling is pretty strong and excellent about all this so I guess it's a 50/50 here).

Yeah, the APIs aren't great on either Windows or Unix. I'm reminded of a @fasterthanlime tweet:

>Few people know the complete quote: >"Everything is a file — the worst possible primitive"

Especially in a multi-core world, you need to know a lot of Unix(/Linux) arcana to write a program that ensures ACID-like properties for files. (And this need applies to almost any program that writes to a file.) If filesystem APIs were redesigned tomorrow, they would be quite different from what we have today.

Had no clue of the original quote, hah. Thanks for letting me know! :)

And I especially agree with the multi-core comment. We need filesystems as full-blown ACID databases, like 10 years ago now.

Surely it can't be so difficult to have a K/V store with ACID semantics and give it a filesystem emulation layer so people can use all the normal coreutils like `ls` and `cd` etc. But then also add a little extra API that allows for locking, transactions and all the good stuff?

Will nobody step up?...

This looks absolutely painful next to FUSE.

Do you have to enable the feature for this to be used on user machines? Is it persistent across restarts?

It's really annoying that seemingly simple things that have been around forever (this vs FUSE, NFS, freaking symbolic links) are gated behind "pro" features or advanced user workflows. It's very difficult to rely on them as an application developer shipping products to less determined/advanced users.

I would love to see this functionality exposed more generally. One thing I thought was cool in ReactOS was the ability to explore the registry as part of the file explorer. Powershell also exposes similar functionality (like the HKLM: drive) but being able to access that through common tools like Explorer would make the Windows internals feel much more approachable.

It's possible on Windows too using Windows Registry shell extension[0], which uses Windows Shell Namespaces under the hood.

[0] http://www.viksoe.dk/code/regfolder.htm

[1] https://en.wikipedia.org/wiki/Windows_Shell_namespace

This is the new pattern emerging... Linux is the trendsetter and Windows is following. Windows is becoming the underdog.

Innovation has never been part of Microsoft's strategy, but believe whatever makes you happy.

Well at least it seems easier than the IFS approach to exposing a foreign filesystem. I suppose if someone wrote a FUSE api wrapper around this it could be even more useful.

It's not a full-blown userspace filesystem like fuse or dokany (which already has a fuse wrapper). The projected file system still needs a local folder where it can place copies of files from the "backing store". E.g note the Remarks section at https://learn.microsoft.com/en-us/windows/win32/api/projecte...

> When ProjFS receives the data it will write it to the file to convert it into a hydrated placeholder.

There's some discussion here: https://github.com/microsoft/ProjFS-Managed-API/issues/54

It looks like it's meant to solve a much narrower problem than FUSE.

Edit:

Since you mentioned Dokany[0] I'll also throw out a shout for WinFsp[1] as a similar project, too.

[0] https://github.com/dokan-dev/dokany

[1] https://winfsp.dev/

oh, I had no idea from scanning the docs that it created a physical file for every virtual file. that really makes it of marginal general use.

I suppose the choice of the name also makes sense in that context.

I really wish they could have just called this WinFUSE

Reply on news.ycombinator.com