The Parrot Virtual File System | Hacker News Reader

The Parrot Virtual File System

ccl.cse.nd.edu

79 pointstempay20 comments

The idea of extending file-related system calls to remote pathnames was present in a few systems in the 1980s: ITS, Lisp Machines, VMS, probably a bunch of others I am not aware of. You can see the vestiges of this in Common Lisp pathnames. Kent Pitman provides a good summary in a comp.lang.lisp post from 2002: https://groups.google.com/d/msg/comp.lang.lisp/UCeOl3sUlFQ/V...

This approach is much more powerful than the Unix style of mounting. FUSE provides some degree of extensibility, but it is not composable. Pitman's post mentions the "chained hosts" of VMS. GNU Emacs TRAMP has a completely general and composable mechanism called "multi-hops": https://www.gnu.org/software/tramp/#Ad_002dhoc-multi_002dhop...

Using explicit remote pathnames as opposed to mounting is a very powerful idea, and I am glad it is becoming practical today. I think it is possible to write something like a portable Parrot as a library and set of shadow include files you could use to recompile applications.

> This approach is much more powerful than the Unix style of mounting.

It occurred to me I should provide a qualifier here. Single-tree Unix pathnames have one very powerful property that qualified schemes from the 1980s lack: self-similarity at every level of the directory hierarchy. This is why filesystem virtualization schemes like loop devices (CD and disk images, and things like EncFS), chroot, and overlays are possible and work so well. The 1980s schemes did not support these use cases and were more tedious to type out, and they rightly died out. But it is not either-or: TRAMP shows that you can have Unix pathnames for your local machine and qualified pathnames for when you need to specify where/how to access files.

Just curious, in what way is FUSE not composable where TRAMP is?

For example, if you want to use SSHFS to access files on a machine that is only reachable via a gateway, you would have to change your SSH configuration with the details of the gateway, or set up port forwarding: https://undeadly.org/cgi?action=article&sid=20070925181947

TRAMP provides a pathname syntax for this kind of connection:

    /ssh:bird@bastion|ssh:you@remotehost:/path

So the steps to get to the remote files are part of the pathname, and you can compose several steps with |. With FUSE, or any kind of mounting, this information is not part of the pathname; next time you need to access somewhere convoluted you have to set up the proxying/forwarding again.

You can also share the pathname. It is easy to copy-paste an SHH to sudo to Docker container multi-hop; setting up and tearing down the mounts to do the same thing would be fairly involved.

This is an interesting method of providing a user space VFS later without the need of a kernel driver, like FUSE.

However, that said, I wonder if it would still work in cases when a file is memory-mapped (via mmap() and friends) and modified directly in memory rather than direct FI/O calls. Would parrot still work in this case?

If you'd like to see code for a minimal version of something similar, I wrote one that's loaded using LD_PRELOAD rather than ptrace. Toy quality: https://github.com/viraptor/libremotec

I wonder how feasible this would be to trick Plex into transparently 'mounting' a remote server's media files. (e.g. a VPS that can burst CPU to re-encode data once/twice a day and another one with barely any CPU but large storage)

I guess things like sshfs or rclone already do similar things, but it would be really neat to see the performance characteristics of Parrot vs other solutions.

I can't resolve the domain. I've seen websites fall down due to load but I've never heard of a domain disappearing due to load.

Edit: And now it's working.

Works on my machine.

on windows: ipconfig /flushdns

Sadly, that doesn't help on the bullshit Chrome is apparently pulling with name resolution internally.

Would this be able to make a device acting as a USB mass storage device work as a network storage backed USB Drive?

I don't see any one command from this guide that the utility could be run before to enable this. https://www.raspberrypi.org/magpi/pi-zero-w-smart-usb-flash-...

no, you would still need something like a file server serving read/write requests to the USB drive.

Think of this program like automatically giving any program that has a "File -> Open" menu the ability to load files from FTP, or HTTP, or any other file serving protocol. You still need something serving up the data - it's just possible for you to treat it like any old file on your file system now.

This is really neat. I was looking at FUSE for something last year and it never occurred to me to come from the other direction, so to speak.

I was interested to read about CVMFS - is this something people have experience with outside of CERN?

Reminded me of the Parrot language prank:

http://archive.oreilly.com/pub/a/oreilly/news/parrotstory_04...

https://slashdot.org/story/17381

Is there any supper for running Fuse file systems using Parrot so that fuse can run completely from userland without any exotic privs needed (e.g. if you wanted to mount a squashfs file system for a process from within a docker container running inside Kubernetes).

wouldn't using Dtrace be a lot more efficient than using ptrace?

What they're doing is trapping the execution on syscall and providing own implementation. That's not the kind of thing dtrace is good for. Also, this works on Linux, so there's no full dtrace.

Cool

Reply on news.ycombinator.com