FUSE is All You Need – Giving agents access to anything via filesystems
jakobemmerling.de
jakobemmerling.de
Implementing a database abstraction as a file system for an LLM feels like an extra layer of indirection for indirection's sake: just have the LLM write some views/queries/stored procs and give it sane access permissions.
LLMs are smart enough to use databases, email, etc without needing a FUSE layer to do so, and permissions/views/etc will keep it from doing or seeing stuff it shouldn't. You'll be keeping access and permissions where they belong, and not in a FUSE layer, and you won't have to maintain a weird abstraction that's annoying/hampered with licensing issues if you want to deploy it cross platform.
Also, your simplified FUSE abstraction will not map accurately to the state of the world unless you're really comprehensive with your implementation, and at that point, you might as well be interacting directly in order to handle that state accurately.
I think there is a gap between “real file systems” and “non file things in a database” where mapping your application representation of things to a filesystem is useful. Basically all those platforms that let users upload files for different purposes and work with them (ex Google Drive, notion, etc). In those cases representing files to an agent via a filesystem is the more intuitive and powerful interface compared to some home grown tools that the model never saw during training.
https://github.com/nalgeon/sqlean/blob/main/docs/fileio.md
fileio_read - Read file contents as a blob.
fileio_scan - Read a file line by line.
fileio_write - Write a blob to a file.
fileio_append - Append a string to a file.
fileio_mkdir - Create a directory.
fileio_symlink - Create a symlink.
fileio_ls - List files in a directory.
Edit: thinking more about it I think its for actually making sqlite interact with the filesystem and not the idea of it acting as a file system itself without too much overhead, I was thinking something like the sqlite database itself stores data and then we could do these operations fileio etc. but this isnt possible from what I could gather.
perhaps this might be more interesting https://github.com/narumatt/sqlitefs but what I mean is if something like the merge of fileio + sqlitefs where things dont have to go through fuse in general if that makes sense hopefully
Maybe I went a little tangential but sqlite is really awesome
The file system as an abstraction is actually not that good at all beyond the basic use-cases. Imagine you need to find an email. If you grep (via fuse) you will end up opening lots of files which will result in fetches to some API and it will be slow. You can optimise this and caching works after first fetch but the method is slow. The alternative is to leverage the existing API which will be million times faster. Now you could also create some kind of special file via fuse that acts like a search but it is weird and I don't think the models will do well with something so obscure.
We went as much as implementing this idea in rust to really test it out and ultimately it was ditched because, well it sucks.
This is a limitation of the POSIX filesystem interface. If there were a grep() system call, it could delegate searches to the filesystem, which could use full text indices, run them on a remote server, etc
On mainframe and minicomputer operating systems they were often far less clearly separated, with a lot of functions which POSIX/Microsoft would view as belonging in a database built into the filesystem – such as record-oriented files and key-indexed files.
That's also true of MUMPS and PICK, which while nowadays language/databases which run on top of a commodity operating system, back in the day were operating systems in their own right.
Maybe some of these old ideas were just bad – or maybe we should reconsider them every now and again?
Unrelated to FUSE and MCP[1] agents, this scenario reminded me of using nmh[0] as an email client. One of the biggest reasons why nmh[0] is appealing is to script email handling, such as being able to use awk/find/grep/sed and friends.
To learn FUSE, however, I started just making everything into filesystems that I could mount. I wrote a FUSE driver for Cassandra, I wrote a FUSE driver for CouchDB, I wrote a FUSE driver for a thing that just wrote JSON files with Base64 encoding.
None of these performed very well and I'm sort of embarrassed at how terrible the code is hence why I haven't published them (and they were also just learning projects), but I did find FUSE to be extremely fun and easy to write against. I encourage everyone to play with it.
FUSE makes me think that the Plan 9 people were on to something. Filesystems actually can be a really nice abstraction; sort of surreal that I could make an application so accessible that I could seriously have it directly linked with Vim or something.
I feel like building a FUSE driver would be a pretty interesting way to provide a "library" for a service I write. I have no idea how I'd pitch this to a boss to pay me to do it, but pretending that I could, I could see it being pretty interesting to do a message broker or something that worked entirely by "writing a file to a folder". That way you could easily use that broker from basically anything that has file IO, even something like bash.
I always have a dozen projects going on concurrently, so maybe I should add that one to the queue.
I built the original version in Python for a job years ago. But the version above is almost entirely vibe-coded in Rust in a lazy afternoon for fun.
However, I disagree that the filesystem is the right abstraction in general. It works for git, because git is essentially structured like a filesystem already.
More generally, filesystems are roughly equivalent to hierarchical databases, or at most graph databases. And while you can make that work, many collections of data are actually better organised and accessed by other means. See https://www.seas.upenn.edu/~zives/03f/cis550/codd.pdf for an particularly interesting and useful model that has found widespread application and success.
Also, looks like my message queue idea has already been done: https://github.com/pehrs/kafkafs
No new ideas under the sun I suppose.
[0] https://github.com/Barre/ZeroFS
[1] https://github.com/Barre/ZeroFS?tab=readme-ov-file#why-nfs-a...
Maybe the most mainstream incarnation is its use in the Windows Subsystem for Linux (WSL).
Agents are uses on a Unix-based computer that is capable of and indeed was designed for multi-user collaboration.
Why not go for the simple solution?
Basically an abstraction that is filesystem-like, but doesn't require a filesystem. Though you can both export storage-combinators as filesystem and, of course, also access filesystems via storage-combinators.
[1] https://dl.acm.org/doi/10.1145/3359591.3359729
[2] https://2019.splashcon.org/details/splash-2019-Onward-papers...
It opens up absolutely bonkers capabilities.
I've done exactly that with Filestash [1] using its virtual filesystem plugin [2], which exposes arbitrary systems as a filesystem. It turns out the filesystem abstraction works extremely well even for systems that are not filesystems at all. There are connector for literally every possible storage (SFTP, S3, GDrive, Dropbox, FTP, Sharepoint, GCP, Azure Cloud, IPFS....), but also things like MySQL and Postgres (where the first level folder represent the list of databases, the second level is tables that belong to a database, and each row is represented as a form file generated from the schema), LDAP (where tree nodes are represented as folders and leaf are form files), ....
The whole filesystem is available to agents via MCP [3] and has been published to the OpenAI marketplace since around Christmas, currently pending review.
ref:
[1]: https://github.com/mickael-kerjean/filestash
[2]: https://www.filestash.app/docs/guide/virtual-filesystem.html
[3]: https://www.filestash.app/docs/guide/mcp-gateway.html https://github.com/mickael-kerjean/filestash/tree/master/ser...
- agents tend to need (already have) a filesystem anyway to be useful (not technically required but generally true, they’re already running somewhere with a filesystem)
- LLMs have a ton of CLI/filesystem stuff in their training data, while MCP is still pretty new (FUSE is old and boring)
- MCP tends to bloat context (not necessarily true but generally true)
UNIX philosophy is really compelling (moreso than MCP being bad). if you can turn your context into files, agents likely “just work” for your use case
Yes, it should be able to generically use a filesystem, but there has to be a better way to find an email than greping through each email as a file.
So, I see merit in the idea in theory, I’m just skeptical in practice.
You can test it here ==> https://ainiro.io/natural-language-api
There is tons of more complexity to sandboxing, I agree!
https://en.wikipedia.org/wiki/Sandbox_(computer_security)
Notably, a sandbox exists to separate one thing from other things. Limiting/filtering/monitoring what the sandboxes thing can do are often components of that, but the underlying premise is about separation.
Containers, VMs, etc. are 100% examples of sandboxing based on the actual industry definition of the term.
You are incorrect.
If you’re going to get fired up about people you feel are misusing this term, and then ignore citations about its actual definition, I think the ball’s in your court to back up your claim.
I’ve asked what background leads to your conclusion, because if you have eg written some sandboxing tooling, I’d be curious to give it a look. Always up to learn things, and I am more than a little baffled by how upset the comments I’m replying to here sound. You’ve linked me to Wikipedia, and another commenter asserts I can ‘just look it up on google scholar’. That seems pretty dismissive and reductive overall.
Firecracker kind of ends up being in the VM categories and I would place gvisor in a similar category too under the VM
So in my opinion, VM's are sandboxes.
Of course there is also libriscv https://github.com/libriscv/libriscv which is a sandbox (The fastest RISC-V sandbox)
There is also https://github.com/Zouuup/landrun Run any Linux process in a secure, unprivileged sandbox using Landlock. Think firejail, but lightweight, user-friendly, and baked into the kernel.
Your mileage may vary but I consider firecracker to be the AI sandbox usually. Othertimes it can be that they abstract on a cloud provider and open up servers in that or similar (I feel E2B does this on top of gcp)
Furthermore, running lots of random 3rd party programs in the same instance, be it a container, or an ec2 vm, or a firecracker vm all have the same issues - it is inherently totally unsafe. If you want to "sandbox" something you need to detail what exactly you are wanting to isolate.
A lot of people might suggest not being able to write to the filesystem, read env vars, or talk over the network but these are table stakes for a lot of the workloads that people want to "isolate" to begin with.
So not only is there this incorrect view that you are isolating anything at all, but I'm not convinced that the most important things, like being able to run arbitrary 3rd party programs, is even being considered.