Hacker News Reader

Comment by daenz | Hacker News Reader

Somewhat related, I wrote a fuse-based file system in Rust recently that used SQLite as the backing store for file records, though not the file contents. I imagine I could use it for file content as well, so it's good to know more about its performance.

https://amoffat.github.io/supertag/

View on HN

I wanted to learn about Fuse and SQLite in Golang, so I built a filesystem that was entirely stored in SQLite. It's incomplete (read-only for directories at the moment, among other nits), but was surprisingly easy and performed with decent performance on some tests I did; Nearly half throughput of a raw filesystem.

I’ve wanted to do something like this for ages- definitely going to check this out! The limitations imposed by the hierarchical structure of traditional file systems bothers me rather frequently.

Are there any notable trade-offs or limitations you’ve encountered with this approach?

SQLite, iirc, has a limit of 1GB per row, and that might too severely limit the utility of your file system if you don't end up splitting files into multiple fragments (rows in the database).

You read too fast:

> SQLite as the backing store for file records, though not the file contents

I believe they are referring to:

> I imagine I could use it for file content as well

So did you:

> I imagine I could use [SQLite] for file content as well

You are right, by default 1 billion bytes per row or blob. Can be raised up to 2GB with compilation parameter[0]. Storing video streams may be not a good idea, but e.g. photos, thumbnails or some JSON documents may be absolutely fine.

[0] https://sqlite.org/limits.html

Very cool! I've been using TMSU for this purpose for a while, but I'll have to check out supertag :)

Reply on news.ycombinator.com