Hacker News Reader

Comment by jankotek | Hacker News Reader

Does it make a sense to turn it into fuse fs, with transparent deduplication?

Author here. Maybe, it's a fun idea. I have toyed with providing a fuse filesystem for access to a pack but my time for completing this is limited at the moment.

Many packfile-deduplicating backup tools (bup, kopia, borg, restic) can mount the deduplicated storage as FUSE.

It might make sense to check how they do it.

I'd also be interested in how elfshaker compares to those (and `bupstash`, which is written in Rust but doesn't have a FUSE mount yet) in terms of compression and speed.

Did you know of their existence when making elfshaker?

Edit: Question also posted in your Q&A: https://github.com/elfshaker/elfshaker/discussions/58#discus...

(Copying from Q&A) Before starting out some time ago, I did some experiments with bup. I had a good experience with bup and high expectations for it. However, I found that quite a lot of performance was left on the table, so I was motivated to start elfshaker. Unfortunately that time has past so I don't have scientific numbers for you measured with other software at this time.

As an idea of how elfshaker performs, we see ~300ms time to create a snapshot for clang, and ~seconds-to-minute to create a binary pack containing thousands of revisions. Extraction takes less than a second. One difference of elfshaker compared with some other software I tested is that we do the compression and decompression in parallel, which can make a very big difference on today's many-core machines.

Reply on news.ycombinator.com