HNHacker News
TopNewBestAskShowJobs

Eikon

1,270 karma · joined December 12, 2019

submissionscomments
Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
Kind-of should work :)
Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
NFSv4 is a hard beast to implement correctly, with a lot of protocol surface (state, compound ops, delegations) for benefits ZeroFS mostly gets through 9P with extensions, over a much simpler protocol: https://www.zerofs.net/docs/9p-extensions. NFSv3 stayed in ZeroFS mostly for client compatibility.
Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
ZeroFS aims to be a POSIX filesystem, the semantics here are the standard ones (ext4, xfs behave the same): write() is buffered (that's the batching) and "committed" maps to fsync(), which returns only once data is durable.
Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
Thank you, appreciated!
Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
> I believe the first version of this required the metadata to be stored on the ZeroFS server, making HA kinda hard.

Metadata has always been in the bucket itself.

For HA, there's now a "replicated mode" if you want automatic failover:

https://www.zerofs.net/docs/high-availability

Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
That's fair advice, thanks.
Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
Is it? :)

Ask me anything!

Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
Thank you for the feedback, the idea behind this was to say "We make claims that are backed by workflows you can verify". I'll improve the phrasing.
Eikon··on Show HN: ZeroFS – A log-structured filesystem for S3
Just a self hosted plausible instance :)
Eikon··on I taught a bucket to speak Git
It's there! https://www.zerofs.net/docs/kubernetes-csi
Eikon··on I taught a bucket to speak Git
Most of the pain here is the typical set of issues people run into trying to make S3 a filesystem as-is, common with S3FS-family approaches.

ZeroFS (https://github.com/Barre/zerofs) is 9P/NFS/NBD over S3 on an LSM. Point stock go-git, or just /usr/bin/git, at a mount and skip the gymnastics. Rename is a metadata op in the keyspace, so you get it atomic on any S3, no Tigris-specific X-Tigris-Rename needed.

Different point on the spectrum, but less square-peg, also most probably much, much faster (it works great on linux-sized repos) :)

Eikon··on Show HN: ZeroFS – Make S3 your primary storage
> Is this only was S3 or any S3 compliant service?

Any S3 compliant service.

> Does it support any sort of storage tiering ?

Yes, it has a local cache that's populated on reads, but no writeback cache yet.

Eikon··on Show HN: ZeroFS – Make S3 your primary storage
> Millions of files, Interrupted transfers, Backup Server Reboots, Long-term operation?

Yes, that should work fine!

Eikon··on Show HN: ZeroFS – Make S3 your primary storage
> I assessed the tool on a CentOS 7 system today. It can't run on older CentOS 6. It will generate core dump.

Perhaps your cpu is too old? The prebuilt binaries require newer CPUs but you can build your own.

> Will ZeroFS support large Filesystem size about 800GB?

Absolutely, and much, much more than that.

Eikon··on What is the purpose of the lost+found folder in Linux and Unix? (2014)
Even with journaling, you might need one. ZeroFS [0] almost had a lost+found directory (even with the WAL enabled), because you might have consistency issues between your in-memory state and what was flushed, and especially in what order.

ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).

[0] https://github.com/Barre/ZeroFS

[1] https://github.com/Barre/ZeroFS/blob/main/zerofs/src/fs/writ...

Eikon··on Barman – Backup and Recovery Manager for PostgreSQL
https://github.com/Barre/ZeroFS Should do a great job at this.
Eikon··on S3 Files
It's definitely shared, and can be redundant.
Eikon··on S3 Files
Or ZeroFS which doesn’t require a 3rd party database, just a s3 bucket!

https://github.com/Barre/ZeroFS

Eikon··on S3 Files
SQLite works great with ZeroFS: https://github.com/Barre/ZeroFS
Eikon··on Show HN: Stop paying for Dropbox/Google Drive, use your own S3 bucket instead
See also: https://github.com/Barre/ZeroFS

Doesn’t require an external database (just a s3 bucket) and is a single binary. A webui is shipping in the next few days.

Eikon··on GitHub appears to be struggling with measly three nines availability
As of recently (workflows worked for months) I even have part of my CI on actions that fails with [0]

2026-02-27T10:11:51.1425380Z ##[error]The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled. 2026-02-27T10:11:56.2331271Z ##[error]The operation was canceled.

I had to disable the workflows.

GitHub support response has been

“ We recommend reviewing the specific job step this occurs at to identify any areas where you can lessen parallel operations and CPU/memory consumption at one time.”

That plus other various issues makes me start to think about alternatives, and it would have never occurred to me one year back.

[0] https://github.com/Barre/ZeroFS/actions/runs/22480743922/job...

Eikon··on SSH has no Host header
I am not sure to understand what this is this achieving compared to just assigning a ip + port per vm?
Eikon··on Hetzner Prices increase 30-40%
English: https://docs.hetzner.com/general/infrastructure-and-availabi...
Eikon··on Run Pebble OS in Browser via WASM
https://joelseverin.github.io/linux-wasm/
Eikon··on Litestream Writable VFS
You might be interested in ZeroFS [0], it works great with Postgres [1].

To achieve what you describe, you should be just able to setup a Postgres replica that’s setup on top of ZeroFS.

[0] https://github.com/Barre/ZeroFS

[1] https://github.com/Barre/ZeroFS?tab=readme-ov-file#postgresq...

Eikon··on What is Plan 9?
ZeroFS [0] is very thankful for what it brought to Linux with the v9fs [1] subsystem which is very nice to work with (network native) compared to fuse :)

[0] https://github.com/Barre/ZeroFS

[1] https://docs.kernel.org/filesystems/9p.html

Eikon··on JuiceFS is a distributed POSIX file system built on top of Redis and S3
It’s not great UX on that angle. I am currently working on coordination (through s3, not node to node communication), so that you can just spawn instances without thinking about it.
Eikon··on JuiceFS is a distributed POSIX file system built on top of Redis and S3
> so the commercial license would only be needed for closed-source modifications/forks

Indeed.

Eikon··on JuiceFS is a distributed POSIX file system built on top of Redis and S3
It doesn’t, you are free to use ZeroFS for commercial and closed source products.
Eikon··on JuiceFS is a distributed POSIX file system built on top of Redis and S3
> but I find it kind of off-putting for you to come in and immediately put down JuiceFS, especially with benchmark results that don't make a ton of sense, and are likely making apples-to-oranges comparisons with how JuiceFS works or mount options.

The benchmark suite is trivial and opensource [1].

Is performing benchmarks “putting down” these days?

If you believe that the benchmarks are unfair to juicefs for a reason or for another, please put up a PR with a better methodology or corrected numbers. I’d happily merge it.

EDIT: From your profile, it seems like you are running a VC backed competitor, would be fair to mention that…

[1] https://github.com/Barre/ZeroFS/tree/main/bench

Page 1 of 13Next →