Hacker News Reader

Comment by filleokus | Hacker News Reader

How does it compare to other players in the area? E.g Ceph, Gluster or Seaweed? (I'm no expert myself, only used those as a consumer of already setup systems)

EDIT: There is a whole comparison section in the docs that I missed: https://juicefs.com/docs/community/comparison/juicefs_vs_cep...

View on HN

Bit weird comparison. Like sure CephFS doesn't support S3-like access... because the object store is a separate service that also runs on top of Ceph/RADOS store

It is weird, but it's also a valid use case. I can imagine someone wanting to pull files from a FS that was populated as a regular POSIX filesystem through an S3 api. I'm not sure if you can access the CephFS files from the underlying Ceph store easily.

You could run Minio just like they do.

It's doable to run a MinIO gateway on top of CephFS mount point, but that will has performance issue, especially for multipart-upload and copy. That's why we put MinIO and JuiceFS client together and use some internal API to do zero-copy uploads.

It seems to be a specific part of SeaweedFS, i.e. the "filer+client" components. It will use a database or key-value store for metadata and a blob store for data, and expose that as a filesystem.

The difference is that SeaweedFS has its own blob store ("volume server") while JuiceFS uses S3 (or some other protocols). SeaweedFS also decouples the server ("filer server") from the client ("mount" command or client libraries) while JuiceFS only has a single process, so the machine where you mount the filesystem talks to the metadata and data backends directly; this means you can't mount a filesystem on an untrusted machine if I understand correctly (you need full R+W access to the backends from the machine where you mount).

You can see it as similar to `rclone mount`, which allows you to mount a remote S3 bucket locally. The difference is that JuiceFS is much faster and filesystem-like, by storing the metadata in a separate faster database and by chunking your files in the backend rather than storing files unchanged in the bucket.

What I really want is a filesystem I can span across geographically remote nodes that's transparently compatible. I should just be able to chuck files into it from my NAS like any other. I think Mayastor [1] might get some of the way there?

[1] https://github.com/openebs/mayastor

It compares pretty well to the proprietary WAFL/FabricPool technology by NetApp:

- have "hot" blocks on local storage (SSD cache) - "cold" blocks are stored on S3 - POSIX semantics on top

Having worked with NetApp technology for >10 years this is a welcome addition on the Open-Source side of things.

The comparisons are helpful, but I'm curious why Ozone wasn't included, since that seems like the most directly comparable alternative.

Apache Ozone is not POSIX compatible, even with the File System Optimized format [1].

https://ozone.apache.org/docs/current/feature/prefixfso.html

That's a fair point. Thanks!

Reply on news.ycombinator.com