Hacker News Reader

Comment by jonhohle | Hacker News Reader

> in fact, in most cases, there is no filesystem at all

Huh? With few exceptions, applications write to disk using a VFS provided by an operating system. Can you provide an example of cloud storage that uses raw disks (bypassing VFS)?

There also isn’t a POSIX file system, just POSIX compatible file systems. Those file systems may expose non-POSIX APIs to do other low level work (trim, clone, snapshot, etc.).

View on HN

CEPH would be one of those, which is more like a database using raw devices, and no linux filesystem (nor VFS calls) in between. It used to run on top of xfs, but now storage nodes (OSDs) talks directly to a raw device (or an lvm on top of it). But you can most certainly not run "ls" on such a host and get a listing of what some user wrote over S3 into it.

Are there any benchmarks that justify bypassing VFS? My Google-fu is failing and the one page I found comparing file systems in the CEPH docs doesn’t talk about raw access, all images are broken (stored on CEPH raw storage?), and the text doesn’t seem to include throughout numbers.

https://www.pdl.cmu.edu/PDL-FTP/Storage/ceph-exp-sosp19.pdf my google-fu found this

it's hard to read and summarize. It seems that the main point is that filesystems don't expose many tools ceph needs

If you store stuff on xfs or ext4, it will cause more writes due to the journal, and since ceph is a DB in some sense, it also keeps a journal/WAL so doubling that is a waste.

Hi, similar to nvm0n2 who already commented below, object storage systems often directly write to block storage rather than through a VFS. While minio, for example, can run on top of a VFS, even they say you shouldn't be - you should instead run it directly on block storage across multiple drives / nodes.

It’s not uncommon for high performance object stores to bypass POSIX FS semantics and provide kernel patches etc - eg lustre do this.

Reply on news.ycombinator.com