I've read that CERN is storing more than 1 exabyte of collisions data these days (up from 600PB during the last long shutdown https://information-technology.web.cern.ch/sites/default/fil...).
Not too shaby...
You can have more than one pool.
* CTA for tape storage: https://cta.docs.cern.ch/v5/
* EOS for disk storage: https://eos-web.web.cern.ch/eos-web/
There is a large CEPH cluster as well but that isn't really used for physics data.
> Data is stored natively in XFS filesystems on hard disks or SSDs or on virtualised back-end storage (e.g. RADOS block devices) or distributed filesystems like Lustre or CephFS.
Hard to Google for it without getting AI slop on it, but apparently they built their own stack in 2019.
Not sure I like their solution, "Meta-data is persisted in RocksDB databases using a proprietary KV store called QuarkDB." Unless QuarkDB has magically removed RocksDB's amazing ability to corrupt and lose data frequently, this whole thing sounds like a bad idea.
Also, their data is not stored on any one system (local XFS, Ceph, Lustre), a recipe for disaster.