Hacker News Reader

Comment by objectivefs | Hacker News Reader

An alternative to user5994461's suggestion would be to switch from CephFS to a different distributed filesystem. If you need POSIX and can't go directly to S3 you could try ObjectiveFS[1]. Using local SSD instance store and memory for caching you can get very good performance even for small file workloads[2].

[1]: https://objectivefs.com [2]: https://objectivefs.com/howto/performance-amazon-efs-vs-obje...

View on HN

Another option would be to move to HDFS. HopsFS, a new distribution, can scale to over 1m ops/sec: http://www.logicalclocks.com/index.php/2016/10/14/hops-smash...

As someone who has run Hdfs in production on bare metal, I wonder how much you dislik gitlab...

TL;DR - HDFS is rarely the answer, mainly due to pains usually involving crashed nameservers.

HopsFS has multiple redundant, stateless NameNodes.

ObjectiveFS looks interesting but I wonder about the latency if you do many small random reads (this is what we're seeing with git).

Latency for small random reads (16 parallel threads), using the Linux kernel source tree, when hitting in the SSD instance store disk cache have a 95th-percentile latency of 6ms with ~50MB/s throughput. When hitting in the memory cache the 95th-percentile latency is <1ms with ~380MB/s throughput. We have more performance data available at https://objectivefs.com/howto/performance-amazon-efs-vs-obje...

That sounds very good, but I think we prefer Ceph because it is better known. Do you have a comparison between Ceph and ObjectiveFS?

Reply on news.ycombinator.com