Hacker News Reader

Comment by chucky_z | Hacker News Reader

We use ~100GB with a lot of in/out for EFS, as a kind of 'nearline S3,' mounted to a hundred ec2 instances or so. It replaced an NFS server doing the same thing, and it ended up being slightly cheaper cause you don't pay for the compute, only the storage (efs vs. ebs + ec2).

Also I haven't had a single issue with performance. It's not particularly busy though (~800 iops?).

View on HN

We’ve seen horrendous performance on EFS unless you’ve provisioned quite a bit of storage on the volume.

Underlying S3 bits do bite you when you deal with lots of small files. Our team even considered maintaining NFS or Gluster servers over horrible EFS performance.

Another file system that works well with lots of small files and that doesn't require maintaining any extra servers is ObjectiveFS[0]. It is a FUSE based log structured file system (with snapshots) using GCS or S3 for log storage with memory cache and (optional) disk cache for performance.

[0] https://objectivefs.com

Even if you have quite a bit of storage. ~40TB on EFS, and yet performance is abysmal due to the access time being about ~3ms.

So large files work well, but if you have many smaller files it's killer.

We built our app to do caching locally into NVMe drives though, so we async pull data from EFS and push back as necessary.

Definitely going to give Filestore a spin and see if it is a good fit for our use cases.

Reply on news.ycombinator.com