Hacker News Reader

Comment by throwitaway222 | Hacker News Reader

I don't understand why EFS never gets major shout outs - it's way better than S3: systems can mount it as a drive, shared across systems, already has had super low latency... Not sure what s3 express is really useful for if EFS already exists.

View on HN

EFS is really expensive and has terrible latency with small files in my experience

Yeah the main reason is that it's incredibly expensive. You can improve performance by allocating ahead of time but NFS has never been at its best when working with a bunch of tiny files.

Do you have any more details you can share about the performance of EFS? I've never met anyone who has actually used it in anger.

Throughput scales with the amount of data in it, it is in the docs. So depending on the application, even if latency is better, the speeds are atrocious at lower volumes of persisted data.

That’s not true anymore with EFS Elastic Throughput

Yes, I built a moderately large system on it that used lots of small shared files. The performance was fairly terrible. There's weird little niggles with it--we had random slowdowns, throughput issues, and things just didn't work quite right.

It was an ok solution for what we were doing, but several times I came really close to just dumping it and standing up an NFS server using EBS volumes.

I also used it a couple of times to store webroots and that was a complete disaster with systems that had lots of small files (Drupal I'm looking at you).

Note that EFS One Zone is priced the same as S3 Express One Zone with similar latency. One isn't better or worse than the other, it only depends on what kind of access your application needs.

When you set up EFS did you maximize the IO settings?

Before doing that it was unacceptably slow. After doing that it was unacceptably expensive.

EFS exists if you don't care much about spend and performance while having to forklift a POSIX compliant use case into AWS for persistent data.

Thats basically how we were using it. It could have been worse.

i'm sure there's some cases where mounting storage as a disk is desirable, but from my perspective "systems can mount it as a drive" is a negative, not a positive.

treating storage as an application-controlled thing that doesn't need systems management is a good thing. i want "put this file in this spot" logic in my application code, not "put this file in this spot on the filesystem, and hope that location is backed by the correct storage layer"

I wonder if Mountpoint for S3 along with this new Express option makes it a direct competitor to EFS for some use cases.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountp...

I'm quite curious about this too - both from a cost and performance perspective. If S3 Express is close enough to EFS on these metrics, then I'd say it wins out due to the sheer ubiquity and portability of S3 these days.

Looks like support for S3 Express was merged in with version 1.30 just a few hours ago https://github.com/awslabs/mountpoint-s3/pull/642

In my experience the biggest drawback with EFS is startup time for systems that mount it in.

For example a container or EC2 instance might only need a tiny bit of your storage and with s3 can just download what it needs when it needs it.

As opposed to EFS where the container or instance needs to load in the entire datastore on startup which can add minutes to startup time if the EFS drive is large.

My understanding is that EFS is exposed as an NFS share. I haven't used it personally, but NFS mounting is generally fast, nearly instant. What does "load in the entire datastore" mean?

EFS mounting is definitely nearly instant. I use it constantly.

Many servers start up, load a ton of data from storage into RAM, and then happily serve that data for a long time. The latency of the server when starting up before it can service its first request is entirely based on the throughput of the data load.

Often these servers will load 128+GB of data into RAM (crazy, huh?) and even if you have 1GB/sec it's still two minutes for the server to start up.

1GB/s is the same speed from EFS or S3.

I didn't say otherwise.

Reply on news.ycombinator.com