Google Acquires Elastifile | Hacker News Reader

Google Acquires Elastifile

cloud.google.com

91 pointsbjoko17 comments

Looks like Elastifile allows users to "mount" cloud buckets to their servers as NFS. This allows them to run old desktop workflows without the need to migrate these workflows to the cloud.

The upside is that Google Cloud is getting serious about the enterprise because most of them still use desktop applications.

The question is how long this will last since the future is the cloud.

It will be interesting to see how well this scales. I have seen AWS EFS fall over under heavy read load (lots of reads for the same TB-scale data). A lot of poor souls using Horovod with no data team use EFS to “distribute” data in data-parallel SGD.

EFS generally only falls down when you hit your IOPs limit for the measurement period -have seen this soooo many times. Buy more storage/IOPS, and monitor that puppy.

IOPs is one of the main pitfalls of cloud platforms and I'm baffled that neither GCP nor AWS make it easier to detect that you are hitting a limit there.

I've had multiple gigs where I was tasked with solving performance pitfalls, and most of the time it was as simple as upping the IOPS. In one case even though the problem was resolved in the end, the client migrated away from the cloud again because of that bad experience.

For TB-scale heavy read loads ObjectiveFS[0] is an alternative to EFS that don't have EFS's IO/bandwidth limits. It is often used for distributing data for machine learning and large scale simulations on EC2 and GCE.

[0]: https://objectivefs.com

Do you have more details about this?

I would be interested to see how it compares with AWS EFS under a variety of load.

AWS EFS implements NFS 4.1 which support has support for massive parallel IO.

http://www.pnfs.com/

I guess this will be the Google Cloud equivalent of Amazon’s EFS?

There's already GCP Filestore.

Filestore is basically a single google-managed instance running NFS. It's much less sophisticated and resilient than EFS, but also can be much faster since it doesn't have the overhead of a distributed system. Don't try running something that touches lots of files like mercurial on EFS, trust me...

The article says Elastifile (which I know nothing about) will become part of filestore.

Filestore is also really expensive for what is basically just NFS.

Managed services are generally for enterprise, where you're fully costing out support + security mitigation.

Eliminating team coordination tips the balance.

EFS is... surprisingly cheap, as long as you don’t need heavy sustained IO for more than a few hours at a time. Great for a simple file store and some big data purposes.

https://www.youtube.com/watch?v=H27rfr59RiE

Elastic is next

A disk that can be attached to multiple VM's and support simultaneous reads/writes is a huge addition to GCP.

Google Cloud Filestore has existed for some time: https://cloud.google.com/filestore/

Reply on news.ycombinator.com