My main take aways from this are that cloud vendor lock-ins are real, and they can be hard to break free from.
Perhaps that's more of a cautionary tale for new projects than a justification for the expense though.
Perhaps that's more of a cautionary tale for new projects than a justification for the expense though.
You can find case studies for both positions:
- migrate to AWS to save money: Netflix, Guardian newspaper [1]
- migrate away from AWS to save money: E.g. Dropbox [2]
A lot of companies (especially non-tech businesses) don't have the technical skills to run internal datacenters at the same competency as AWS. Thus, they don't want to be "locked in" to their own IT department that's slow and handicaps their business.
Dropbox, Facebook, and Walmart would among the very few that can competently run their own datacenters with advanced services like AWS.
[1] https://web.archive.org/web/20160319022029/https://www.compu...
[2] https://www.google.com/search?q=dropbox+migrates+off+aws+sav...
[0]: https://aws.amazon.com/solutions/case-studies/dropbox-s3/
The story described Dropbox moving "34 PB of analytics data (Hadoop)" to AWS.
My reading of Dropbox's Magic Pocket / Diskotech appears to be storage for customer raw data -- similar to BackBlaze type of raw storage.
It's 2 different use cases so it's not surprising Dropbox found AWS to be effective for analytics workloads. AWS has an extensive portfolio of software services to analyze data so Dropbox may have concluded paying AWS would cost less than reinventing the analytics pipeline in-house.
It's not that migrating out isn't possible, it's that Amazon is providing "Engineering/SiteOps Departments as a Service" at a price that's hard to compete with in house.
From the GP's post - I was trying to say I read this as "the collection of services AWS provides would require several in-house engineering teams to compete with" not "vendor lock-in".
A single service is relatively easy to replicate if it is core to your business, but an entire on-demand datacenter w/ abstractions like Time Series databases, pub/sub services, etc. isn't as trivial to do yourself. Many of these services require teams of engineers to manage at scale, and engineers that _understand_ them well.
The on-demand service catalog of the cloud provides significant value. It's more than "pub/sub" or "blob storage" as a service. It's an entire engineering organization and data center as a service w/ pre-built architectures for you to start using today.