I encourage my competitors to marry themselves to a vendor’s proprietary stack.
Source: 18 years in tech. Have done my share of unwinding from vendors
What you're talking about here is survivorship bias -- yes, indeed you will find people paying big sums to migrate off of AWS for whatever reason. But they're still around! Unless of course you assume using AWS (or any vendor for that matter) provides no competitive advantage whatsoever, even in the short term.
The specific category of risk you are talking about here are tech ventures that fail because of a specific sequence of events:
- They choose AWS
- They, miraculously, create a successful business on AWS
- Some existential risk emerges due to their use of AWS
- They either fail or cannot afford to move off of AWS
- This singular decision causes their business to fail catastrophically
- And, they would have been successful had they not chosen AWS in the first place
This is a very specific path to failure and is highly unlikely relative to the more common cases of "they never found product-market fit" or "they had a toxic culture" or "they decided to build everything themselves and ran out of runway."
I work for a ~billion dollar organization in risk management. If developers can't certify to our team that you can pull your app off of $cloud_provider and have it running on prem or in another cloud provider in a few weeks, it doesn't go to a cloud provider. We'll assist with their IaaS/PaaS tooling selection, infrastructure orchestration, container lifecycle process, and other ancillary needs to ensure this requirement can be met.
The issue isn't survivorship bias, it's risk management, business continuity, and cost management. It's entirely possible your organization doesn't place a high dollar value on these risks, but that's a business decision each organization needs to make for itself as a whole, and its products and services individually.
A) I start a company, get some investors and sell the company to a larger company that has the resources to either keep maintaining it or change out the infrastructure (Instagram). Result: I profit
B) I start a company, gain product market-fit, become successful and I have the resources to gradually move my infrastructure when the need arises: Result - profit.
C) I worry about "vendor lock-in" from day one, spend a lot of time and resources on my own infrastructure and run out of money and never survive long enough for A or B.
I know which one I'm going to choose.
And as far as not "needing the complexity", the entire purpose of going to AWS is to avoid the complexity and up front cost of providing your own infrastructure.
The HN darling Dropbox started out on AWS because it was faster and then when they grew they built there own infrastructure.
Netflix built their own infrastructure and decided that infrastructure wasn't their core competency and went entirely to AWS.
There is no one size fits all.
The reality is you'll need to do it when you are tight on resources and you're stuck rebuilding what has become a very complex infrastructure (because you spent the last 5 years hooking into every proprietary bell and whistle) from scratch - with no internal expertise because you didn't start early when it was still a tractable problem.
I'm not saying you need to choose between AWS or running on your own physical hardware. But I am saying you don't want to lock in to the proprietary parts of AWS, which do tend to add lots of complexity to your infrastructure because they make that complexity easy to use.
Spinning up some ubuntu ec2 servers is fine - you can move those anywhere easily.
I've worked for a company that had two issues - badly written software and a badly architected AWS environment. Guess which one was easier to fix?
All of the companies that tied their fate to Webforms, Ruby (Twitter), or some other deprecated tech have far bigger problems than someone who tied themselves to AWS.
As far as AWS what complexity is that?
Databases - all of their databases are compatible with standard databases except for DynamoDB. DynamoDB sucks anyway. If I wanted a managed NoSQL database, I would go with one from Mongo.
Messaging - You usually end up sending and reviewing messages from one central place in your code, changing your messages system out is pretty simple.
Caching - ElastiCache exposes either a Memcached or RedisCache client.
Load balancing - the only thing special you do as far as software is have a health check endpoint. I used the same endpoint that I used on prem with Consul+Fabio.
ElasticSearch -same as using your own.
Autoscaling - I've got nothing. I've never tried doing autoscaling on prem.
Codepipeline/CodeBuild/Code Deploy - the triggering event is nothing more than a standard Web hook from your git repo of choice, Code Build is a preconfigured Docker Container that executes commands based on a yml file. I use a Python Lambda function for deployment, but I could run that in anything.
Even Lambda if written correctly - your handler should be skinny and treated like a Controller - shouldn't be that hard to port over.
Fargate/ECS - orchestration for standard Docker containers.
Parameter Store - yeah I could (and have) set up a cluster of Consul/Vault servers but why would I instead of using a service that ties in security roles and encryption with everything else?
Load Balancing - I've also set up my own Consul+Fabio clients on web servers/app servers for load balancing and service discovery. Not trying to do that anymore either when I can configure an ALB and an autoscaling group.
The software wouldn't be the hard part to move over. Even if we ran everything in EC2 instances, the hard part would be duplicating multiple geographically redundant VPCs in multiple locations in the US and an environment in Asia that was close to the outsourced developers.
Netflix went all in on AWS and much of their tooling is tied tightly into AWS. If Netflix can trust its entire business to AWS, I think anyone can.
This is a terrible argument. Netflix is simply a media streaming company, with very little importance compared to other businesses. Most of its data is served from its own CDN!
Check out a few of Netflix's blog posts, AWS re-invent presentations (available on YouTube), the podcast interview, I posted earlier, and thier open source project. Netflix uses a lot of AWS services.
Edit: just realized I duplicated scarface74’s comment. My bad!
On a more advanced level, using CloudFormation or Terraform is not rocket science and neither is setting up a CodePipeline.
When you grow big enough to need someone to handle netops full time, you can hire an overpriced consulting company or find a developer who knows the ins and outs of AWS and dev ops. But for smaller companies, managing AWS is not worthy of being a full time job if you know your stuff around automation.
I'm mostly either a "Senior Developer" or an "Architect" depending on how the wind is blowing, but from working with "AWS architects" and consulting companies, I know I can hold my own against most non-Netflix level AWS architects.
But I'm glad we have a third party consulting company to handle the drudgery that I don't want to do even though I do have admin access to the console.
The most likely outcome has already played out. A lot of people aren't moving out of their datacenters, they're just putting new workloads in AWS. Sure, some big players actually moved, like Netflix, because the gains outweighed the cost of moving, but most enterprises don't have the budget or don't care.
If something ever comes along that is significantly better, most people won't move out. They'll just put new workloads in the new thing.
Drew Houston, CEO of Dropbox, confirms my stance argued in this thread on Dropbox’s first earnings call: https://www.cnbc.com/2018/05/10/dropbox-earnings-q1-2018.htm...
Netflix moved out of their datacenter and into the cloud because their workload was different enough to warrant the investment. Most enterprises don't have enough of a different workload for that to make sense, so they don't move out of their datacenter.
Dropbox didn't actually move out of AWS, they just started storing new data in their new datacenters. They did it because it was cheaper at their scale, it had nothing to do with lock-in.
And for the same reasons, when the next big thing comes along, most companies won't move out of AWS. Unless there is a significant cost savings or significant new architecture, they won't move. They will just put new stuff in the new thing. Lock-in isn't an issue for most enterprises today, and won't be in the future. Inertia is the biggest thing that stops them from moving old workloads to AWS.
If the provider of a proprietary solution stops offering it you might have no easy path of migration where as hypothetically FOSS solutions are easier to self host or find a new hoster for (and thus reducing the cost of a potential migration)
But that goes back to the old saying "no one ever got fired for buying IBM." The same can be applied to Microsoft or AWS. Yes MS has put plenty of technologies in maintenance mode but you didn't have to rush to migrate. IBM still sells system that are compatible with what they sold in the 80s.
I would be more worried about my one colocated data center getting shut down or getting destroyed than the entire global AWS infrastructure that has data centers in 50+ availability zones.
As far as software, most AWS services are just managed versions of available software. Even if you're using lambdas for API endpoints, if you treat your lambdas like Controller actions, it shouldn't be that much of a heavy lift to concert them into standard MVC APIs equivalent.
For another perspective, listen to the podcast interview of Adrian Cockcroft, the guy who led the migration of Netflix's infrastructure from self hosted to AWS hosted.
http://www.se-radio.net/2014/12/episode-216-adrian-cockcroft...