I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
Yes there are open source public/private cloud alternatives like OpenStack but they won't solve the problem of lock in and you'd still have to change all your tooling.
That said before I found Kolla I did deploy openstack manually a few times before messing things up and having to restart.
Even "just" to emulate a single region with 3 AZs would be useful for testing and for prod for many people. I know of several large organizations that to this day haven't scaled their CI beyond one region.
Same deal with Azure and GCP, although note that Azure has Azure Stack, which is a self-hosted version of Azure (I guess it's still expensive though, and I don't know how many services it includes).
"Setting Up DynamoDB Local (Downloadable Version)" https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
> The downloadable version of DynamoDB is intended for development and testing purposes only. By comparison, the DynamoDB web service is a managed service with scalability, availability, and durability features that make it ideal for production use.
"...Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow..."
Conflating lock in to an open source tool with lock in to a vendor who you literally have to keep paying money to survive is a shitty tactic used by companies like Amazon.
Using open stack absolutely solves the problem of lock in from a business perspective. You can’t be cancelled, the price can’t be changed, etc.
Just because something is open source does not mean your org will actually have the expertise to be successful at running it yourself. That's how most "open source" projects like Kubernetes and OpenStack become so successful: the supporting orgs behind the project make lots of money charging companies for support and maintenance and issues that pop up with the setup.
There are many forms of lock in. Buying a big upfront investment that has to pay itself off for the next X years is a form of lock in. Paying for expensive professional consulting support to build out the system and keep the system up and running is a form of lock in. Or even worse, hiring or training expensive dedicated employees to learn the open source system and become the internal professional consultants for the rest of the org, that is also lock in.
Here’s the part where it’s not lock-in. We stopped paying for support from Redhat when we got big enough that our own SRE expertise to run our own open stack cloud.
You know what the impact was on our applications running on it? Absolutely nothing.
How much do you pay Canonical to use grep every month? Inexperienced developers have just been conditioned by cloud providers to think that an IaaS platform must cost something in payments per month to some tech company. It does not.
This is no different than Windows vs Linux on servers. I can’t wait until 20 years from now when all of the proprietary shit looks ridiculous in hindsight.
I agree fully! As you explained, it can cost several full time employees and their payrolls and their HR and their management!
Edit: And before someone misses the point, I'm not saying the math never makes sense for in-house, but to me the inexperienced take is thinking either approach should obsolete the other...
> I can’t wait until 20 years from now when all of the proprietary shit looks ridiculous in hindsight.
I'd posit that in 20 years there will still be tech which is more efficiently managed by a central provider, rather than having each company hiring their own independent expertise.
Of course every solution costs something. Lock-in means that there is no other choice, or that the cost to switch is so steep that it becomes prohibitive to do so.
I share your disdain(?) for openstack, but the 12 node count there is minimum I think (to deploy the control plane). You can likely add a reasonable number of hypervisors at no cost.
(I think I'm correct based on the table in the last page of your doc, but happy to be corrected).
See how your comment ages in five years with kubernetes.
I've absolutely seen disasters with open source that bleed companies dry. Openstack is a great example.
Datacenters are fucking expensive to build and run, buying network and compute hardware is a pain in the ass, maintaining sufficient capacity to sustain growth while operating on a 3-6 month deployment lead time is atrociously expensive, and that's without all of the financial calculation around depreciation/taxes/etc.
The fact that cloud providers are able to ball all of that shit up into a flat opex rate for a server is unbelievably attractive.
[0]: https://docs.ansible.com/ansible/latest/user_guide/playbooks... Intro to playbooks - Ansible Documentation
The way I made systems/applications before AWS existed was to create bootable images. I could already use a hypervisor, Xen, because the OS fully supported both guest and host, all virtualisation modes, before Linux did, and before AWS existed. But because I am not a hosting provider I saw little need for virtualisation. The OS I used also had "unikernel" capability, before Docker, etc. existed. As it happened, this inexpensive, self-determination was not to be the future, instead we got "the cloud". Sharing servers with other "tenants". Less expensive for the hosting provider, but more expensive and more limited for the subscriber. No argument, the "limited" part has improved since then, but it is still expensive (for continuous use that is, spot pricing was a neat benefit of "the cloud").
Anyway, I "deploy" images to "local bare metal" which is a laptop, RPi, or some other smaller form factor. I can use unikernel for some kernel drivers in userspace. Basic filesystem is embedded in the kernel, and larger filesystem is on the USB stick filesystem in compressed format. Updates are easy enough. I put two kernels on the USB stick, one is the running kernel and the other is the update kernel. Same for the larger filesystem which may contain servers and configuration files. Can update or go back to last working kernel/configuration by selecting one or the other in the boot menu/renaming the larger compressed filesystem.
Here is someone running a search engine out of his living room. AKAIK, his setup survived a sustained HN thundering herd, hug of death without a hiccup.
https://news.ycombinator.com/item?id=28552805
The expense of AWS is an obvious point of discussion but another one not mentioned here is control. When I create images for bare metal I do not need to jump through any hoops as I would in order to create an image that will run on AWS. Nor do I need to fiddle with all the AWS knobs. There are no silly marketing names for every program I run. I know the system I am creating as well as I know the OS and the software I choose. That is much better than how well I know every aspect of AWS which just gets more and more complex every year. AWS documentation is as cringeworthy as it it is voluminous. The ever-increasing complexity of AWS, including the "tooling", is, IMO, How To Create Lock-In 101
That is an interesting point. Complexity creates lock-in. Why? Because when you are interacting with a complex system you depend on it working in the complex ways it does. It is unlikely that anybody else could duplicate those features of the AWS your application is depending on.
This all runs counter to the idea of "encapsulation". You should be able to use a system via a well-defined interface. Once the interface is well-defined, other providers can provide their own implementation of the same interface.
So, AWS is basically bad software engineering, lacking encapsulation?
The company works because every night there’s 1000s of engineers sleeping with a pager by their head.
we have often delayed calls because it's often hard to explain it to some "foreigner"
I’m pretty sure it ended up where all software goes to die: HP.
- https://docs.eucalyptus.cloud/eucalyptus/5/install_guide/int...
- https://github.com/corymbia/eucalyptus/
There are quite a few moving parts. I think I got stuck around just comprehending the networking bits.
https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...
CloudSeed has been a key part of several >$1bn contracts for KnightPoint/Perspecta/Peraton, most recently DHS DCCO ($2.7bn).
We were heavy EC2/RDS users, now we run their NX whitebox nodes with their KVM based hypervisor, and their database management platform "Era".
Everything is one click for deployment and upgrading of things, down to stuff like upgrading the SSD firmware, NIC firmware, etc. It auto migrates the VMs around hot, and does one node at a time. They also have a kubernetes platform called Karbon that seems to be pretty good.
All that's old is new again.
I guess the (business) reason is that the centralized computing model enables enormous economies of scale. (Makes perfect sense from an operations point of view.)
What it seems to miss in the current incarnation/iteration, though, is the "power of distribution" - leveraging the fact that local compute capabilities (especially during development/integration) can help reduce the cognitive load, increase the efficiency, and thereby reduce overall costs.
There seems to be a tendency to focus mostly on the operational aspects, rather than the overall end-to-end developer journey. What remains to be seen is whether future iterations of "the cloud" will do a better job of embracing the power of distributed and/or hybrid.
Now it's exactly the same thing in AWS. My next guess is that you're going to be running production code on fleets of devs computers because you don't have to get extra budget for AWS next financial year to afford spinning up another instance.