If you manage to run on prem infra with vms and clusters in a true cattle mindset then cloud doesnt offer you as much for sure
I use Ansible at home as a replacement for the 1990s style of sysadmin where you manually managed your servers (which I did until the mid 2010s). Yet, I’m aware that Ansible is not declarative. It’s looks declarative, if you take your glasses off and don’t think too hard about it, but the key to understanding Ansible is really that the configuration is idempotent (not declarative).
Servers are cattle either way, it’s just that with Ansible, your servers will keep changes that you made months or years ago that have been deleted from your playbooks. That, to me, is what makes it old-fashioned.
I don’t think old-fashioned is bad. I’ve gotten into arguments where people have told me to manage my personal server(s) using K8s and I, politely as I could manage, told them to fuck off.
What about the specialized services from AWS provides leverage outside of the “cattle mindset”?
In my experience, AWS’s primary value and leverage is based on intermittent burst compute. It’s why you rarely get a hard answer on the Hz of each vCPU and have credits to the overuse or underuse of said instances. The cpu is then used to arbitrage value in the dedicated services like lambda and others - so your mindset and infra needs to match AWS’s incentives to get the same value, correct?
We use ~50 fairly high performance (RAM, CPU, disc) servers pretty much continuously, so the savings compared to AWS etc are considerable. Every couple of years there's a software upgrade required, and it would be convenient to have another 50 servers to use during the transition, just for a few days.
Compared to AWS prices, we can easily justify keeping some old servers around to help with this sort of thing. A managed server company (Hetzner etc) would be somewhere in-between; presumably able to rent us 50 extra machines for the duration, but (last time we checked) still more expensive than managing the servers ourselves.
This could be only the case is t* instances? I learned the hard way that CPU misses can add even 30% of request latency, especially when you often do I/O eg to external services like db/Redis/etc
I didn't notice such behaviors on AWS. but when thinking about it, I read somewhere that one reason they have dedicated CPUs is for partitioning CPU cache so it's not shared between VMs. so maybe with some tricks they could get some free compute without side effects?