Semaphore – Modern UI for Ansible

I don't think I agree anymore; most of what I'd get done with Ansible now I'd do with a container, and if all you need to get done on the host is networking and a reasonable Docker setup, Terraform is good enough to get that done. I'm a lot less likely to ever use Ansible now.

But, I mean, the answer to the question above is easy. :)

I've noticed that I used Ansible less and less because of kubernetes.

Now we also use crossplane.io and we never touch terraform anymore as well.

It's all helm charts now here.

I am a Systems Engineer / "DevOps Engineer". Every day I regretfully use Terraform, Ansible, etc. I have been using them for years, and I have been in the industry nearly 2 decades.

They are shitty at what they are designed to do. Like, Terraform is supposed to make it "safer" to make changes, right? Except tools like Terraform are not actually designed for safety. They are designed to "make changes", and then they have one or two random features added, and somebody goes "welp must be safe" and goes on with life. Later you find out, oh, this thing isn't safe at all. Like, it has no actual safety features. It just has a DAG that you can preview. That's not a safety feature.

A safety feature would be the ability to control, for example, if you always want to create some infrastructure, including blowing away anything stupid enough to have the same name as the thing you wanted to create. If you don't have that feature, you can not guarantee that your infrastructure will always apply. Often, very often, including as I write these words right now, Terraform refuses to do the one thing I want it to do: apply my changes. If it can't guarantee that it will make a best effort attempt to apply my change, I can't trust it.

Then there's just, like, useful features. Like automatically generating Terraform based on some existing infrastructure. Since version 0.0.1, people have been asking for this. Eventually Waze actually created their own project just to do this. And you know what? This feature is fucking amazing. Rather than spending hours looking up docs, crafting configs by hand, and trying to apply your Terraform in a test environment, so that you can finally have some IaC... with Terraformer, you just click some stuff in the AWS Console, and then dump it out as Terraform. It saves hours, days, even weeks, off of creating these stupid configs. But the geniuses at Hashicorp decided they just didn't like the concept of this, or that it might require some work to create, and abandoned the idea.

Then there's 1990s-level technology, like locks. Locks. It can't figure out fucking locking. Even by the late 90s, our shitty init scripts that created lock files figured out how to tell if a lock was stale and overwrite it (or prompt you to overwrite it). And modern systems have been doing all kinds of complicated distributed locking for a decade-plus. Terraform has not figured that out yet. So if you happen to work with multiple people using remote state, and there's a state that's been locked for 12 hours? Terraform has no clue what to do. And of course, the person who accidentally locked the state 12 hours ago is nowhere to be found, so after you ping a bunch of people and back up the old state by hand, you force-unlock it and pray.

Then there's just... running it. It's hard to use. Every single person who has ever used Terraform at a company has written their own wrapper just to run Terraform. Terragrunt is the most well known wrapper, and that god-awful mess just makes it more complicated without making it significantly better than a shell script.

Then there's the state file. It's practically pointless. Oh, great, it can point out when the state changed. But it can't reason about what version of the state matches what version of the code. And if something in AWS changes that Terraform didn't do? Or that Terraform did, but didn't get committed to state? Good luck to ya. You are left to clean that shit up on your own. No attempt to automatically import the existing infra, you gotta do that by hand. No ability to overwrite the existing infra, you gotta delete it by hand.

....... Oh , what's that? It's your production environment, and now it's down, and there's absolutely no way for you to fix it? So your site is now down until you can manually fix all the bullshit that Terraform needs to run correctly again? Well apparently you should have expected that!

What a fantastic tool. You only have to use it for years and run into enough brick walls to learn the "right way" to use it to minimize (but never eliminate) all these problems.

I'll start keeping a legit log of every problem I run into and publish it online. I rant into Slack like every week with another random TF problem. People need to understand that this tool is a lemon.*

It's not idempotent, it's not even "declarative" (what a joke that concept turned out to be), its dependency tracking / detection sucks, trying to manage changes between state files and code is a nightmare (there is no relationship between the state version and the code version - and these people made a DAG?), the arbitrary way it's designed to locate and use files / modules is unnecessarily painful and restricting, the plugins are usually pretty bad (no consistent conventions, inconsistent code coverage, difficulty in passing/embedding state or data between providers), the limitations of how you can compose HCL (as in what parts of what functions / data objects / attributes / etc you can use what syntax or features) is literally because they just designed it wrong and you have to wait for the next breaking change for them to re-design it to function correctly again, the state file is literally a boat anchor that you can't pull up, they intentionally leave auto-importing out because "we think it's a bad idea" so somebody else had to invent a whole other project just to implement it, magic environment variables, the "features" you can only use if you're a paying customer, the shittily designed opinionated features that don't work outside one use case (workspaces), the masses of duplicate code from modules being a crappy abstraction, lack of backwards compatibility, the incredibly consistent ability for the state to break on apply causing you to copy old state, look up man pages, import current state, manually edit json files, and hack up code just to be able to apply one single change to one single resource, and the biggest flaw of all, there is no abstraction that allows you to just say "make me a VM in these 3 clouds" because everything is so tightly wound around very specific APIs for specific providers with specific features etc, templating sucks, and the "structure" of how you "use it" is so unnecessarily stupid that there are at least 5 projects just dedicated to making it manageable for a human being to run the damn thing.

That is literally just from the past 3 days of using Terraform. It's not fantastic, it's cancer. The designers are either morons, or don't use their own tools so they don't see all the problems, or they just don't care, or they intentionally made it Regretware so that once you feel enough pain you will pay them for better features or to manage it for you. Probably a combination of all those.