- Good CF template is 10x less code for the same solution.
- No corrupted state problems.
- Native tool, supporting all properties of resources
Writing good CF templates takes good AWS knowledge, and system thinking, you group resources that belong together, it actually teaches you good architecting.
- Much better than Cloudformation at telling you what it's going to change before you apply the changes and the ability to record those changes. (much better than those dreaded 'conditional' changes)
- The ability to import changes if you found some that were done outside of Terraform. It's not perfect, or easy, but mostly doable.
- The ability to look at the code, the state file and the plan to get a good representation of what's actually deployed.
Those three are more significant than it looks, but together it makes sure you:
- Don't get into a situation where automation is broken and you can only recover by rebuilding the stack.
- Don't get unexpected downtime because a change replaces a resource unexpectedly.
- Being able to track, record and manage changes in easy to read diffs and plans.
CF: to look at template code only, to know what was deployed.
The latest example I have is the cloud formation used to generate EKS clusters(https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-07...) it clocks in at 168 LOC and the equivalent terraform (https://raw.githubusercontent.com/terraform-providers/terraf...) is only 53 LOC
resource "aws_s3_bucket" "x" {}Furthermore, the other comments on this post should disabuse you of the notion that there are "no corrupted state" problems with cloud formation: they happen all the time.
Disclosure: I worked on Terraform for quite some time at HashiCorp and am still a (community) maintainer of the AWS provider.
CFN has its warts, but I full-stop don't trust HashiCorp's operations or their attempt at a SDLC and I wouldn't trust my business's health to them as a company (and if the results of using it weren't enough, their clownshoes sales team's bad attempts to upsell would cinch it).
I mean, it's weird, because I agree with your statement that "The most important part of infrastructure engineering is being able to debug and fix things quickly by isolating issues to the smallest possible domain." And that's why I so strongly prefer Terraform, because I actually have control over the state file, how Terraform interacts with it, and I can move things in and out, and change things in-situ if necessary.
That. It's most dangerous thing for long term projects - third party tools and services. You want to minimise that to absolutely necessary ones.
As for state, I'm not sure what you did to corrupt your state, but we use Terraform to manage thousands of resources across dozens of AWS accounts for the past three years and haven't had any state corruption, except when a human messes up editing a state file by hand. Obviously in that case you back things up first (or hopefully you are using remote state with some kind of versioning). But the fact that you are _able_ to manipulate the state with the CLI tool, or by hand in extreme cases, is itself a huge advantage over CloudFormation, which has no such capability.
As for coverage, my experience has been that Terraform often has coverage of new resource types and properties _before_ CloudFormation. And it's extremely rare for any new features to take very long to show up in the AWS provider. Anything significant is usually picked up in 2-3 weeks from the API release at most.
That said, my experience has been that both CloudFormation and Terraform are irritating, just in different ways; they both are warty.
I do ultimately prefer Terraform - even in a single-cloud setup.
Recently, I decided to use Terrform over CloudFormation specifically because you can't create an EKS cluster (with nodes) in a single stack.
The rule of thumb that you should generally stick to CloudFormation if you are full bore invested into AWS has some truth.
My issues with CloudFormation are lack of control over rollbacks, missing features for existing and mature services like the above, and forcing me to use custom resources to do anything that vaguely resembles coding that Terraform does just fine like IP address math functions.
Unfortunately, this is far from the case. I can name 3 things off the top of my head not supported in CloudFormation.
1. The ability to create Route 53 records for Certificate Manager
2. Create a full EKS Cluster (with nodes).
3. EC2-Fleet.
* Error messages are overly verbose yet cryptic, and sometimes even unrelated to the actual error raised by the cloud provider themselves. Coupled with the lack of line numbering or other helpful identifier aside the unnecessarily long module hierarchy and debugging those scripts is a massive exercise in frustration and usually far more time wasted than really should ever be necessary.
* HCL is a hateful "language". The fact you cannot order stuff procedurally means you're constantly running into dependency issues on larger deployments. And dont even get me started on the "count" kludge to work around a lack of proper iteration.
* There is a lack of internal consistency with the support of different methods. Eg "count" does t always work with all resource types. Some resources cannot have properties defined with variables.
* Using calling modules requires so much bootstrapping code. It's just painful.
I get Terraform is the best we have for multi-provider deployments but their idea to create a superset of JSON only to then compile that back down to JSON anyway was such a poor decision in my personal opinion. I get the point was to have something that was accessable to non-programmers while still expressive enough for developers to use; however instead what they've created is a monstrous language that is too complex for the former group and too irrational for the latter.
I've been been very tempted to write my own Terraform alternative based on my experiences using it (and CloudFormation) - I even already have another programming language that Ive written a parser for and would be well suited for this type of application. But my time is pretty limited at the moment so I struggle on Terraform.
{
"foo": { "bar": 1 }
}
That said, you might be interested in Terraform 0.12 [0], which will be using some new HCL v2 that actually has first-class expressions and dynamic blocks (for loops). And, finally, a ternary operator that short-circuits. Unfortunately, the dynamic block stuff looks like it's based around for-loops and doesn't support just regular if-statements... but we'll see where that goes.
I wasn't aware about the JSON subset / Consul problem though. That's really interesting to read. It's funny because back when I was building test Consul cluster I did wonder why JSON was used for config instead of HCL. I guess now I know why.
Otherwise, about like you, I'm tempted to write a Terraform frontend that interfaces with existing providers...
Infrastructure as code tooling is all very primitive compared to what we take for granted writing most other traditional software but it will take some time and maybe another generation to do it well.
I know Terraform is the best tool we currently have (I even explicitly stated that I'm my previous post) but that doesnt mean there isnt still a massive room for improvement. Starting with the depreciation of HCL, in my personal opinion.
Plus the pace of development in nodejs worries me. All too often I've ran into issues where modules have changed and broken things downstream. When you're running infrastructure as code you really want to be damn sure your tooling is going to be consistent for years to come and I really don't have that faith in nodejs. Sure, if your a JavaScript developer you can manage it easily enough, but if you're DevOps who rarely touches JS then you really want your tools to be low maintenance. So to that end I wouldn't consider any nodejs projects for any serious production work given the kind of customers I work for (high availability stuff for some major brands). I might be fine but it's just not worth the risk.
#backend/state uses less priv keys
terraform {
backend "s3" {
bucket = "mystatebucketisunique-tf"
key = "states/tf/tf.tfstate"
region = "us-east-1"
lock_table = "lock-tf"
profile = "aws_tf_s3-prof"
}
}
Just, no.