Terraform Collaboration for Everyone

Yes! To me, that sounds similar (or one step further) to dev/ops folks exclaiming "Docker!" at the mildest trouble with builds / CI / isolation.

After spending time with Docker, I like it when it's deserved, but my first reaction is one of "Uh, it's one more layer, it will bring new tooling and required knowledge and error conditions, let's only do it if we have to".

While I understand where you're coming from, I can tell you're not a user of Terraform. As someone who uses Terraform daily, and has brought it to nearly a dozen other teams at my employer - these new announcements are pretty cool :-) Including the announcement about a free tier for Terraform Enterprise (Which, mind you - is optional, they are not forcing you to go through it)

Yeah, this appears to be accomplishing the same purpose as the 'Terragrunt' wrapper which uses DynamoDB for locking state IIRC https://github.com/gruntwork-io/terragrunt

I hadn't heard of Atlantis before, thank you. Very interesting.

Do you happen to know of a self-service tool (CLI and/or Web UI) that allows us to do the following: Run these Ansible playbooks, then apply these Terraform modules, then run these Ansible... etc.

We often have to build different environments for all kinds of use-cases (usually for testing, debugging or demonstration purposes). Two examples: We need a three node Kafka cluster on the Hetzner Cloud (Terraform + Ansible), we need a five node Elasticsearch cluster on AWS etc. For that we'd like to specify which Terraform modules to run with which variables, we then read the Terraform state file to generate an Ansible inventory and run a set of Ansible plays/roles.

It's there in the article.

That's what it sounds like from the description:

> Going forward collaboration features will be available for free to practitioners and small teams, at an affordable price to businesses, and Terraform Enterprise will remain our world-class platform for organizations adopting Terraform at scale.

It's not released yet though, so it won't be reflected in the pricing page yet. The beta starts "early next year".

In modest, small team use, a remote state file on s3 served me well. Did you run into problems with it?

Terraform, certainly. It was a bit rough around the edges about a year ago but it's been getting really good. I've only been using the free cli version.

Number one advantage is that it's much better at telling you what's changing. For example, where CF will tell you it will change a security group, Terraform will tell you what specific rule it's going to update with what content. Terraform will tell you if it's going to have to wipe and replace an EC2 instance, or not, where CF will tell you it's 'conditional'.

Number two advantage is that you can go from any state to getting back to managing things with Terraform again. You can start with a hand managed setup, one created by CF, or one created with TF and then broken by manual modifications, you name it, there is a way to get it back to being managed with Terraform. Sometimes it's not easy, but it's always possible. With Cloudformation there are many things that can be done where the only fix is to delete the stack and create it again.

Then for the Serverless specific example, I recently replaced something that was deployed with that with Terraform. The Serverless deployment had some big problems because of Cloudformation, one was updating stacks with where Lambda functions needed interfaces in many VPCs, the other was some Cloudformation string limit issue where the only workaround was to delete the stack and create it again, which is super slow because of those network interfaces and there would be downtime while it's doing all this. With Terraform this went from being a risky deployment with potential downtime of many minutes, to Terraform only changing the resources that needed changing with each update, taking only seconds.

Not everybody is 100% on AWS. For people that work on multiple clouds, terraform is an obvious choice.

Never be bound to a single cloud provider. Ever. Same as before the cloud, whatever is important shall have multiple vendors offering it.

Terraform is a) reliable and b) allows for 1-n providers. It's not just custom AWS, it's custom "ALL THE THINGS"

i've used both CFN and TF on the same infra so i think i have a reasonable (although dated by 1 yr) perspective on this.

my main gripe with CFN is that when things get into a weird state, it is extremely difficult to do anything about it. You are basically stuck waiting at the hands of the black magic box of AWS to either revert or complete the change set. i found that TF was much more clear about what went wrong when a changeset got into a bad state.

i've also had no problems using AWS services via TF, with the added bonus that it's easy to get heterogeneous cloud deployments working.

i last used cloudformation over a year ago and it was a pretty shitty experience overall. bread and butter resources like vpcs would fail to provision in time which would cause a cascading failure and result in the entire stack failing and rolling back. also cleaning up stacks that failed to roll back was a bit of a pain when there were intermingled dependencies - the dashboard was about as unhelpful as possible in determining what order to delete things so i could carry on doing what i was actually trying to do.

i chalk that up to aws resources being backed by an eventually consistent db and the fact that aws has never built dashboards that are better than "acceptable".

it's possible that i'm missing something here but i've found terraform to be orders of magnitude better on both fronts. it takes a much more defensive stance on resource provisioning with retry logic and picking up on a previous `terraform apply` which timed out instead of rolling back the entire thing. and for whatever reason, terraform is usually way faster than cloudformation for provisioning the same resources.

I've used TerraForm to string together (subsets of) AWS, Heroku and CloudFlare. Having set up quite a few greenfield projects recently, being able to re-use the same tool is great. The amount of effort required is so low now, that every project I set up has review apps - which really is a fantastic concept. TerraForm has been key.

If you only use AWS, I'm sure CloudFormation is great too.

CloudFormation is worse than Terraform in terms of tooling and simple expressions. We had to use a framework like Sceptre and write a bunch of Python functions, and Lambda functions for custom resources (and seriously, testing custom resources is not a quick turnaround thing if your function / endpoint doesn’t send a reply back with the right JSON because you have to wait about an hour for it to time out now) to get what data providers and some convenience functions are already baked into Terraform. Furthermore, AWS change sets even when using frameworks / wrappers are pretty clunky to use compared to TF style refresh && plan.

Where CloudFormation wins by design is in deployment strategies like B/G deploys, rolling updates, etc. Nothing stops one from using a combination of Terraform for everything besides applications and to use CloudFormation for app-specific deployment resources like ASGs and Target Groups.

I used TF very early on, for versions 0.3 and 0.4. It was unpleasant at best, but very functional.

I've began using it again with 0.11, and with 0.12 on the horizon it's much easier to use than it ever has been in the past. It used to be very difficult to do one-off tasks with Terraform but I've began managing things piecemeal and the community around importing everything into existing TF things are great.

The statefile borking itself from time to time is still an issue, but no more than any other operational tool of this magnitude.

The biggest downside to Terraform currently (0.11) is that you need to get extremely creative in ugly ways to accomplish some tasks. This can been seen very specifically in the AWS Security Group official TF module, which I could not live without, but would not want to write/maintain.

CloudFormation and Terraform are both excellent tools, and they're sufficiently different that I've seen folks successfully using both. The pattern I've seen and actually hope to implement at some point in my work is building a CloudFormation stack using Terraform and then yielding it to an external, specialised tool that will transactionally manage its lifecycle without all the faff of calling individual SDK methods.

I would say its definitely better for infrastructure building for anyone who does not use AWS..

I primarily deal with a weird OpenStack system but also bare metal, AWS, Azure, and whatever someone in authority decides we need to use this week. Terraform works, kinda, which is more than can be said for most of the alternatives.

If you are on AWS and don't expect to ever leave, then CloudFormation is ok. Not as elegant, but does the work well enough.

If you are not on AWS you are out of luck.

Because not everyone runs everything on AWS?

I've used CloudFormation to maintain a large-scale web-application on AWS infrastructure since 2016. I've been continuously evaluating Terraform over the years (it is indeed maturing quickly), and my team has stuck with CloudFormation for now though we're considering a potential future migration. A year ago I would have recommended CloudFormation for anyone managing an AWS-exclusive (or mostly-AWS) deployment, but today I would say both are equally good choices (and Terraform wins handily if your use-case includes _anything_ non-AWS.) In addition to being able to manage non-AWS infrastructure, here are some more arguments for TF being the better option for a new infrastructure build today:

- CloudFormation is indeed AWS-supported, but it's also closed-source, which means that you never know exactly what is happening under the hood in resource implementations. I've spent days of back-and-forth communication with AWS support (weeks in one case when working with the AutoScalingRollingUpdate UpdatePolicy) trying to clarify/confirm implementation details for various resource-implementation edge-cases that were too time-consuming and/or risky to exhaustively test myself. With Terraform, I could just read the provider implementation code directly and confidently know exactly what it's going to do.

- CloudFormation has an inconsistent and opaque development roadmap. Though CloudFormation generally has more complete coverage of AWS-service resources than Terraform's AWS provider (at least as of a year ago, I haven't done a more recent comparison), you will still encounter situations where a new service, or new API properties (reflecting new/updated features) on existing services, do not yet have a CloudFormation resource to match. Unlike the AWS SDKs which have super-quick, mostly automated release cycles, CloudFormation resource updates are developed separately and can take months or years until AWS happens to update the resource. Sure, you can deploy a Custom Resource to fill in the gaps with the SDK, but since it's community-supported, Terraform development is both much more responsive to incremental updates to evolving services and since the development is done in an open-source project it's much easier to track and know when to expect upcoming feature support.

- CloudFormation doesn't support importing existing infrastructure. This adds a big extra barrier to evolving a complex system, as it prevents you from being able to incrementally create/modify resources through the console and then 'lock down' existing resources into your state/configuration after the fact. Instead, you need to re-create new resources within the stack and then migrate any dependencies over to the new resource. This migration effort can be substantial when dealing with live production systems- I once spent several weeks carefully migrating several old production CloudFront distributions to CloudFormation-managed resources, and still incurred several minutes of unexpected downtime along the way. This effort is completely eliminated using Terraform's import feature.

Why would they if they can just provide a Terraform for Github application for a few bucks per month per team with it?

Wouldn't the fact that passing resource to modules works in 0.12 help with the mass exodus of output variables for modules?

What is time-intensive in terms of deploys? It has been my experience with tens of providers that the time loss occurs when the requests have been handed off to a providers API but not so much in Terraform.

As for Terraform itself, my biggest issue is that it has turned around the view of resources and providers.

It should have been going in the direction of https://github.com/google/go-cloud where you talk in terms of resources, and then a provider loaded would just provide the implementation.

Now I have to build Terraform modules to export resource and hide away provider specifics in the module.