Hacker News Reader

Comment by grubbypaw | Hacker News Reader

I've never understood CloudFormation's lag time in supporting new service functionality.

This launched nearly 3 years ago. https://aws.amazon.com/blogs/compute/bluegreen-deployments-w...

Having spoken with core teams like IAM and Cloudformation teams at length, this appears to an internal AWS organizational issue. Those teams are not responsible for the services integration with them and so they're at the mercy of those teams priorities.

But honestly, I think the reason that Cloudformation support isn't as widespread or a top level priority is that it simply exposes the poor architecture and behavior of many of AWSs second tier services and teams. There are many services that simply do not behave well when managed by Cloudformation, but are also completely janky on their own and I'm betting it's far easier to cover up for poor architecture in the console than expose all the services dirty laundry with a Cloudformation integration.

Additionally, there are a lot of service teams that probably don't have a lot of customers using Cloudformation, so don't prioritize it or half-ass it completely. I'm looking at you DMS, and your terrible turd of a Cloudformation integration.

I'd say nearly the same thing about IAM and service teams inability to implement it well. I still do not understand why AWS has not mandated all services need to support both tag and resource based policies and predictable IAM semantics (looking at you Glue with your little fu of love called the write action "glue:GetMapping").

Cloudformation and IAM are, to me, the two of the most killer services from AWS, neither of which I've seen replicated at other providers.

Ex AWS here. I had the fun of digging into the rabbit hole of IAM and its convoluted logic. It's definitely possible to do what your said, but it's super easy to make mistake and the internal documentation is lacking. It took me multiple trips talking to people to deliver the integration we wanted.

It's also very old with some odd decisions in there - I can't go into the specifics. And it's practically impossible for the IAM team to deprecate those impossible corners

I am not surprised, being that it's one of the oldest AWS services? What I do love about IAM is that with the work that the Automated Reasoning Group is doing with Zelkova, it's really a dream to be able test IAM policies before deploying them. I really hope their work trickles back to the service teams so that they too can leverage it to see their way out of those dark corners in IAM :)

It's one of the worst products on AWS. It's so bad, that companies would rather spend engineer's time to avoid it. That's why there are hundreds of products that replicate its functionality.

Has the GUI been fixed to be somewhat useful? Did they migrate from their god awful JSON crap? Can I embed simple infrastructure logic, like automatically adding a group of nodes to a Route53 zone?

Because while "ready" for new features includes API support, it does not include CloudFormation support. The problem is entirely AWS politics and culture.

Who builds the cloudformation support? I could understand not baking it in on an initial feature release but it should be there ’soon’.

Same with Config.

I've been told that each product team is supposed to add CloudFormation support themselves. For some reason, it's not treated as a high priority and often lands on the laps of the core CloudFormation team.

It may turn out that the tools that CloudFormation team provides don't make integration easy, especially if the operation takes longer than 15 minutes (meaning they can't implement support via a single lambda invocation as an under-the-hood custom resource).

You can also invoke a custom resource via SNS and then the SNS topic is subscribed to an API endpoint.

But it’s hard to believe if the team responsible for the blue green deployment functionality developed an API endpoint to do it, they couldn’t just hand it to the CF team to call. At the end of the day that’s all CF does. Call APIs based on the different lifestyle events as far as how it actually creates resources.

>lifestyle events

Intentional or not, I'm going to start using this. :P

Ughhh. And it’s outside of the edit window...

You would have thought that product teams would love using CloudFormation to create and tear down test resources as it already has code to handle all related resources that they want to interact with (networking objects, DNS, etc)

> The problem is entirely AWS politics and culture.

Someone higher up should enforce it. I've been in teams, where if something isn't in cloudformation it doesn't exist and that attitude is totally understandable, having to do some operations by hand seriously hurts IaC efforts.

I understand the logic of not waiting for the CloudFormation team to do their work because it could introduce delays but it makes it a considerably less useful tool.

When considering Terraform vs CloudFormation I certainly took that into account and that's why I don't use CloudFormation for anything.

I wonder if there are any killer features of CloudFormation I missed when I looked into this (years ago now).

- AWS Business Support Plan

- The ability to configure everything about a lambda and export the template.

- Serverless Application Model tooling

- Quick Create Links - you can give out a link to developers that let them create infrastructure by using CF and just enter parameters. You can restrict them to only being able to create a stack that you specify.

Not so much a killer feature, but I find that some actions in newer services are only possible to automate through Cloudformation (looking at you Workspaces), because of the time they take to open up the API.

For Workspaces it took them something like 2-3 years to release to the public API what you could do with Cloudformation

You can now write your own resource providers [0], which seems similar to TF. It's pretty decent. The only issue is you can't easily migrate from a custom provider to an official one once they release it (without deleting and recreating the resource), which can be a dealbreaker.

[0] https://aws.amazon.com/about-aws/whats-new/2019/11/now-exten...

Somehow Terraform manages to roll out new features before they are even public while Amazon's own framework lags by years. I guess the CloudFormation team must be starved for resources.

I believe AWS updates their SDK first, and as Terraform roughly wraps that SDK, as soon as it is available they can update the provider

I wonder why none of the big cloud providers has bought Hashicorp.

It seems like a really obvious move to me?

I don’t have revenue breakdown on AWS orgs to back this up but I suspect CloudFormation isn’t one of the money printing products so it’s hard to justify investments as large as acquiring Hashicorp.

This is despite the obvious role of CloudFormation as a productivity multiplier inside the AWS world

That’s an interesting point, but Hashicorp does a fair bit more than just Terraform.

Why? Hashicorp provides them with tooling at no extra cost. Majority of that is free to the end user. Having Hashicorp separate is great for the cloud providers, since they don't have to manage or spend money on Terraform... but still can contribute the code.

Because they could focus extra resources on making the Terraform providers for their own products really good. They could have really tight, quick communication channels between the teams building their features and the Terraform team. Much harder to do when you're in separate companies.

Of course, AWS could be doing the same for Cloudwatch, but apparently they're not bothered.

I think it's a difference in prioritization and how CloudFormation is treated as a separate product as opposed to a fundamental part of releasing features. The other two major cloud providers treat templates as a core part of the release process (now).

As I understand it, the three cloud providers differ in this way:

1. Microsoft Azure releases template/API first - no features can be released unless an Azure Resource Manager template can access it. I think they use ARM templates for their integration testing, as well, but I'm not sure. (If you're an Azure 'softie, please chime in!).

2. Google Cloud Platform releases APIs first - no features are released without extensively being tested via API integrations. They do have a CloudFormation/Azure RM template offering now called Deployment Manager so that my change. (This means that Terraform-like tools can target features quickly, and I have no experience with Deployment Manager but I suspect it does as well.)

3. Amazon AWS releases to the web console first. You might say, "Whoa, what about Bezos' service oriented architecture email?"[1] As far as I know, that's true for internal features. But all user features are exposed through the web console, then wrappers around those APIs are exposed, then, sometimes years later, CloudFormation gains the ability to target those APIs.

For this reason, I trust Microsoft and Google Cloud a lot more to maintain parity between what I can do in Terraform/Pulumi/native template tools and in their web consoles.

[1] - https://gist.github.com/chitchcock/1281611, per Bezos: "All teams will henceforth expose their data and functionality through service interfaces."

Reply on news.ycombinator.com