Hacker News Reader

Honest question. If you need to pump data into datadog from your cloud, why not just stop doing that, and use the cloud tools? Datadog isnt' creating metrics. The metrics are already on your cloud provider (gcp,aws,etc) -- AND all these providers have ways to view metrics. Why do people pay 20k per month for data dog? They dont' want to use GCP metrics explorer? Same with TF cloud. You can just setup a jenkins job and do all the features TF cloud has in a few hours. Why pay TF cloud? Are people confused? I must be missing something. I worked at startups that thought all metrics need to be in datadog, but they didnt' understand they can just use metrics explorer. You can't even use sql to create graphs in datadog, it's awful.

IMHO If you are serious, dump the metrics into bigquery/redshift and start doing sql like an adult.

I would not consider metrics explorer to be a particularly good product. For queries where the query builder isn't good enough and you write PromQL instead, they don't even let you alias the legend; instead you must see the entire query as the label for that line.

A pretty minor nitpick, but indicative of the level of attention they give the product.

If you start pulling all the metrics we currently use in DataDog out of something like BQ/RedShift you're going to spend WAY more in terms of engineering hours and infrastructure than we're currently paying DataDog.

I keep hearing something about a company that switched their monitoring from DataDog to DataBricks and I can see that yes you probably could go build a monitoring solution on top of a datastore like that. But I certainly wouldn't want to.

For viewing, cloudwatch is simply nowhere as convenient, writing whole custom queries takes a lot of time, and setting up altering for just the right thing sometimes requires you to copy the complex result as another metric. Basically I can create a time-shifted difference of two metrics reporting at different intervals, smooth it and make it alert the right teams through slack and pagerduty in a minute or so (while getting a visual feedback on the query the whole time) - this would take a significant amount of time to do through the plain AWS process.

It's a bit like "you can write everything you want in assembler without the overhead of extra layers of X". Yeah, sure, I can. But I appreciate the extra layers of X. I'll take the right cost/convenience balance, like an adult, thank you.

At a prior company, we had a terraform slack channel, where people would post lock and unlock emojis and apply terraform manually.

Obviously, it's not a perfect system, and it doesn't indefinitely scale, but it worked well enough.

We use Atlantis [0] for CI/CD automation of Terraform pull requests to a centralized repository. It's pretty good too, especially for a self-hosted solution. I can't see how Terraform Cloud's costs would be justifiable for us without a custom contract.

[0] https://www.runatlantis.io/

At my last gig we built a bespoke version of this on top of our CI provider. Now i'm a technical co-founder in a startup and this is something I haven't quite solved yet, and this looks like exactly what we're after. Thanks!

You can setup remote backends which support locking (e.g. Azure storage, Amazon S3, etc..) that way it's automatic.

You can see a list on the left-hand side here: https://developer.hashicorp.com/terraform/language/settings/...

StackOverflow famously manages database migrations this way; you post a message in a shared channel and get the next revision number.

How does this work? I thought the terraform state file was the single source of truth - if people are applying terraform 'manually' I assume that means on their local device? Are people sharing around the state file but don't have a central location for a lock file? Apologises if this seems obvious...

One can just commit the lock file to Git or use any backend (like S3, HTTP, ...) to share the lock file.

My assumption is that they're still using a remote backend for state, but they haven't set it up to use the locking features of the backend.

For example, I've use S3 as my Terraform backend for years, but I've never bothered to set up the locking feature, which uses DynamoDB.

In a small team that deploys Terraform changes rarely, you may never encounter the problems solved by using locking. Maybe good communication and a Slack channel works well enough for you.

As others said, we checked the state file into git. It was up to you to ensure you pulled after taking a lock.

I think Terraform Cloud (and most Hashi's enterprise offerings) aim at absolutel behemoths of deployments with many, many infra teams where the complexity comes from scale and the companies are not Google or Facebook and therefore this problem can't be solved through talent. For many such enterprises it's easier to throw money at the problem.

Then they should price them with smaller margins and make it up with volume.

Although it's certainly not always the right strategy, it's certainly a valid strategy to serve a smaller number of relatively price-insensitive customers (and provide them with ancillary services/high-touch sales/etc. that they want). The fact that you may not personally be on the market for that sort of thing doesn't mean there isn't a market.

In particular think major regional non-technical companies, like grocery store chains or logistics companies or hospital networks. Beyond basic help desk support none of their IT is in-house and they don't want it to be in-house any more than an IT company wants to run an MRI machine for its employees, so they have dedicated a block of money to making that problem go away and don't really care how much it all costs as long as it comes in under that line.

You would think that, but having just went through trying to implement Terraform Cloud at work, its tools for doing anything that approaches behemoth deployment are abysmal. Permissions management is a nightmare and any sort of orchestration between different layers is barely there.

Except those same companies then run into hidden limits that exist on Terraform Cloud and hadn't been previously mentioned. Honestly, even bigger companies are better off going with Spacelift as it can scale extremely well, has reasonable support, and is much more feature rich.

As a disclaiming I'm currently writing a book on Terraform, and I've been interacting with a lot of the people in the space. Up until a few years ago I was a huge fan of Hashicorp, but their price changes and lack of support were what made it so I couldn't recommend Terraform Cloud anymore. The license changes they made were the icing on that cake.

I personally like what render.com is doing with IaC through their blueprints format. It's definitely a kid's toy version of IaC, but it's a really nice step up from Heroku.

It appears to be proprietary.

Yes. Still a great tool for small dev shops

Why would you pick a proprietary solution for a small dev shop?

Less people available to own self-hosted solutions.

Proprietary also can mean a well-funded product with money allocated to DevEx and ease-of-use.

> Less people available to own self-hosted solutions.

There are other ways that are not proprietary and don't require self-hosting.

> Proprietary also can mean a well-funded product with money allocated to DevEx and ease-of-use.

How did that work out for Heroku?

I wouldn’t lay blame for Heroku’s financial problems at the feet of architecture choices it made customers adopt.

Also, Heroku’s buildpacks were anything but proprietary. They’ve been adopted by a bunch of other products.

What financial problems? They were bought out by another company and tanked. This is the exact reasons why you don't want to tie yourself to proprietary solutions.

Sure, the concept of buildpacks were copied to other platforms (I personally worked on building one of those platforms, Cloud Foundry), but they were a proprietary solution that others adopted for ease of transition off of Heroku, not because it was some great solution to IaC.

This is a bizarrely combative way to discuss this. Can you please instead tell me what you think I should be doing?

Why would I care about proprietary/opensource if it's affordable and saves time as a small dev shop? I just wanna ship and the company might be dead before any of this matters.

It is mission critical, just not the SaaS version.