Terraform is currently not reviewing community pull requests
github.com
github.com
It would be easy to say that this is a failure of open-source in some way, but to do so would be unfair to the huge amount of work that companies put into tools like this, and the stewardship that they offer, both of which take a lot of time and money. If periods of low activity while teams change are the cost the community needs to pay for that, I think that's very fair.
I stand by my statement then and now: there is nothing worse than contributing to something open source and then have your PR completely ignored.
Of course I don’t work on open source out of passion, so I could imagine this is different for the true believers.
More importantly, as you disclosed your solution, other affected by the issue can rely on your code in the meantime (= the rest of the project's life in some cases. I've been there). If it really is a critical issue that isn't solved, your fix can always be used in a different fork with better maintenance.
All in all, the upstream repo not responding to a PR isn't the end of the world I think, and the openness of the system makes it an acceptable state in many ways IMHO.
Not sure what's best practice so... just curious how people have handled this - I usually leave my forks of stuff pretty stale and focus on my own little sub-pieces to achieve what I want but not too much else.
First of all, building a provider isn't straightforward. The best way I've found to do this is to wrap `terraform init`, and have it `docker run` a build process for a plugin version that never existed - then dumping the built provider into the `.terraform` directory for the project. It's prone to failure; new users of the Terraform project complain that the build eats 8GB of RAM and takes many minutes.
Second, providers are constantly changing, and it's not always possible to cleanly rebase a set of community changes on top of master. Part of the trouble with letting PRs wither on the vine is that they themselves become stale - in one case, the code still compiles, but the end result is completely wrong.
For what it's worth: my use case was needing to use Terraform with some more "unusual" features of CloudFront and ALBs. The 80% use case for support is great. There's a remaining 15% that's well implemented by unmerged PRs, and another 5% entirely that's completely unsuppored. I've kept it IaC by using the `null_resource` provisioner to shell out to the AWS CLI where absolutely necessary.
i'm gradually coming to the conclusion that all the tools that are supposed to make provisioning cloud infrastructure easier aren't as good as a bunch of crappy custom scripts using boto or aws-cli.
Some people prefer to do the CDK thing where you use a general programming language to synthesise the IaC stuff and then run it that way, but that doesn't really fix anything because a CDK is just built on top of the same SDK. As an added insult to injury, you now don't have a domain-specific language so save you from yourself (and your team) with all the anti-patterns you now have at your disposal ;-)
I sometimes fool myself into thinking I can use Ansible in simple cases and that Terraform would be overkill but so far I've regretted those decisions every time.
What made it popular was that it was easiest to start with, but that's pretty much all of its strength compared to the competition.
I use it as an orchestrator / clusterssh replacement, but for configuration management it makes me nervous because I can't trust it to just not break for stupid reasons.
I suspect what was happening is that Amazon Linux on start run yum to apply all available updates, and ansible was not respect yum locks, which is very surprising given that Ansible came from Red Hat so they should have known how important locking is for yum.
I ended up using salt (which has its own set of issues) and never looked back.
There's other ways to sideload providers on that docs page too
Locally, I just make the change and check it in to my project.
Here's an example:
https://github.com/xwpongithub/vue-range-slider/issues/3#iss...
Contributing to a project doesn't mean just slinging code and calling it good. Communicate--talk to the people maintaining the code. Ask them, hey I have an idea and want to add this feature/fix this bug/etc.--do you have bandwidth for that change? Mailing lists, discussion forums, chat rooms, e-mails, etc. are the place to sort this out, not snarky or even angrier and angrier replies to an unsolicited pull request that goes unreviewed.
But I still think it's basic decency to close the PR and tell them No. Or if you plan to leave it around make a comment to that effect.
Something like that?
Cathedral -> centralized control Bazaar -> decentralized control
https://opensource.stackexchange.com/questions/511/whats-the...
Why does GitHub persist? It's not like forcing the feature enabled helps anyone. The maintainers will still not merge if they don't want to. There are bots in the marketplace that will close PRs with a message. GitLab has a toggle. Why is this so important to GitHub?
I think if it is made clear "don't hold your breath" and someone decides to contribute anyway, well, it's only fair to wait.
In the same way that the maintainers don't have an obligation to review my PR, I don't have an obligation to go find them and learn how to use IRC/bugzilla/mattermost/mailing lists/smoke signals/yodelling to communicate with them with the exact secret handshake to get a review. I can just throw a PR out there, point our code at my fork, rebase it whenever they make changes, and otherwise ignore their requirements until they either fix the bug themselves or merge my code.
I am not saying projects must merge in pull requests. I am saying they should not have "please contribute it is welcome" messages in their readmes.
And on second plan, there are those campaigns to make people contribute. And shaming of companies/people for freeloading if they dont contribute - especially here on HN. Again, if then people conclude that contribution is something expected, it is not only their own fault.
Really? Nothing worse?
Perhaps this is apples and oranges, but I'm most interested in open source projects that are governed by a foundation.
Also, foundation != community.
It's distressing personally though, that it's an issue unknown to me before I read this post, and affects a product I use and champion.
I have seen far more commercial, closed source products go through similar staffing crunches. The difference is that the problems are hidden away behind misdirecting sales teams and so on.
I can't tell you how many times I've reached out to someone on the inside of a company to get a straight answer as to whether a product is being properly staffed and supported. Or, conversely, how many times I myself have had to decide to orphan some commercial, customer facing work to meet a goal with a higher priority.
In my experience, useful open source products are less likely to suffer from inadequate staffing than closed.
Phinze was a large part of feeling like my contributions were welcomed, even just as a community member. A few months back I encountered something of Paul's, noticed his GitHub work had tapered off and found the blog posts entitled "diagnosis" and "treatment" on his blog.
I know nothing more about current events pertaining to Hashicorp or Terraform, but seeing some negativity here coupled with the experience of encountering that sad news led me to comment.
That team has made world class tooling, with the community. The people I know who work there are some of the sharpest and kindest people I've ever met. I'm willing to give them the benefit of the doubt and I hope you yourself will as well before presuming anything that leads to more negativity.
It's obviously not ideal, but perhaps there are legitimate reasons and at least they were transparent.
I have a lot of respect for the team personally (as in myself), though I do recognise what you say completely too. I just think there's a lot of focus on trying to do things right, and long-term-maintainably, that it often presents with a poor outlook or like there's a lack of interest.
@apparentlymart from OP in particular - I have no idea who he is in Hashicorp hierarchy, I just recognise him from GitHub - in particular is really excellent in responding to good issues and adding information along the lines of 'yes we want this but unfortunately XYZ so we're hoping PQR is going to make this easier but first we need to ABC so yes but sorry not a priority right now' sort of thing.
I don’t think it’s conceivable that any reasonable number of employees could satisfy the demand of maintaining the first-party provider set in the current form, without leveraging the community. However, I for one will not sign a CLA that allows proprietary relicensing, and I’d guess most people who could give meaningful reviews are in a similar boat, or already work on the provider teams.
However, I’m also not sure that there is a “priority problem” as such - most providers don’t make HashiCorp money from consumers or contributors, and employee time is better spent on products which contribute to a positive bottom line.
The Terraform Provider Registry has made it much more palatable to run a fork of any given provider than it was previously - I’d recommend doing so if you have functionality you need that hasn’t been integrated.
Why the objection?
For it to have a material effect, both (a) Hashicorp would need to take Terradorm proprietary within a few years while your use actively need updates, and (b) there would have to be no one else maintaining a fork based on the existing MPL2[3] code.
They say[2] why they need a CLA, which doesn’t seem deceptive.
From the CLA[1] “you reserve all right, title, and interest in and to Your Contributions”, which is unlike the FSF which demands copyright assignment[4] “Put simply, this is the legal transfer of copyright on a program from the developers to the Free Software Foundation.”[5].
[1] https://www.hashicorp.com/cla
[2] https://www.hashicorp.com/blog/introducing-a-cla
[3] https://github.com/hashicorp/terraform/blob/main/LICENSE
[4] https://www.gnu.org/licenses/why-assign.en.html
[5] https://www.fsf.org/bulletin/2014/spring/copyright-assignmen...
I would also not assign copyright to the FSF.
https://registry.terraform.io/search/modules?namespace=cloud...
https://github.com/orgs/cloudposse/repositories?q=aws+terraf...
With Terraform (and its AWS provider) I've found things are generally implemented consistently, I've not encountered any crashing bugs yet, and all of the manual steps I've had to do with the AWS CLI have Terraform equivalents.
We dogfood both the SDK and CloudFormation internally, we just deal with its numerous gripes much the same way you would externally (although we can also contact service teams directly if needed).
I say alleged because who knows if that's the code that's really running. And related to this current thread even if it is the code, with this being Amazon who knows if I ever found a bug and PR-ed it that it wouldn't /dev/null for 5 years before being closed by some stale-issue-bot or something
I'm a timid investor and am pretty nihilistic about tech in general. I forget the exact Charlie munger quote, but it was something about staying in your circle of competence.
If that worked sufficiently well, I'd initially mirror every repo from https://github.com/terraform-providers into that same GitHub organization and continue that exercise. IMHO the providers suffer from bitrot a lot more than formal terraform does
I think of that hypothesis as a "open source optimist" versus "open source pessimist:" are people who go out of their way to open PRs trying to improve the common good, or trying to drain the life out of maintainers?
Pieter Hintjens "Why Optimistic Merging Works Better" (2015) http://hintjens.com/blog:106
The beauty of open source is you can run your project however you want: bazzar, cathedral, or something else. If people think its a bad choice, they can fork.
So I start with them equal, but then I think understanding can be harder to ascertain from the PR than the initial investigation, or if the solution didn't follow the same lines as you might've chosen yourself.
I'm gonna say I think that's flat out wrong in most cases.
Obviously there's a grey area for trivial stuff.
For anything more than a one character code patch, there's so much more complexity that goes into a good review than most people appreciate.
Not to mention the weighing of potential maintenance costs, changelog messaging, etc., even if it may just be a tiny tweak or small parameter change.
The opposite of this: "code is much harder to read than it is to write" is held up as a ten-commandments style law of programming.
Here's why: When you write code, you as the author know exactly what it does, so you have exactly one copy of the code in your head.
But as you read code, you repeatedly run into "forks", where you encounter something you aren't sure of the meaning of. Even at a very small rate, like understanding 95% of what you're reading, and being unsure about 5%, it adds up. At every one of these points, you create multiple hypotheses of what the program actually does. Each one of these hypotheses is a full "copy" of the program, running in your head. Frequently to __really__ read code, you have to rig it up and test these hypotheses to keep the mental burden low (since directly testing it and confirming one of them collapses/nullifies all the other ones). (This is a huge reason why software that can be inspected live (lisp, javascript, etc) has a fairly high value, and why companies like MS have built fancy IDEs to enable the same thing with compiled software like C++, C#, etc. Past a certain point, you need to poke it with an inspector to test what parts of it do, in order to "read" the code.)
If you just "read code" and think you know what the program actually does — specifically by skimming over those parts where it's like "yeah, I'm not sure, but it probably does XYZ", it's a very juvenile, dangerous mindset. I don't have a polite way to put it, but it's in exactly the same bucket as the usual brogrammers who think their software has no security holes, for no reason other than that they trust their own work. This is where "programming as craftsmanship" breaks down; like other fields like structural engineering, it's better to build a bridge and know it will hold up because you did the actual material calculations (i.e. to not trust your own judgement, but to verify it externally). As opposed to building one, and simply having a hunch that it's sturdy enough to hold for no reason other than that you've built a lot of stuff, and your gut says it's solid.
If you're the maintainer, then you already have knowledge of how the system works. The PR just has to fit into your mental map of how things should be.
The main reason to accept community PRs is because it helps you get passionate users, not because they're free labor.
But, isn't that a strange comment to make on a thread where there was an announcement "sorry, we don't have bandwidth to even look at any problems that aren't on some PM's roadmap"
To tug on that a little more, community PRs (and issues, but I'm focused on the folks who want something to work bad enough to actually contribute a fix) are far more likely to be some edge case that a real user has stepped on which the core project either didn't consider, didn't test, or thinks "who would use the spacebar to heat their computer?"
One can get passionate anti-users, too, if they have their PRs thrown in the trash
On top of that, a lot of developers tend not to enjoy reviewing and massaging community PRs all day. They want to write code themselves, and they want it to be important code. Putting your team on review duty is a great way to make people feel like their role is low impact and unrewarding. Again, they’d rather write the code themselves.
I find it takes a lot of experience for developers to recognize the value, impact, and reach of indirect contributions like that, so it’s rare to have a team with enough people who will do a great job of reviewing, supporting, and maintaining quality community submissions. If you assign it to relatively inexperienced developers you’re likely to wind up getting a lot of things merged that shouldn’t be in a rapidly growing project that’s increasingly difficult to maintain.
It’s a hard problem to solve. But again, this is just my experience.
Open source is great, but a single SPOF in one company is becoming too much of an healthy norm. If you love your software, set it free. And if you're worried about people^W companies taking it proprietary and not giving back, then use copyleft.
(More details: ARM is simply a dumb script that says "do this, do that", it doesn't interact with your cloud resources in any smart way. As one example: You can download a template for an Azure SQL instance from the Azure portal. That template will then randomly fail to execute, because it contains two "configuration" child resources of Azure SQL, which ARM will try to deploy in paralell, but oh, they touch the same parent sources, so they crash with an error...
Then try to add depends_on in order to have it, perhaps, not fail.
Or how about the Azure SQL feature where, if you block Azure SQL to only AD logins, you'd have to declare an administrator password on first ARM run (resource creation), then remove it on subsequent ARM runs (resource update) because having it is then prevented...
These aren't minor issue or glitches; it's symptomatic of a system that's just built in the wrong way from ground up. ARM isn't a stateless description of your resources, it's an awkward scripting language (although since the APIs it interacts with are idempotent the difference isn't obvious at first).
How Microsoft could decide to have Bicep compile to ARM is beyond me -- Azure has some good features (refer to resources by name rather than allocated ID) that could greatly simplify the approach that Terraform/Pulumi takes if they only targeted Azure -- why didn't Microsoft do that instead, give us Terraform/Pulumi for Azure without having to keep a statefile (especially a statefile with secrets in it).
I’m not sure exactly what you’re looking for with regards to referencing by name rather than ID - names must be qualified with a resource group and so forth, and to actually unambiguously identify things, you’d need all components of the ID.
Bicep looks like a solution in search of a problem to me, though fortunately I haven’t had to spent more than about 5 minutes looking at it.
I said that referencing by name is one of the few things that are good about Azure. It means one does not have to persist a lot of resource IDs when deploying infra, one can just query the cloud state.
What I am looking for -- or what I would love if someone built for me -- is utilizing that feature to deliver Terraform or Pulumi without the need for a statefile. The statefile should not be needed on Azure due to the resource naming scheme -- just query the management APIs to get currrent state.
When we looked at Terraform and Pulumi we saw that the tools by default sucked down evey secret in the resources into the statefile. This is behaviour I very much disagree with, we do not want secrets stored anywhere or pass through anywhere, we want to rely on service identity everywhere.
PS I am speculating here about why the statefile is needed...I have been assuming something about Amazon made it needed, but I may be wrong.
The reasoning is explained pretty well in the Bicep FAQ: https://docs.microsoft.com/en-us/azure/azure-resource-manage...
It's not the recommended devops tool for any cloud provider.
> and now they won't even accept PRs to improve and add features from the community?
Random people writing code doesn't make that code valuable or of any sound quality. They have employees whose jobs it is to improve and add features to their software.
> Clouds should be as
Clouds are and should be exactly as simple or complex as their customers request. Clouds should not be built around opinionated forum posts.
The Deployment manager templates under CFT for GCP specifically state they recommend the Terraform modules.
It is also the first provider listed under Oracle cloud. Pretty much every cloud list it as the preferred DevOps tool.
> Random people writing code doesn't make that code valuable or of any sound quality. They have employees whose jobs it is to improve and add features to their software.
It doesn't mean that it isn't. You act like only employees can write valuable code with sound quality, when in fact the opposite could be true and often is.
> Clouds are and should be exactly as simple or complex as their customers request. Clouds should not be built around opinionated forum posts.
The concept of a cloud provider is not anything unique. Hence why there are so many of them implementing their own API. Cloud providers budgets are heavily skewed to marketing, because if everyone could get the rates available through OVH, Hetzner, etc while still getting the same features through their own 3+node zero-config cluster then no one would pay the outrageous prices of AWS, Azure, etc.
So, "It's listed as the recommended tool for GCP" means "it's the recommended tool for any cloud provider" ???????????????
Both Azure and AWS maintain and promote their own resource provisioning systems. Also you just totally ignored the GCP Deployment manager templates listed on the same page. Meaning, Terraform is _an option_.
> It doesn't mean that it isn't. You act like only employees can write valuable code with sound quality, when in fact the opposite could be true and often is.
The average quality of an internal employee will be higher than the average quality of a random person submitting PRs, because you'd fire your employees if they weren't. This is common sense.
> The concept of a cloud provider is not anything unique. Hence why there are so many of them implementing their own API. Cloud providers budgets are heavily skewed to marketing, because if everyone could get the rates available through OVH, Hetzner, etc while still getting the same features through their own 3+node zero-config cluster then no one would pay the outrageous prices of AWS, Azure, etc.
The concept of a paid service isn't unique. Adding features to a paid service to increase users' engagement and/or acceptable price is normal and good. Again, common sense.
What is the recommended devops tool for azure in your opinion?
Per their documentation, this is Azure's recommended devops/resource management tool.