OpenTF repository is now public
github.com
github.com
As many of you asked for, we've finally made the repository public and will continue developing it in public from now on.
This took us some time, but you can now read our announcement for more details[0].
Thanks everybody for the support so far, and I welcome you all to interact with the repo, join the discussion, or even contribute.
A possibly noteworthy detail as it's been discussed on HN here a bunch, we've settled with the DCO[1] for contributions.
Also, happy to answer any questions!
[1]: https://developercertificate.org
Disclaimer: Work at Spacelift, and currently temporary Technical Lead of the OpenTF Project, until it's committee-steered.
There will always be someone who will gleefully jump down your throat for imperfections.
But on balance it's probably better to work in public anyway and ignore the haters (while also continuously improving processes).
If they had worked on this publicly from the start I'm sure a lot of the current PRs would have generally been of better quality as they could get community input earlier.
A project being run by professional developers who have a lot of experience working with the Terraform code should be capable of doing this.
If your problem with a project is that it advertised itself to its target audience prior to releasing a repository, then a good faith comment would say something like, "My problem with this project is that they have spent more time writing manifestos and blog posts and social media comments so far than they have spent developing strong processes and working on publishing a repository for the project".
But if that's your real problem with a project, then a bad faith comment might look like "This is a bad pull request, how could you let it get merged?". The reason that is in bad faith is that it says nothing about your actual problem with the project, it's just a totally random swipe that you don't even actually care about. If there turns out to be a good answer to the question, like "Oh you're right, we forgot to require squashing pull requests when merging, I've fixed it now!", then instead of recognizing that your complaint has been spoken to, you are likely to simply find further things to criticize, because the first thing wasn't actually your real criticism, it was just a smokescreen.
Now, if you are making this other, better, point that the OpenTF project should have more mature software development practices, at least equivalent to and ideally exceeding the better-known project they have forked from, then yep, I agree with that criticism and do not think it is necessarily being made in bad faith.
However, I would be significantly more sympathetic to that criticism if it were a comment on an article entitled "OpenTF project celebrates one full year since its conception" rather than one entitled "OpenTF repository is now public".
The project is brand new, and they clearly rushed to get out a public repository because of other haters who were giving them crap about taking too long to publish the repository, and their processes clearly haven't matured yet. (Or I dunno, maybe it was even many of the same haters, because again, this is the tendency with people who criticize in bad faith, to just move on to the next random criticism that they don't actually care about.)
So if we get to a year from now and their processes remain immature, then yep, I'm right there with you on your criticism. But I think it's crazy (and, sorry to keep repeating myself: likely in bad faith) to make this criticism of such a new project. There is absolutely no indication that the developers running the project are incapable of having mature processes for the project, from the tiny amount of data available from the tiny amount of time that has elapsed since they announced the project.
I think there's a stronger argument of an X/Y problem in the statement, since they talk about what they see as output vs what they expect to see instead.
And frankly, I am disappointed too, as these people working on this project should have spent all that time understanding how Hashicorp actually handled the codebase and ensured that their own engineers followed the same quality level for development. They're all definitely capable of it, so they should actually do it.
Otherwise it's going to be hard to trust the fork to succeed.
(I do think you may be missing that I was responding to the entire sequence of comments, which started with just a drive-by swipe at pull request merging process, not just the later comments expressing disappointment with the perceived emphasis on marketing over engineering.)
I think your direct expression of disappointment is totally reasonable. I personally think it's also super premature to be disappointed, but it's your prerogative.
But I just think the other commenter's approach to expressing this criticism was way off base.
But tonnes of people don't care about git hygiene or using tools well in general, GP's in for an exhausting time caring about it much in projects they're not in control of.
Someone is going to hit this with `git bisect` and it wont be their lucky day.
Or if not: you're welcome!
I've made sure via repo config that only squash commits are enabled from now on, so this will not happen again. Thanks for the feedback!
Figuring out bugs with `git bisect` is not going to be a fun endeavour for people trying to understand incompatible changes.
Branch protection doesn't allow merging without a passing test-suite.
> The squash merge is not going to solve the lack of proper commit messages
Could you expand? You choose a sensible commit message on squash, while the PR's commits become fairly irrelevant at that point.
https://github.com/opentffoundation/opentf/pull/243
EDIT: and just to point out. If you have 1 PR with 19 commits that break the test suite. The last commit fixing it doesn't matter as you will be hitting one of those 19 commits at some point during a bisect.
>Could you expand? You choose a sensible commit message on squash, while the PR's commits become fairly irrelevant at that point.
It's optional. Nothing prevents you from just adopting whatever the PR said initially. Turning it on doesn't automatically make it better.
But not if you merge the PR as a squash-merge, which turns those 19 "development" commits into a single "permanent" commit.
Using PRs as the unit of development, with all intra-PR work squashed into a single atomic test-passing commit, is a well functioning process that many teams use.
The test suite did its job by alerting you to make changes before merging your work to the trunk branch, I don't see this as anything anybody could possibly pick a fight over.
And to be clear, I still use mainline terraform because I never really wanted a wrapper around terraform. It's just silly to me to take issue with the terraform wrappers for this non-issue.
Imagine you are a terraform provider developer that is working on making sure their code is working with `opentf`.
Lets imagine opentf does an initial `v1.0.0` release of their code and your provider doesn't work. But you know it worked with the last FOSS release of `terraform`.
What do you do?
You find the common ancestor between these two projects, lets say 8a085b427b74ce3829500a59508b77465f1bbef0 (as that is the last commit `opentf` has from `terraform`).
You will now run `git bisect` on the history between `8a085b427b74ce3829500a59508b77465f1bbef0` and `v1.0.0`.
You will do a binary search on the 100-200 commits here, and everytime the source fails to build, or the test suite doesn't pass for whatever reason, you are making it much harder for the downstream provider to figure out why their code doesn't work.
You can easily just try do this today and see what happens. Does the current untidy git history cause you any problems?
I really recommend adopting this strategy.
Don't forget, a project like Terraform is too big. No one person can know how the whole system works. Trying to look through PRs as a means of debugging is a fools errand. You have to take on a different mindset when working on these large codebases.
Then I'll `git bisect --first-parent`, and either isolate it to your merge, or mark to merge OK. `git bisect` doesn't have to walk your branch.
The "real" solution is making the developers aware of the issue and cleanup up their history before doing an MR.
For a start, it's easy to review, and easier to disect if something breaks.
When messy git history is provided (i.e. a history of the developer fixing their own code that they missed), squashing can be a reasonable fallback. But it's never the best option, IMHO.
It's a separate project from git [0].
And the benefit is that instead of humans spending time deciding whether or not to squash commits in some branch before merging, or being asked to do so by a maintainer, a computer just does it automatically. It's a lot like automatic code formatting; it's a big win to have tools do things automatically, rather than requiring human time and judgement and communication.
They're not independent, they're related. Having multiple levels of "namespace" - repo -> PR -> commit - makes things easier to navigate, just like having multiple levels of folder in your filesystem.
> And the benefit is that instead of humans spending time deciding whether or not to squash commits in some branch before merging, or being asked to do so by a maintainer, a computer just does it automatically.
If you demand that each PR must be a single commit you just force the human to make that decision earlier and more manually.
> If you demand that each PR must be a single commit you just force the human to make that decision earlier and more manually.
How so? I think you may be misunderstanding the suggestion... The pattern is that people can have whatever commits they want on their own branch, but when that branch gets merged into the trunk, all those commits are automatically squashed into a single one. I don't see how that is "forc[ing] the human to make that decision earlier and more manually".
Right, so that means people have to decide whether something's going to be a separate commit immediately when they're working on it, and start a new branch if so. Or they could mess around splitting their branch into smaller branches at the end when they're ready to PR, but that's fiddlier than rebasing to split/merge commits.
I don't know what you mean by making PRs being "fiddler than rebasing to split/merge commits". What is the fiddliness you see?
That's a horrible way to work because when you squash a branch it breaks the history of any other branch that's based on it. So you'd have to do a fiddly rebase when you merge your first branch in.
> I don't know what you mean by making PRs being "fiddler than rebasing to split/merge commits". What is the fiddliness you see?
Imagine I've done a bunch of work in a branch, and then decide I want the contents of that branch to become three commits on master. With your workflow I have to split my branch into three branches, retroactively. That's fiddlier than just turning the commits on my branch into three commits.
It really isn't. Branches are just as cheap as commits. It's super easy to create three branches pointing at three arbitrary commits.
The community was well aware that once you tag a license to a version, you can't undo that. And they're well aware that they can fork from that license forwards and build their own "new" project, version by version, that stays open source.
This is going to be fascinating to see play out, and I think a case study in software licensing going forwards...can't wait to see how OpenTF does down the road.
Copyrights are never not tied to a "version." There is no concept of "version."
Copyright applies to a bunch of data. You have a copyright on a song. If you create a second song you can call it a new "version" of the original or not, none of this matters for copyright.
Well, some of it matters:
"If you pay close attention, you can hear it: there’s a new lushness in the opening banjo twangs, and an extra beat when she sings the lyric “Just say yes.” But the difference between the 2008 version of Taylor Swift’s “Love Story,” which helped propel Swift to pop stardom, and the 2021 rerelease of that same song is pretty subtle..."
"Her hope, it seems, is to override those archival works with these new versions..."
"What’s truly different about Swift’s “new” work is the intention behind it, and developments that have brought her to the place to own it."
https://time.com/5949979/why-taylor-swift-is-rerecording-old...
See also:
"In an interview with CBS’s Tracy Smith, Swift said that she planned to sidestep Braun by rerecording everything in the songbook that he now owns, meaning all the songs she had released prior to her August 2019 album Lover."
"When Swift switched over to Republic Records in the fall of 2018, she negotiated to own the master rights to all the music she creates going forward."
"By rerecording her old songbook with Republic Records, she will own the copyright to all of the new recordings."
https://www.vox.com/culture/22278732/taylor-swift-re-recordi...
TL;DR: Arguably, re-recording and re-publishing is just a new pull request and commit, but that gets you a new copyright.
But technically, it's the performance that has these financial rights, so perhaps this is more like a clean-room re-implementation so you can re-license the binary. :-)
Oracle almost always seems to be involved in these things, but surprisingly not with Terraform :D
Interesting that having "TF" in the name could cause issues.
Source: https://github.com/opentffoundation/opentf/issues/273#issuec...
My second guess is that using "Open" together with something that implies "Terraform" is also a problem, as Hashicorp might sue them for defamation, as that name would imply that Terraform is not "Open". I'm not personally against that, I wouldn't consider Terraform "Open" anymore, but I'm not sure a court would see it the same way.
Given a product TerraForm, known also as TF, how likely is it that a consumer would be confused into thinking "OpenTF" is made by the same people who make TF?
That's the bar to clear. In the past, some trademark owners have been fairly lax, allowing unrelated "Open" versions of their software to be established. The classic case was "SUN OpenOffice", but I suspect they got away with it because "office" was not a wordmark - the trademark was for "Microsoft Office", which is very different. In a lot of cases, the "Open" versions emerged once the software was not sold anymore, so it was impossible to be confused.
In this case, though, the software is still sold and likely carries trademarks and wordmarks. Unless OpenTF's pockets are deep enough for some serious lawyerin', it will be easier to rebrand.
I'm well aware of that, I'm mentioning two potential issues, one being the "confusion" part and the other being the "defamation" part.
On the other hand, Terror Fort...
Read https://en.wikipedia.org/wiki/Wordmark for more details
(disclaimer - env0 founder here, co-lead the OpenTF initiative)
AWS created OpenSearch, not OpenES.
"planet change" being an alternative to "terra form"
1: https://en.wikipedia.org/wiki/Network_Information_Service
The first is, please provide a standalone registry package for both modules and providers - the only I'm aware of is Artifactory, and I don't really feel like running another big repository software in a Nexus shop.
The second is related: please allow for easier forking of provider modules. The current workflow of either building locally and distributing binaries with collaborators that are manually copied sucks or waiting for upstream to accept PRs, particularly if upstream wants CLAs signed.
> The first is, please provide a standalone registry package for both modules and providers - the only I'm aware of is Artifactory, and I don't really feel like running another big repository software in a Nexus shop.
Could you expand on this? Do you mean you'd like a self-contained binary to run a private provider/modules registry? If that's the case, then there are some open source projects for that, and we've done a proof of concept of an approach in which you can distribute providers via OCI registries like DockerHub or the GitHub Container Registry[0] as those are actually perfect for the use-case.
This PoC will be followed by a public RFC soon.
> The second is related: please allow for easier forking of provider modules. The current workflow of either building locally and distributing binaries with collaborators that are manually copied sucks or waiting for upstream to accept PRs, particularly if upstream wants CLAs signed.
Do you possibly have an ideal workflow in mind?
[0]: https://twitter.com/opentforg/status/1696913055576387599
Disclaimer: Work at Spacelift, and currently temporary Technical Lead of the OpenTF Project, until it's committee-steered.
Yes, indeed. But it seems the problem more was something with Google's algorithm, I swear a month ago this here [1] wasn't on the first page when searching for "terraform private provider registry".
That also answered the second question.
Disclaimer: I worked for TIER before
It's sort of a disclosure (which is almost an antonym) but you're not acknowledging a potential bias, you're saying this is why you should listen to me/care to reply, so even that doesn't really fit IMO.
Disclaimer: Work at Spacelift, and currently temporary Technical Lead of the OpenTF Project, until it's committee-steered.
I guess I don't really understand what had to be changed given the new license debacle/changes.
Open Technology Foundation is one suggestion.
Backronyms are a thing people.
I'd personally love to see an open source path forward on other products that Hashicorp re-licensed like Packer, but those will need to be a separate project and initiative.
Cloud Native Computing Foundation
That is why OpenTF is on its way to CNCF. To ensure it stays OSS forever.
There is a difference between "true OSS" like K8s, OPA, etc and "temporary OSS" (backed by a company) like what Terraform used to be, Pulumi, GitLab ,etc. Those can be changed in the future.
When developers chose OSS, they should consider if it is a CNCF OSS or a vendor backed OSS. What Hashi did is an important example.
(disclaimer - env0 founder here, co-lead the OpenTF initiative)
https://www.pulumi.com/blog/pulumi-hearts-opensource/
Disclaimer: the following is my own opinion as an engineer at Pulumi.
Pulumi is true open source, with a relationship similar to git and the many SaaS services that layer on top of git to provide meaningful value.
To contrast with our competitor, Pulumi relies on open source languages and protocols. We could not, even if we wanted to, change the Python license. Nor could we change our protocols without breaking our users and our growing ecosystem.
That's the value of building on open protocols and standard languages.
But our philosophy at OpenTF is that users would rather participate in an open ecosystem where multiple vendors compete for their business. If you're not happy with one vendor, you can easily switch to another; competition works to make all vendors better.
When we look back at this comment a year from now, I'll wonder how your company will feel about the responsiveness and new features they're getting from Terraform Cloud when the primary incentive to stay is not because you think it's the best product available, but because switching costs are so painful.
Eventually they saw the writing on the wall and moved to Linux systems and replaced all of the hardware, and have an enterprise RHEL subscription that we could call when needed.
I think the story is going to be the same with the current company, mainly "who can we blame if there is a security incident, or get me a Hashicorp person on the phone if we have some kind of Terraform related production issue." Having this in place seems to matter more than everything else honestly.
I would like to see more companies that leverage open-source contribute back to the very community that enables their value creation.
I have come around to the idea that it's good to firmly discourage this kind of late-in-the-game license change.
But I also think that, on net, this episode will lead to fewer businesses choosing the open source model for their software, from the start. It just seems like playing business on hard mode to try to build an open source or source available product, when you can just build a SaaS and charge for it. I think this is really bad (I have a strong preference to not be stuck with opaque SaaSes for most things), and I'm not sure what incentive there is to try to find new business models, when you risk becoming public enemy number one amongst a big chunk of your potential customer base.
Not sure what you mean.
Just see the latest serde "drama", where everyone disregarded the "no warranty" clause in the licence and loudly demanded changes/wanted to fork the project/etc.
FOSS has a huge problem of expectations from both upstream and downstream. There are very common arguments about "I use this, you broke it/made changes I didn't like, so you are a horrible maintainer and person and you will have zero credibility forever". If anyone uses FOSS dependencies, they also accept the risk of future versions being different. No one breaks versions already released, this is always about future versions. Demanding the maintainers to make specific changes/not to make them for your usecase is extremely entitled.
Well that's the core problem - what is the product here. Terraform Cloud and Terraform Enterprise are products for sure. They're not open source, though. Is Terraform a product? Well, it doesn't do anything on its own, it requires plugins ("providers") for anything it does. Plugins are developed by or at least with third parties. It's a gatekeeper of an ecosystem that at this point is a common good.
Whether the ecosystem would exist without the permissive license and external contributions - really hard to say. But if your main play is to foster the growth of an ecosystem and then turn it into a product exclusive to your business, then I guess you should look for alternatives.
(Marcin from OpenTF, private opinion)
While I guess that for many vendors they don't care whether it's open or not. For instance AWS. I doubt they truly care about it being open or not.
Even Terraform core remains source available and so 'community' users can still take a look at the source code and identify/report/fix errors.
I was just responding to a comment on how AWS didn't care if the provider was open. IMO there's value for them in it being open.
One would think that the companies thought this through before publishing FOSS code, but seemingly there is a lot that didn't do that.
But unfortunately I think the lesson they will take from this instead is "we should just build a SaaS with no source availability because that's way easier and source-available just makes people mad anyway".
I think that's a shame.
I think that's the general attitude the software companies will have going in the future. Why even bother dealing with negative PR and push back against their ability to make money by going with FOSS? In hindsight, TF should have been released with BSL from the beginning.
I am not a huge fan of Hashicorp changing its licenses for future releases but I am also skeptical of OpenTF's motives since their members have big financial stake in that decision.
Acknowledged, there are always self-serving motives involved. Generally things don't happen without a reason. But we donated the project to a foundation, and will over time build (and fund) a dedicated independent team who will follow their own vision and the community needs, not ours. So please judge us by our actions, not assumed motives.
> their members have big financial stake in that decision
I can't speak on behalf of others but to us at Spacelift it's less about direct financials (we are actually not directly affected by the license change!) and more about being in charge of our own destiny and product roadmap.
It doesn’t mean “everyone does our work for free and then we keep the profit”.
I'd also love to know how much Hashicorp chips in to maintain the projects they build upon. For example, I'd bet the vast majority of Terraform usage is on Linux. Do they support Linux development? Do they support Go language development? It seems like the companies complaining about "leeches" (eyeroll) aren't the ones actually paying people to work on upstream FOSS projects.
At least if the "something else" isn't free software.
I think its the ethical side, rather than the legal side, thats more complicated...the contributions from the community contributed to the Terraform "brand" that got bigger and bigger, and now Terraform is attempting to secure their monopoly on capitalization of the brand when previously there was an implicit understanding that the "brand" was open-source.
However, you might also argue that there was an implicit understanding on Hashicorps side that the community wouldn't build directly competitive projects when they held the lions share of the funding on the contribution/maintenance side...
I think the whole thing is pretty complicated - is Hashicorp leeching off of the contributors or are the competitive contributors leeching off of Hashicorp? Honestly I see both sides.
The beautiful thing is that its totally legal and acceptable for OpenTF to do their own thing and continue Terraform under their own terms...so either way we get to see this play out :)
I'm coming down strongly on the side of the users, though. Hashicorp chose the original license, and the one they picked is perfectly fine with the idea of someone else building off it. I mean, it was written by the Mozilla folks. They want people to build off their projects and make a nicer Internet!
Hashicorp could've used a different license if they wanted to. They deliberately chose one that gives users the rights to build on Hashicorp's work -- and yes, even to profit off it at a competitor. What I don't think HC has is the right to act shocked when others use the software under the terms they were allowed to use it.
Surely, if maintaining it was such a burden, Hashicorp could just stop maintaining it?
Also, when FOSS was created, these concerns were simply non-existing. Times have changed massively in 30 years, "leeching" like this simply didn't exist when those licences were drafted. So no, I don't lack principles, the world simply changed.
FOSS generally allows competitors to use your product to compete against you, and always has (the "free" in FOSS refers to freedom), so either you were never okay with FOSS, or you were okay with it and then abandoned those principles to not be okay with it, but having a "principle" that only holds true until you don't like the result is unprincipled
have times changed in a way that justifies abandoning FOSS principles, which include being okay with your competitor using your FOSS software to compete against you? I don't think so.
and also, the whole ecosystem didn't exist when FOSS was created such as selling your software AND a cloud offering for it. If you created let's say an office suite and made it open, your competitor couldn't sell SaaS based on it 20 years ago because SaaS just didn't exist as a concept.... they could try to sell the same product debranded but that's not a very good business strategy.
hashicorp made a good decision here by being FOSS while it made sense, and when the leeches appeared, they decided to restrict future versions. This kind of leeching is the next m$-like EEE (extend, embrace, extinguish) - steal FOSS, host it and capture all the profits, while not maintaining or improving the software at all.
This kind of rent-seeking was simply unimaginable in the 90s when FOSS became a thing.
Also, hashicorp didn't leech off anyone (they didn't make the licence change retroactive, it's only future versions) because their contributions remain available under the same licence - it only applies to future hashi commits. Why shouldn't hashicorp have the freedom to commit in the fuiture with a different licence?
the "rent seeking" in question is Hashicorp seeking rent, here, now
FOSS generally allows competitors to use your product to compete against you, and always has (the "free" in FOSS refers to freedom)
also, nobody leeched off Hashicorp, they used the software as the license and Hashicorp intended, until Hashicorp changed their mind on being FOSS. Why shouldn't they have the freedom to use software in accordance with the license?
"FOSS generally allows competitors to use your product to compete against you, and always has" Sure, I know, but I don't think that's viable when you are running a business. You have to take some freedoms away if you want to have a viable product.
> I don't think that's viable when you are running a business. You have to take some freedoms away if you want to have a viable product.
I don't agree with this opinion, but even if it were true, it raises the question of why, then, Hashicorp chose that path in the first place, when they knew what FOSS meant?
If they didn't like the freedom part of FOSS, they didn't have to embrace it, but they did, and they did
I'm all for proprietary companies not pretending to be opensource companies and actually using proprietary licenses.