Git vs. Fossil: what you should have done vs. what you did

Then there's everything that branching can do, that shelves cannot. For example: history, and undoing your mistakes/experiments. Shelvesets contain no history, and tacking dates/numbers after the name is exactly the problem that VCS sets out to solve. To name one example.

Shelvesets are a hack/kludge.

This forces individual programmers to think somewhat carefully about leaving a sensible commit history behind.

I don't understand the desire to squash every PR into one commit each either…

This way it is even easier to find the offending commit, because changes are smaller (yes, sometimes not working at all).

Fossil's strong resistance to changing committed history encourages a culture of not committing predictably-broken stuff, at least not to long-lived branches. This seems like an inherently good thing, to me. Every commit to long-lived development branches should pass all tests and should build on every supported platform. Fossil has tools for fixing this after the fact, but test-first development is your first line of defense for avoiding problems here.

If you're doing experimental work, you do it on a branch in Fossil, which unlike in Git is sync'd globally across all repo clones, so you can do things like check out the branch on another machine and test there before merging it down to the parent branch, presumably one of these long-lived development branches. This is useful in cross-platform development, so you catch portability problems.

(Incidentally, this merge point provides most of the benefits of Git's rebase without rewriting history and without losing the record of how you got to that merge point.)

Fossil just got a feature to avoid the need for a branch in that case: "fossil patch", which creates a Fossil repository (a specially-formatted SQLite database) with as little as one proposed commit, then gives you tools for schlepping that to another repo and integrating it temporarily so you can test on another machine before committing anything. These patch files solve all of the problems with "diff -u": preserve commit comments, file additions/deletions/renames, branches/tags, and so forth.

Fossil's shortest-path bisect algorithm can sometimes cause it to go down unstable branches, but there are ways to redirect it back to the stable branch or to skip broken commits.

Fossil is quite the antithesis of unusable and nightmarish in my experience.

If you've never received a 500-line patch with a one-line error in it, you haven't been developing collaboratively for very long. Me, I want all 10 commits so I can see from the commit message why the third commit broke the build on platform X, the one clearly tested only on platform Y. I don't want to disentangle the whole mess-o-hackage with a single vague commit comment ("added the foo feature") when faced with such problems.

Am I going to want each and every commit for all history? Of course not, but they're cheap to store, and Fossil makes it easy to ignore the ones not immediately in front of you.

You can usually tell if a difference in viewpoints is ignorance by how the other person talks about the idea they disagree with. If they don't mention any of the perceived advantages (or try to but miss the most obvious ones -- like, say, the ability to do bisection or review changes), there's a good chance that it's out of ignorance.

EDIT: It seems I was slightly mixed up; I was referring not to to the text linked story (which just talks about valuing the "messy" history) but to "Rebase considered harmful" [1], which having just re-read, am confirmed in my opinion was written by someone ignorant of what they're arguing against.

[1] https://www.fossil-scm.org/home/doc/trunk/www/rebaseharm.md

> It does not help the conversation at all to have an anonymous commenter dismiss SQLite as a toy project.

First, I'm not anonymous -- I don't have my full name in my description, but if you wanted to track me down it shouldn't be too hard based on my comment history.

Second, I never said SQLite was a toy project; on the contrary, I said it was "AWESOME". What I said was that it was small; and what I meant was that it had a small development team. My understanding is that SQLite doesn't accept external contributions, and that the team developing it is 4 people (or something on that order of magnitude). With only 3 other people, all hand-picked, committing to your tree, you can afford to just look at the total diff of the branch before merging it (and/or presumably have a conversation about what's in the branch). That's simply not going to work if you're reviewing dozens of patches a day from random people on the internet.

the patch rewiew workflow of the linux kernel community depends on being able to rework the history until all reviewer agree: this is how this new feature is done, and this is how it is exposed. and then it is added and it history will never be changed again.

is that type of workflow possible with fossil? e.g. reworking the history for a feature branch?

So first, let me reiterate that SQLite is awesome; you and your team have probably had a larger positive impact on the world per man hour of coding than any software project in the history of the world. You've already given us an amazing self-contained database system; I'm hardly in a position to demand that you write me a version control system as well.

Secondly, when I wrote "this could only be written by", I was really mixing up the linked text with another article on your website, "Rebase considered harmful" [1], which is far more opinionated. I'm sorry for getting that slightly mixed up; but only slightly sorry, since the "original designer of Fossil" wrote both; I understand that person to be you.

[1] https://www.fossil-scm.org/home/doc/trunk/www/rebaseharm.md

I didn't actually say anything bad about Fossil or that it wasn't a useful system. What I did say was that presenting a change as a series of chunks presented in a logical order, targeted at easy reviewing, is necessary for a large project. I stand beside that assessment.

> Git comes with the idea that individual developers keep their own private branches and only share them after they've been cleaned up. The idea behind Fossil is that everybody shares everything.

And I can't see how this is remotely sustainable for a large project, or a project hoping for "drive-by" contributions. I understand SQLite to be developed by 4 people or so, who have worked closely together for years; I can see where having every development branch of every developer would be an advantage in your case. And since you don't generally accept external contributions (as I understand it), "drive-by" contributions aren't a consideration.

The previous release of my project, which represented only 8 months of development, had changesets from 61 different people; and we frequently get submissions from people who just have a single fix or a single feature they wanted to add, after which we never hear from them again. I can't imagine having every version of every development branch of every person who'd ever contributed to the project (or thought about contributing to the project) in the project log.

> By analogy: Git is like where everybody has their own office with doors that close, and developers emerge from their own office from time to time to share their work or collaborate. Fossil is more like an open office with no walls or doors.

Git was designed so that thousands of people in different workshops and in different towns could come together in an ad-hoc fashion to share what they'd been doing. Having four people working together in the same keycarded space sounds nice; but having thousands of people in one gigantic gym sounds like pandemonium, and having the doors open so anyone can walk in and start vandalizing the place is unworkable.

> False. All you need to do is start a separate "presentation branch" and cherrypick and/or merge changes from your "development branch" in as necessary to make it logical for review. Not difficult to do.

And if you need to iterate over that "presentation branch"? Do you make a new "presentation branch" every time you realize some change from patch A would logically go better in patch B or vice versa? Is the final version of the presentation branch automatically linked to the intermediate presentation branches, and the original development branch(es)?

In your documentation you seem to be aware that Fossil will "nudge" developers in certain directions -- including "nudging" people to delay checking in changes (e.g., before proper testing). Don't you think having all these previous versions dangling around will "nudge" developers into doing fewer iterations of their "presentation branch", resulting in a branch which is more difficult to review and/or do archaeology on? Don't you think the delaying of check-in, in particular with the knowledge that what you've done will be seen, will negatively affect the way people develop?

I mean, I don't inherently have a problem with keeping all my random development stuff locally. git actually does keep all your old commits around when you do a rebase -- they're just not linked anywhere, and they get garbage collected occasionally. (There's no reason someone couldn't build a porcelain on git to make branches for those commits and link them in the comments.) But I wouldn't to have to manually hide all those branches -- I'd want them to disappear by default, as they do with git, unless I actively look for them. Having them not disappear would nudge me in a direction I don't want to go. And as a maintainer of an open-source project I don't want random people on the internet sending me all of the development versions of their patches; I want to see the actual change they're proposing to make to my project, and nothing else.

I'm willing to accept that I maybe I just don't understand how your model works; and if I did I'd be less skeptical of it. But having read [1], I'm quite sure you don't understand how my model works, or you wouldn't say it "has no offsetting benefits".

This is a very, very different way of using git from how almost everyone else uses git.

So I can take a look at some prior work of how it gets used in large projects and make sense of it.

> Hour-long test suite runs are a problem in and of themselves.

I'm not sure I agree. Let's say I have a data integration system that has 70 modules. Each module have some integration tests and some baseline benchmarks.

At the time of development I'm only concerned with the 1 or 2 modules I change whose tests can finish in < 5m easily.

But when I push the change I want to be sure nothing else broke. Maybe because the change was in a base module used as a library by some other modules. Maybe I forgot to run tests on 2 of the dependent modules. So running the entire suite on merges (and optionally on PRs if required) is a necessity in my opinion.

Thanks for the link though - I'll take a look.

> You either need multiple levels of testing so you get some feedback quickly, or you need some sort

> of parallel build system that breaks the tests up so that the whole set completes in a reasonable

> amount of time. Without that, you're going to have people either sitting around waiting on test

> runners or skipping the tests entirely.

100% agreed. With most of these points you can now ignore my previous paragraph. By a 1 hour test suite I mean the wall-clock time of a single build-test cycle on CI runners. i.e. to run ~130 test suites in parallel against various containerised databases and some DBaaS (like maybe Redshift, BigQuery etc.) takes 1 hour where each individual module takes 5 to 10 mins. Some may take longer depending on the depth of things you are testing.

> Effort and resources spent on a massively parallel CD buildbot sounds like a good

> way to get things done to me. Computers are cheap; developer time is expensive.

100% agreed there again. In my opinion the organisations that treat CI improvements as technical debt and not as first class citizens of the software development flow are doomed to move slowly and have a hard time getting things done.

Thanks.

Amen to that.

FWIW... i've actively contributed to fossil since 2008 and can say with some authority that we (the "regulars" on the project) have never done "PR pushes" for more users. It's simply not how any of us function. We welcome those who stop by, and we're happy to evangelize when approached, but we don't go actively hunting for more users. We're more of a "build it and they will come (or not)" project. Every couple of years someone pops up in the mailing list/forum proposing ways we could gain more users if we'd just do this and that and that other thing, but that's simply not a priority for any of us who actively work on Fossil. Continued work on Fossil is not about dominating the DVCS field (git has, for better or worse, long since done that). As Richard is quoted in earlier responses in this thread, Fossil is, first and foremost, a tool for maintaining sqlite. Similarly, i make use of Fossil for literally all of my own code because it's a great tool for the job, whereas git is... errr... less so.

PS: the overwhelming majority of FOSS projects are far closer in size and scope to sqlite than they are to the Linux kernel (for which git was created).

SQLite has had about 33 contributors (after you combine people using multiple login names). But 92% of the commits have been from just two people. 97% from just three people. Then there is a long tail of other contributors.

Fossil itself is similar: https://fossil-scm.org/home/reports?type=ci&view=byuser - 5 or 10 people account for most of the commits, and then there is a long tail.

But aren't most projects like this? A few core developers are responsible for most changes and enhancements, and then there lots of others that might contribute a patch or two here and there? I suspect there exceptions to this (the Linux kernel comes to mind) but I think they are rare. Correct me if I'm wrong.

ironically, i read “A supposedly fun thing I’ll never do again“ and subsequently resolved to never go on a cruise, either…