Fossil vs Git
fossil-scm.org
fossil-scm.org
If instead of "squashing", it were "grouping", I'd be happy. I could encapsulate a bunch of messy commits that I made while I didn't know what I was trying to do. The intent would be clear at a higher level, but if you want to dig in to see what it actually took me to achieve that, you can see all my experimentation by looking at the commits inside the group.
Groups should be able to be nested, of course.
The only way I know how to achieve this in git is my relying on no-fast-forward merges. There's a post detailing this approach [1], but unfortunately a lot of git tools don't support this workflow that well. I haven't gotten it to work well in gitlab, for example.
[1] Git First-Parent-- Have your messy history and eat it too : http://www.davidchudzicki.com/posts/first-parent/
That oddly sounds like a feature available in mercurial.
The latter is, of course, wrong as it makes the repo history harder to read, while on the other hand the former improves readability.
Personally, how I tend to work is if there’s some link between commit and ticketing system I can refer to, it’s about the best you can expect.
An example is the "About branch begin-concurrent-pnu-wal2" at the top of the page https://www.sqlite.org/src/timeline?r=begin-concurrent-pnu-w... - the page shows all check-ins for the branch, and the Wiki at the top gives a summary of what that branch is about.
Another example is the detailed discussion in the "About" section for check-in https://www.sqlite.org/src/info/718ead555b09892f - important information that records the thinking about this commit but which seems too prolix for a check-in comment.
Let me know your thoughts on this idea.
In the last week I have been quite charmed by many of Fossil ideas, I will for sure try this feature too.
> Revert: Some WIP didn't work out.
Git blame again from prior to that commit.
> Added missing semicolon.
and again:
> Fixed spelling.
and again:
> Stupid typo, wrong method call.
and again:
> WIP, going to see if X can work.
Before running git blame once more, finally getting to the commit message that actually pertains to the current line of code you're seeing. Including the explanation (commit message) for why it is the way it is, and very importantly when this line of code actually made it into the software?
...and maybe consider adopting atomic commits -- the main reason I like them is not actually because they make it easy to look at history, but because they're easy to review and catch these types of errors. If each commit stands on its own, it's obvious when one doesn't.
Granted, most tools don't make that easy to do. But most Git tools are rather blind mimics of simple CLI commands with a nice UI (which can be a huge help), rather than being value-adds in terms of behavior or understanding.
Yes, it would be very frustrating. But you're presenting this as if this is the only alternative, it isn't: I wouldn't approve a pull-request that have commits like the one you mention, I would ask to rework the history of the PR to be a logically sequence, just like exposed in this comment: https://news.ycombinator.com/item?id=19007171
"I think each of your commits should be individually rewritten until each commit makes sense and tells a single, indivudal story that makes sense on its own, while at the same time be completely atomic and as small as possible."
These points usually indicate communication/comprehension errors involving two developers. When I bring the problem to the developers' attention, their reactions differ based on their commit style. If they have atomic commits it's usually a five minute conversation because the nature of the problem is immediately apparent. If they have large squashed commits, I usually have to bring both developers together and have them fight over whose problem it is.
So I would say... a couple times a week, but the overall time savings of finer granularity is significant because it limits the number of parties that end up huddled around a single screen.
Besides, squashing means the identity of the commits changes, doesn't it? So you can't merge the same branch into 2 different branches (like merging a bugfix into both the release branch and the trunk) while keeping the identity of the commits - then when you merge your release branch into your device branch you get wonky duplication of commits in the history.
I notice that move detection seems to get messed up by squashing sometimes...
But maybe I'm using Git wrong, which imho is the biggest flaw of Git - it's so flexible that there are so many ways to "use it wrong".
But squash commits do let you locate the identity of the authors. You just have to look at the PR, where all the original commits are listed
Yeah but again, have you ever needed it? Because you can say the exact same thing about preserving every sequence of backspaces, deletes, and key typings into an editor. But when was the last time you or anyone needed that level of granularity?
Git makes it extremely easy to edit history, with the ability to amend any commit; even several commits back with simple CLI tools like `git rebase -i <ref>`.
However, what it doesn't like you doing is ripping the rug out from other people i.e. editing history team members are basing their work on.
The entire purposes of distributed version control is that development should always happen on a branch. Whether that's a local branch, or some temporary pushed branch (pull/merge request branch etc.) In both cases you can safely rewrite history.
However, `master` (or whatever mainline branches you have) should never contain simple mistakes (e.g. non-compiling code), because the code should have been reviewed before being merged. Of course bugs (non-simple mistakes) will happen, and these ought to be fixed in future commits. Bugs that make it into pre-release or release builds (i.e. `master`) shouldn't be edited out of history or forgotten.
The biggest part of my problem is a total lack of commit discipline but there are times when I'm working on a branch where my commits don't tell a clear story (changed something then changed it back because I decided to do it a different way). That's when I most wish for better ways to tell that story.
I feel like an idiot for not knowing rebase could solve some of this for me. ...will definitely try it next time.
It isn't exactly rare that I go through the blame history on some project to find out why something was done in a way that seems stupid at first glance, just to get stuck on a giant squash commit saying "Implemented X".
No-one cares about stray keystrokes other developers make, it's just noise.
Yes, we absolutely care about the design of the software we're working on, and that's what commit messages, self-documenting code, comments, issue trackers and project management (planning session etc.) are all for.
When you squash commits in Git the default generated commit message is even to merge together all your previous commit messages. Now is your chance to look at those old messages and change "Did X" to "Attempted X, but didn't work because Y".
When I'm investigating when and why some code was implemented the way it is; I don't want to look at a Git blame trying to find when something was changed, just to see that the most recent change was reverting some earlier messing around. Just to git blame again starting from just prior to said messing around, just to see the same thing again - noise is bad!
Sure, and `git commit --amend` is fine for those cases.
> When I'm investigating when and why some code was implemented the way it is; I don't want to look at a Git blame trying to find when something was changed, just to see that the most recent change was reverting some earlier messing around. Just to git blame again starting from just prior to said messing around, just to see the same thing again - noise is bad!
I guess that depends on your setup. My Emacs is set up so that `b` is "reblame from before this change". GitHub's blame UI has a similar button (though that, sadly, doesn't preserve in-file context).
At that point the cost of the "noise" is more or less zero.
If you are working for a larger company where processes are clearly defined, then it’s good for you and that feature is not needed. But you are loosing all of the agile feature of git in the first place.
In my situation squashing history takes away my other ability to use git as a wiki of “things that didn’t work out”. It’s important to keep that.
Wouldn't a wiki be the best solution for that? Git is a software development tool, not a design or project management tool.
Is that similar to what you’re talking about?
E.G. you have worked in a feature branch, and committed 10 times - let those commits be kept in the log, but when running git log there must be a flag that allows for filtering based on how granular the output must be.
That can be done with rebase, but then you loose history.
I am pretty sure you can implement something like that in git based purely on commit message content.
No VCS I’m aware of supports this. But they COULD.
I think each of your commits should be individually rewritten until each commit makes sense and tells a single, indivudal story that makes sense on its own, while at the same time be completely atomic and as small as possible.
You created a new function? That's one commit. Take a moment to explain why this function is going to be useful in future commits.
You called that new function from several new spots? That's another commit. Explain why each of these calling sites requires this function.
You decided that there had to be some style and whitespace changes? That's another commit. This one can go without explanation.
You found an old bug along the way that can be fixed with a one-line change? That's another commit. Hey, nice catch. Perhaps a few lines about this bug and why your small change fixes it?
Together, these are all individual scenes of a larger story. But the larger story doesn't need the behind-the-scenes of how you came up with these scenes. I don't need to see all your drafts of each scene. I just want the final scenes that make up the final story.
How does this work with multiple developers working on the same repo? I'm assuming everyone should work on their own feature branch and send PRs once their branch is done? Should the commits also be tagged by the feature branch they're on? Should the CI approval workflow be run against any combination of commits on the feature branch or against the final HEAD?
An individual commit is a single patch, intended to do one thing (and hopefully do it well), and a feature branch is a patch series.
A pull request is then a request to review the series. If you need to change things, git allows you to rewrite your commits to send in a revised set of patches.
Before merging, your CI would create a temporary branch off the current master, merge your feature branch to that, and run tests against the result. I don't think testing individual commits (fully, at least) in a series makes much sense if you're going to merge all of them to master anyway.
They can say "to get your code upstream you have to do twice as much work" or 3x or 4x or whatever. It's not their cost to bear.
An internal team pays that cost. They have to consider whether the trade off of having a pristine commit history is worth the additional overhead of doing it.
I personally care more about PR size and am happy to squash all commits in a PR. If that is too big I'd rather see multiple smaller PRs.
If important things are lost then perhaps the PR was too large.
"You misunderstand me. Almost everyone does."
That sounds really frustrating. It's not clear from your post whether you mean this as "everyone who reads this comment does not understand me" or "people frequently misunderstand me"... but typically, someone who feels this way experiences this in the latter, general sense.
Good news: it is possible to dramatically improve the ratio of people who understand you. It will, however, require that you change how you communicate some kinds of information.
What I'm saying is that if people don't understand you, the first place you need to look is your own tools for communicating with people. Of course, what you are saying makes sense to you. But there's an intellectual fallacy at work if you assume that "almost everyone" is the problem and you are the solution. :)
Have you ever heard the saying, "The only thing all of your crazy ex-partners have in common is you?"
There are people who are tasked with explaining far more complicated concepts than source control who people claim to love learning from. That means there is hope for us all.
It requires that you either stop your development workflow to commit as you go along, or that you untangle all the pieces after they're already entangled.
If you commit as you go, it's an expensive mental switch to fire up git and also run all the tests (since surely part of this workflow is to apply the principle that no commit should ever break the build). You also take an extra productivity hit every time you change your mind about something a little later (e.g. you added the function getFoo() but realize it should have been called findFoo()).
If you work for a while and then try to bundle up small, atomic changes, that can also be very difficult. Tools like git group together contiguous chunks of changes when committing, and prying them apart later can be difficult. I often do this with a combination of "add -p" and then "stash save -k" to temporarily get rid of things unrelated to what I'm committing, but it's a chore. During a selective "add -p" session you have to mentally keep track of what belongs together, thus what dependencies are between every chunk you're adding.
Committing as you go is easier, but it's slow, and doesn't work well when you're working across many files with a big change that introduces new semantics in a lot of places. Both techniques require that you keep track mentally of which parts are related, of course.
I figure if I don't carefully read my own diff, why would anyone else? And once it's untangled, I am hoping others will find it easier to read too.
Git doesn't provide as many tools as I would like to make this process easier. It's partly why I don't use git. Mercurial's absorb command helps a lot: it absorbs changes from your working directory into the appropriate draft commit that corresponds to the same context:
https://gregoryszorc.com/blog/2018/11/05/absorbing-commit-ch...
Wait, it appears someone finally ported it to git:
It still doesn't solve how to disentangle changes that have become interdependent. For that you have to concentrate on committing atomically and planning ahead a lot.
If you're doing two semantically different things, put it in two different commits. If it's one, put it in one (merge commits work too). That's just good change-hygiene, for the same reasons you try to isolate behavior in code, rather than mashing it all together into a single func just because you happened to be doing it all around the same time.
tl;dr we don't name our funcs "june_27_through_29", don't name your commits like that.
[1]: a huge investment, so I totally get why this isn't an early-coder practice, and it's rather painful. but IMO worthwhile, usually I see people spending far more time fighting it than it would take to learn it.
I've been using this approach successfully for 8 years now on tens of open source projects and various company code bases of all sizes.
It does take a small amount of overhead (I measure this, and for me it's around 5%). But that pays off immediately as soon as you or someone else reads it a few weeks later when investigating an issue.
I wrote a post three years ago about my switch:
> You created a new function? That's one commit. Take a moment to explain why this function is going to be useful in future commits.
> You called that new function from several new spots? That's another commit. Explain why each of these calling sites requires this function.
In my opinion each commit should make sense on its own. It doesn't really make sense to create an unused function, so these two changes should really be one commit.
I think my proposal can also make sense because (1) it splits up the commits into two units that still keeps the codebase in a stable state [this is my rough metric for what "atomic" means] and (2) defining a function requires some independent contemplation about why that function is defined that way. Inserting that function into calling sites can be a logically distinct operation if the calling sites are varied and distinct enough to each have a different reason to now require this function.
Thus the two commits can be semantically distinct.
But if the developer stored that information in a commit, and there is a cheap (computationally and cognitively) way to keep that information, then I don't want to delete it.
It's not charitable for you to call it a distraction at best. There might be useful information available in the dev history. You shouldn't be forced to see it (that would be distracting), but you shouldn't be precluded from seeing it either.
So perhaps fossil really “shows what actually happened” but perhaps the more accurate statement is it shows the actual commit order. And that may not reflect the actual order of what was done, leading to an inversion of the logical progression of the code.
That said, I’m going to try it anyway for the ticket system and wiki. I use a two tier vcs system. “Official” work goes into svn and I have no control over that. Interim work for me is in git. But I’d like tickets and wiki docs to help me track issues that I can’t get to right away and sometimes forget about after a month or two.
Example files changed view in Github:
https://github.com/TechEmpower/FrameworkBenchmarks/pull/4311...
I choose it because it had a very clear and easy UI, guided you towards a way of working that is suitable for small teams (ie 99.99% of all projects), and had an approach to history which made it unlikely you'd ever accidentally lose work.
The only thing which has ever made me sad about my choice is pressure from people who think everyone should use git.
It also has some very nice features, like being able to give you a full web UI just by running it as a CGI process on a web server.
It is worth trying (and it can import and export git history). Just remember it's not git.
Fossil: Many check-outs per repository
git allows multiple checkouts per repo.
Official docs (good luck): https://git-scm.com/docs/git-worktree
Random person's blog that explains it more clearly: https://www.saltycrane.com/blog/2017/05/git-worktree-notes/
A 'git checkout' on a large repository such as the Linux kernel may take a while (bound by I/O performance). Regulary switching between branches with many changed files becomes annoyingly slow.
If you keep multiple clones, you actually keep multiple copies of the full history, which seems like a waste of disk space (yes, deduplicating filesystems exist, but are rarely used).
git worktrees are also very useful if you have unfinished on a release branch. You do not have to make temporary commits or stashes you forget about, you can just leave the modified files as they are and switch to a different worktree to continue your work on another branch.
From all I've read so far (quite limited!) they just sound like replacements for `git stash` or making a temporary commit / branch, but with a new set of commands and rules to learn. I don't find `git commit -am tmp` (literally, I just do that) to be particularly worth optimizing further, and worktrees so far sound like substantially more work.
---
edit: ah, yeah, updating multiple clones is definitely annoying / easy to forget, totally agreed there. that alone might make it worthwhile. I only need it like once a year so I probably won't, but I do know some coworkers who do it a lot.
The same also applies to languages such as Python or Node.js, where you might have a different set of dependencies depending on the branch and don't want to regenerate your virtualenv or node_modules on every branch switch.
(yea, there's .git/info/exclude, but you can't add it to a branch and have it only exist locally. and it has weird interactions when something becomes tracked later.)
Sometimes this is fine, but if the history is large enough (like in the case of the Linux kernel), it can start to get out of hand space-wise. If you have a deduplicating filesystem, that can help, but only so much.
Of course, there is a way to use clone which hardlinks the objects.
Have a "develop" branch that does not contain any generated files, and a "master" branch that does. Have a post-commit hook that uses a worktree pull every commit from "develop" to "master" and generate any files.
That type of workflow is possible by cloning the local repo, but is kludgey and brittle enough prior to worktrees, I had never considered that workflow viable.
- website generator (JS stuff) on "master" branch checked out at $PROJECT_ROOT, and website content (Markdown files) on "content" branch checked out using git-worktree at $PROJECT_ROOT/content.
- Project source code on "master" branch checked out at $PROJECT_ROOT, and images needed for GitHub README file in "media" branch checked out using git-worktree at $PROJECT_ROOT/media.
I never use it for quickly switching topics (that's the git-stash use case, it you can just make a temporary commit).
Worktree adds the ability to share config and save disk space.
> Branches in Fossil have persistent names that are propagated to collaborators via push and pull. All developers see the same name on the same branch. Git, in contrast, uses only local branch names, so developers working on the same project can (and frequently do) use a different name for the same branch.
Since the default is to use the name from upstream when checking out a branch, the vast majority of the time devs working in git also use the same name for a branch. Given that, this seems better stated as a feature for git (Easily Renamed Branches, perhaps).
That's really the bottom line when it comes to Git vs. all the others: Git doesn't enforce a merge-only workflow, not even as a default, while all the others generally do.
Mercurial, for example, resisted adding cherry-pick and rebase for the longest time (just like Fossil), and when they finally added it they had to do it in gratuitously different ways from Git (e.g., you can rebase, or rewrite history, but not both at once) because they are so damned opinionated about how we all should do do our work. So now Mercurial has rebase, and it has "bookmarks" (i.e., Git-style light-weight branching), but you still can't really use it without heavy-weight branching, and you can still get into trouble where you end up with multiple tips on a branch, and it still tries to hide the machinery underneath, which makes it difficult to reason about.
Fossil also tries to hide the machinery underneath it, and that's its big sin. Just like Mercurial. The irony is that under the covers Fossil is a lot like Git but with a fabulous SQL store.
I believe the Fossil maintainers simply do not understand rebase. Or why it's necessary when you have thousands of people working on a codebase.
At Sun we did rebasing long long before that term existed, way back in the 90s (well before I went to work for them), and we did it using ancient tech known as Teamware. That's where Larry McVoy got some of his ideas, and thence Linus Torvalds. Later, when we selected Mercurial, we ended up doing rebase in Mercurial long before Mercurial adopted the feature -- we still didn't call it that, and it was all scripted. Indeed, you can rebase on Fossil just fine, if you're willing to script it around Fossil's primitive cherry-pick (which, incidentally, is pretty much how rebase is implemented in Git, though nowadays it's not scripted).
My theory is that Git -or any VCS- is only difficult to understand if you insist on abstractions that hide its mechanics completely. Abstractions are absolutely critical to what we do, so this might seem a bit incongruous, but I'm not asking for no abstractions here, just that their being leaky is actually a very useful thing in the context of VCS.
I also would rephrase this as a missing feature from Fossil rather than a feature that is missing from git.
I think this is an interesting distinction. The question is: what is the function of the history? Is it to document what happened - and if so, shouldn't every keystroke be committed? Or is it to document which changes relate together, and e.g. should be reverted together? Or perhaps something else entirely?
One thing that isn't mentioned is whether Fossil supports working offline as well as Git does. My impression from this page is that it emphasising creating every branch on a server as well, which implies you need to be connected? If so, I'd consider that a "missing feature" as well, as it's something I use regularly enough and that's important enough that it'd be something I'd miss if it weren't there.
The way fossil works is that by default autosync is turned on, which means that if you are connected to the internet all of your commits, wiki and issue tracker changes are synced to the server.
In the case that you're not connected, everything still works as expected, and will sync back up next time you're online and try to commit or just run `fossil update`.
If I never commit until I'm done you will not see my mistakes, but if there's nothing like `git add -e`, then I won't be able to split up my commits logically and so the upstream history when I push will be devoid of useful detail. Even if I do commit early and often, I will almost never do all my work in the order and logical splits that makes most sense for others to see after I'm done... unless you give me a powerful rebase facility.
No rebase -> no clean history upstream, only lots and lots of merges with pointers to messy history (where authors commit early and often) or shallow history (where they commit only when done) in branches you'll almost never want to see. This is the worst possible outcome! The only logical history organization here is "the large merge commit". You get far too little abstraction on the actual history and far too much noise if you want more detail.
There's also the risk that, because you cannot separate bits of work as "done" and push them separately, your branch will accumulate ever more change and reach a point where you cannot continue without doing much more work, and then you won't be able to easily salvage any of the work done to that point in that branch. This one is a big deal to me too.
If you have to rewrite the actual history to get what you call a 'clean' history, rather than just overlay the detailed history with a sparse abstract lattice joining the key points, then it is not a history at all. This could be a problem if you need to do a post-mortem analysis, or if you make a mistake in the rewriting of the history.
I tend to regard frequent extensive rebasing as a process smell - not necessarily a problem, but a warning that there might be one.
When I later started using git, my workflows simply became safer and easier.
I realize that we generally have to work with the available tools, but it is also useful to think about how things would optimally work. When I wrote that frequent rebasing looks like a process smell, that could because the tool is not optimal.
It's the only way to do things that yields useful history in the upstream. What is useful history in the upstream? It's history that others can read (linearly!) that is informative and makes sense and makes it easy to bisect, git blame, and so on, to find bugs, to understand changes in the smallest logical units.
This is impossible to do without rebasing.
If you find yourself copying changes to another clone to commit them one at a time, then you are rebasing, and you just didn't know it.
History in the upstream is sacred. Unpushed history is absolutely not.
Given that we have this secondary history, why require a completely different tool to track and access it? That's just pointless duplication. We should track all the history using the same tool.
Now to your point, it's still useful to call out "final" commits for bisecting, blame, etc. So it would be good to group commits, and hide the detailed history by default.
Still, I've worked with codebases sized in the hundreds of millions of lines of code. To deal with that level of complexity one needs things like OpenGrok, cscope, and so on, to find one's way around. And when it comes to history, I could not care less about past code reviews or history internal to a feature branch. When I need to do `git blame` or look through commit history or a large codebase, I want to see clean history with a high signal-to-noise ratio. The more noise, the slower I'll make progress on understanding whatever code/history I'm trying to understand, therefore the slower I'll make progress on bug fixing or feature development -- I might even give up on history, and lose a lot of important information, if the noise level is too high.
For me the ability to rebase, and to require clean history, trumps all the great things in Fossil -- each and every one -- that Git lacks. And this even though I love Fossil's design.
Regardless it's possible to have both. An example is hg's changeset evolution. With changeset evolution, each commit has two histories: the repo history and the changeset history. Commands like `blame`, `log`, etc. show only the repo history; a separate set of commands accesses the changeset history.
An example where this is useful: sometimes rebasing can inadvertently produce bugs, such as collapsing two identical lines which ought to have been duplicated. `git blame` cannot check if that happened. But the changeset history, by tracking the rebase, can tell you that.
EDIT: I suppose I might look at past code reviews when evaluating a candidate for employment. Still, there is no need to store those along with code. And if a code review comment needs to be recorded for posterity, it gets recorded in the code or in the commit comment.
Clean, linear history has never been required anywhere else I've worked, but I've done it ever since Sun taught me to. Just because it's not required doesn't mean it's not a good idea, and it can't be forbidden ("what happens on my dev instances, stays in my dev instances", and the only thing seen in the end is what I choose to publish, and it's going to be clean and linear).
Large projects at Sun used a rebase-heavy / rebase-only workflow like so:
+-----------------+
| Upstream "gate" |<------+
+-----------------+ \
/ \ \
/ \ \
v v \
+--------------+ +--------------+ +--------------+
| Project gate | | Project gate | | Project gate |
| (re)based on | | (re)based on | ... | (re)based on |
| build N of | | build N+1 of | | build N+M of |
| upstream | | upstream | | upstream |
+--------------+ +--------------+ +--------------+
^ ^
| /
| / ...
v v
+-----------------+
| dev clones |
| |--+
| periodically | |
| rebased --onto | |--+
| next rebasing | | |
| of project gate | | |
+-----------------+ | |
| ... | |
+------------------+ |
| ... |
+------------------+
We did this with early 90s tech known as Teamware, with lots of scripting on top. We later did it with Mercurial (again, with lots of scripting on top). Mercurial was a mistake. Git is much, much easier to use this way than any other VCS I've ever worked with, which for me includes: CVS, Clearcase, PRCS, Subversion, Mercurial, Git.
Anecdotally, as a fellow Sun Alumni, I strongly disagree (and I know many Sun Alumni would as well). I greatly miss using Mercurial instead of git now that I work elsewhere. The cadmium extension we had in-house at Sun more than made up for any plausible deficiency and Mercurial phases let me mostly drop use of cadmium.
You wrote this:
> > > When I worked with VCS that couldn't rebase, my strategy was simply to not commit until everything was perfect. I had a local branch. It was just not version controlled.
if I were to do that (and I have) with anything other than Git, I'd have a hard time splitting up the commits in the end. Mercurial has `hg record`, which is akin to an atomic `git add -p && git commit`. I don't think Fossil has anything even like Mercurial's `hg record`, and it famously lacks an index/staging area.
(So Mercurial has an index! but as always with Git features belatedly adopted by Mercurial, it's a pain to use in Mercurial. If you want to stop in the middle your choices are: say 'N' and commit what hunks you've accepted so far, or quit and abandon the hunk selection work you've done so far. And you don't get to edit hunks.)
> > > When I later started using git, my workflows simply became safer and easier.
Mine too.
This highlights one of the benefits to framing comments in a positive manner: they tend to be inoffensive even when misunderstood.
I think I might’ve read something like that in the Pro Git book and it rings true with me.
The philosophy of Fossil is to preserve the full history, which I think is a mistake.
You could achieve the same by using an empty commit, or tracking those in the bug tracker.
DCVSs are already complex enough, this just seems like scope creep, trying to emulate a feature that is actually part of another tool (the bug tracker or the mailing list, depending on the structure).
Another issue is that it would foster a mentality where people would simply don't care about the exact history and push really badly structured commits, relying on the second-level history to explain their work. Except that if someone publishes something, it's not for this person, it's to be read by other people. The dev could keep a local branch with all their commits if they are so inclined, but that should never be sent to a shared server.
Most people are already bad enough at writing proper commits and dividing their work, I think your feature would make it worse.
Fossil takes a stance, git doesn't. It makes fossil less versatile but it is not a mistake. They never claimed Fossil to be the perfect solution for every project.
You can argue about whether “the Unix philosophy” is best for specific tools like version control, and maybe you can argue that “the Unix philosophy” isn’t really about simple vs complex tools. But I don’t think I’ve ever heard nor would I think you could find evidence that the Unix philosophy is “it just works”. That claim makes me pretty certain that this is a tool I want to avoid at all costs. And the tone of the rest of this article confirms it.
This seems like an inaccurate phrasing of this. Git (the tool) doesn't specifically emphasize this, but it does support this as a possible approach without excluding the other approach. Many teams have settled on development approaches with git that do indeed emphasize this, but not every team does. If you want to show your work, you sure can, but unlike fossil, git does not force this upon developers. Like the article, I don't see either approach as right or wrong, but know that I prefer the clean history approach rather than every typo commit ever.
git log --graph --oneline
Mercurial can rewrite history in the same ways as Git.
I never want to see junk commits, but I can't force other people to spend time cleaning them up. Squashing cleans them up quickly but loses useful information.
Add feature XYZ
Fix bug I just found around that code
Fix typos in XYZ
Fix bug in XYZ
Add tests for XYZ
Fix another bug I found around that code
Fix bug in XYZ
fixup! Add feature XYZ
fixup! Fix bug I just found around that code
Fix bug ... (not related to XYZ)
Fix second bug ... (not related to XYZ)
Add feature XYZ
Reword the commits to add ticket IDs to the commit subjects for all of these, if you have this as a requirement or have an issue tracker.
Force push your branch, do one last round of build, test, review, and push when all looks good (else rinse, repeat).
Maybe you'll realize that XYZ is two features, not one, and then you might split that up into two commits (again, using `git rebase -i`, mark that commit for "edit", `git reset HEAD^`, `git add -e`, `git commit -m 'Feature WXY'`, then `git add`, then `git rebase --continue`.
Some commits got squashed, maybe some got split, and all got reordered.
The resulting history is easy to read.
This takes a bit more work. But once you get into this workflow, it begins to become second nature, and you get good at it and it doesn't slow you down. The benefits are well worth the learning curve, and it's not that steep.
In fossil's case this was architectural simplicity, an easy-to-use basic set of commands with a discoverable set of expanded commands that always seemed to meet my needs as they cropped up, collaboration features in the pre-GitHub era were killer. There are a few discrete layers of hackability for customization with minimal risk of blowing your project away.
Fossil still meets my needs admirably, and I stick with it even though it's harder to make the case for it in the current era of feature convergence.
Fossil also lacks an equivalent to git submodules, which should be added to the list of features found in git, but not in fossil.
Also, this page seems to do the common but frustrating thing of advertising how something is implemented as a feature. I'm sure it's very interesting to the developers that it uses a database in the backend or that your side-project is written in a niche language but, as an end user, I just don't care.
When the design of a system makes something difficult, people tend to avoid doing that thing. The Git data design makes extracting some kinds of historical information difficult, with the result that many common tools (ex: GitHub) don't bother. This reduces the information available to users, resulting in reduced situational awareness.
I make no claim that Fossil is perfect in this regard. My only claim is that Fossil is better.
Whether you use Fossil or not, I don't much care. But I really would like you to understand some of the features of Fossil that (I think) make it better than Git, and perhaps start adding some of those features to your favorite VCS, be it Git or something else. Understand the general concepts, then port those concepts to other systems.
Fossil is worse. Because it doesn't have a CoW design in the format of its DB, though it does have a CoW design in its SQL schema. However, because SQLite3 is so good at handling power failures and such, this has not been a big deal (and indeed, Git has had more problems there owing to its ad-hoc storage model). This, of course, could be fixed by adopting a CoW backend for Fossil. This problem is hardly fatal.
Fossil is worse. Because of its insistence on merge workflows and not implementing rebase.
Fossil is worse. Because it has minimal mindshare. Mindshare is absolutely critical. The lack of rebase support is part of the reason that Fossil cannot get the kind of mindshare that would allow it to replace Git. And it may well be too late, but you never know.
It turns out that rebase is the more critical thing for me and many others. I think the Fossil community doesn't know it and doesn't get it because the projects it deals with don't have thousands of developers active every day. And, of course, rebase is really just a feature that can be built by using scripting on top of a cherry-pick primitive (which Fossil does have), so there's no real excuse for not having it.
Incidentally, GitHub used to also insist on merges in its browser UI, but eventually they added the "rebase and merge" feature.
Mercurial too resisted rebase. Boy did they. And they gave in and implemented it. Ironically many people rebased in Mercurial long before the feature was adopted -- we did, in Solaris engineering, a dozen years ago.
My prediction is that Fossil too will eventually support rebase (out of the box) for the same sorts of reasons that all other VCSes have tended to get it.
If I had to use Fossil, the first thing I'd do is script a rebase around cherry-picking. Because I could, because the market demands this feature, because it's a built-in feature of competing, more-successful VCSes, Fossil ought to adopt it. You wouldn't have to use it. You might have a way to turn it off on a project-by-project basis. But the rest of us should get to use it on our projects.
All in all, I neither use nor recommend Fossil. But I am envious, and I would use it, if it had rebase.
I'm also not convinced it's a good idea to merge your bug tracker and version control.
The wiki had its own weird markup at the time, so people who might have helped me write documentation were faced with learning a weird new markup (they were volunteers, that's a tough ask.)
The bug tracker had no way to send emails, and you were expected to rig up some external systems to do that. Maybe it's changed now?
Just lots of little usability things too.
These issues had such ardent defenders on the mailing lists, too, I became convinced the situation would likely never improve.
Aside from the cute idea of having your version control in one self-contained file with a single-binary server, I can't think of anything fossil does better than any other system.
YMMV.
It is just like distributed code review tools, I wish there was any that was good enough but the ones that exist are too primitive to be worth the small gain of being able to seamlessly review code while on a flight.
> The fossil push, fossil pull, and fossil sync commands do not provide the capability to push or pull individual branches. Pushing and pulling in Fossil is all or nothing. This is in keeping with Fossil's emphasis on maintaining a complete record and on sharing everything between all developers.
This is a kind of deal breaker if it just pushes all of my private WIP branches to remote. Is there any genuine use case for this kind of functionality? Apart from that, this looks interesting.
[0] https://www.fossil-scm.org/xfer/doc/trunk/www/private.wiki
Most projects are invisible to you, like in-house projects that never see the light of day. If Fossil is used by 1% of those projects, that's still a lot of projects.
How can you tell if Fossil is popular in HN? By my reading, only two people in this thread use it. More than that mention having never heard of it before, or make statements where it's clear they don't really understand what it is.
There's really no reason to think that Fossil is used internally at a rate higher than its public-facing usage. And I've literally never seen a project that uses Fossil aside from Fossil examples.
From https://news.ycombinator.com/from?site=fossil-scm.org there are 20 links to fossil-scm.org in the last 2 years. I manually looked at the last 2 years of submissions with "fossil" in the name and found an additional 4 which were not to fossil-scm.org.
It's over 50% higher for Mercurial. I counted about 30 postings with the name 'Mercurial' over the last two years, of which about 7 were from mercurial-scm.org. I left out some of the obvious duplicates.
You write "I've literally never seen a project that uses Fossil aside from Fossil examples".
How much effort did you put into looking, and would you have recognized one if you did?
After SQLite, the most widely used project which uses Fossil is likely Tcl. http://core.tcl.tk/tcl/wiki?name=Index says it uses Fossil 2.7. For obvious reasons, there is an affinity between Tcl projects and Fossil.
A search for '"This page was generated in" "Fossil"' in DDG finds some non-trivial active projects: https://duckduckgo.com/?q=%22This+page+was+generated+in%22+%... . "Active" means "commits in the last few weeks." For examples:
"MySQL++ is a C++ wrapper for MySQL’s C API" - https://tangentsoft.com/mysqlpp/home
"Jsi is a C (+/-) embeddable JavaScript interpreter" - https://jsish.org/fossil/jsi/doc/tip/www/home.wiki
"Cxxomfort (cxx as in C++, comfort as in comfort) is a small, header-only library that backports various facilities from more recent C++ Standards" - http://ryan.gulix.cl/fossil.cgi/cxxomfort/index
"SquirrelJME is intended to be a Java ME 8 compatible environment for strange and many other devices" - http://multiphasicapps.net/doc/ckout/readme.mkd
[0] https://fresh.flatassembler.net/fossil/repo/fresh/index [1] https://flatassembler.net/fossil/repo/fasmg/home
There is a online hosting service: http://chiselapp.com
A list of public projects (I was kind of surprised there were so many): http://chiselapp.com/repositories/
I would be happy to use the site for publishing software. But to see if I can rely on the site for backup, aswell as simply general interest; I would enjoy reading (maybe a page on the site) about where it came from / what your plans are for it / longevity / intent
i.e. I want to hear more about you ;)
The fossil-scm.org server gets between 1500 and 2000 distinct human visitors per day on weekdays. (Lower traffic on weekends. Robots are excluded from the count. "distinct" means visitors having different IP addresses.) This is perhaps orders of magnitude less than git-scm.org (I'm guessing - anybody have stats?) but it is also non-trivial. With ~1750 visitors per day somebody must be using it. And I would guess that most actual users do not visit the site daily. (When was the last time you visited git-scm.org?)
FWIW, today has already seen in excess of 10,000 distinct human IPs, likely due to this HN post.
Like most people, I go to Google which then redirects me to the correct Stack Overflow question/answer the most often. I think git-scm.org is mostly a guide to get started and the reference documentation.
Is this not terrifying? I've always thought the distributed nature of git is part of why it's so incredible. Not relying on a few handful of individuals.
This is interesting. I wonder if the same thing could be achieved fairly simply in git though by keeping an extra area in the .git folder that handles reversing / denormalizing the DAG for some of the things that these reports show.
Microsoft are innovating in this area (bringing the improvements that they have in VSTS to the client). There's a good series of blog posts on this at https://blogs.msdn.microsoft.com/devops/tag/git/ (first article https://blogs.msdn.microsoft.com/devops/2018/06/25/superchar...)
In general upstreams in Git do not allow history rewriting. It's only the forks (local branches) that allow history rewriting. This is just fine.
Also, when upstreams do accept non-fast-forward pushes, there's `git rebase --onto` for recovery by downstreams.
In git, this is pretty much a oneliner script dropped in your path and named `git-all`.
Something like (typing from memory on my non-work computer):
#!/bin/bash
find . -type d -depth 1 -exec git -C {} "$@" \;
git all status
git all status --short --branch
git all pull --rebase> ”SQLite uses cathedral-style development. 95% of the code in SQLite comes from just three programmers, 64% from just the lead developer. And all SQLite developers know each other well and interact daily. Fossil is designed for this development model.”
FWIW, I consider this an extraordinarily productive development model.
Agile may be about People and Interactions over Processes and Tools, but (and it’s too often forgotten) all dev must first be about effective output in the problem space. So, how do you optimize ratio of Outcomes to People/Interactions/Processes/Tools? This model.
FWIW, I also consider Linus’ and Guido’s “BDFL” approach a way the bazaar is channeled back into a cathedral.
The comparison re. licenses is factually mistaken in several ways, but that's probably not worth a great deal of discussion - it's too religious a topic.
As you can see from bullet point #5 from the fossil home page, it works offline:
> CGI/SCGI Enabled - No server is required, but if you want to set one up, Fossil supports four easy server configurations.
and #6 concerns later online merging:
> Autosync - Fossil supports "autosync" mode which helps to keep projects moving forward by reducing the amount of needless forking and merging often associated with distributed projects.
There's also the quick start guide with an overview of distributed-ness - https://fossil-scm.org/fossil/doc/trunk/www/quickstart.wiki .
1) UI- terrible, terrible UI
2) Unncessarily complex data model
3) Doesn't scale well to large repos (until Microsoft's VFS- windows only)
(and many others...)
The git data model is certainly complex, but I'd be curious to hear why you think it's 'unnecessarily' so. I often try and drum up a new SCM in my head and the data model gets pretty complex every time.
$ fossil init wibble
project-id: c8d025508afd55c31cebedfe244a3b62f39fb6eb
server-id: 8b340295f8268f742f91c5e972195efc0c6e578f
admin-user: gjvc (initial password is "c5d7ba")
$ file wibble
wibble: SQLite 3.x database (Fossil repository), last written using SQLite version 3026000 - hg was horribly slow compared to git;
- I love the branchs model used in git to let several persons work on different parts of the same project, and I could never find a satisfactory equivalent using hg idiomatisms.
At this point, I can basically do most of Git commands in Magit with just muscle memory; say, stage everything in the tree, commit amend, reset author and dates, then force push (I know) to remote I would just press with evil-magit: <SPC>gsSc-Racp-fpy (<SPC>gs for invoking Magit, S to stage everything, c to enter commit mode, -R set the reset author flag, a amend commit, p enter push mode, -f set the force flag, p push to origin, y confirm force push)
It may sounds complicated, but the Magit UI is discoverable, and once you're used to it you can do anything without even looking at the UI...
I used mercurial successfully, quite heavily, and I couldn't tell you much about how it's implemented.
2) Data model brings speed that was not possible before. git won VCS space because of sheer performance.
3) Scale just fine. Just windows have a really bad filesystem and Windows codebase is the pathologic case.
The problem is that the edge cases that come up have solutions which need to be looked up- not derived from understanding. And when you're scared of data loss, its a very frustrating situation
Can you specify a version control system which doesn't make it easy to lose uncommitted changes?
This makes it easy to compare, say, the state of the file now with the state of the save from 3 hours previous.
https://en.wikipedia.org/wiki/Versioning_file_system points out "Subversion has a feature called "autoversioning" where a WebDAV source with a subversion backend can be mounted as a file system on systems that support this kind of mount (Linux, Windows and others do) and saves to that file system generate new revisions on the revision control system."
Quoting http://svnbook.red-bean.com/en/1.4/svn.webdav.autoversioning... :\
> the use case for this feature can be incredibly appealing to administrators working with non-technical users: imagine an office of ordinary users running Microsoft Windows or Mac OS. Each user “mounts” the Subversion repository, which appears to be an ordinary network folder. They use the shared folder as they always do: open files, edit them, save them. Meanwhile, the server is automatically versioning everything. Any administrator (or knowledgeable user) can still use a Subversion client to search history and retrieve older versions of data. ...
> however, understand what you're getting into. WebDAV clients tend to do many write requests, resulting in a huge number of automatically committed revisions. For example, when saving data, many clients will do a PUT of a 0-byte file (as a way of reserving a name) followed by another PUT with the real file data. The single file-write results in two separate commits. Also consider that many applications auto-save every few minutes, resulting in even more commits.
It adds that Clearcase supported a similar feature.
I have never used that combination.
* Pushing changes to a central repo requires including upstream commits. With 1 commit/s to that central repo, all developers are stuck in a loop until their push succeeds. It is a human spinlock with high contention.
* Some algorithms scale linearly with the number of server branches, such as pull-without-specifying-a-branch, which becomes too slow with 100K branches (a consequence of central repos).
* Some algorithms are linear with the number of files, like git status.
* Binary files don't compress nor deduplicate well, slowing pull and clone.
Those issues apply to fossil and Mercurial.
cf. https://docs.microsoft.com/en-us/azure/devops/learn/git/tech...
Can you provide a reference? I was searching a bit and only things I found was bugs in windows[1] for git lfs.
> You can call this "pathological" but this throws a lot of shade on monorepos without much critical examination of how or when they might be useful.
Windows codebase has 3.5 million files and its repo is 300GB in size. It is not normal. This is google or MS type of problem and not average git user. MS instead changing workflow decided to create GVFS[2]
[1] https://github.com/git-lfs/git-lfs/issues/2434 [2] https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-large...
Apologies, I hastily mistyped, I meant 500 GB, not 5. (5 GB is about the size of my repository, which is not really so big at all and certainly something git can cope with on its own).
This series of articles should illustrate some of the issues that VFS for Git tries to address. ("GVFS" is now called "VFS for Git".)
https://docs.microsoft.com/en-us/azure/devops/learn/git/tech...
And this is a series of articles from an engineer who's been working on improving perf in large repositories in general, not strictly related to the Windows repository:
https://blogs.msdn.microsoft.com/devops/2018/06/25/superchar...
> Windows codebase has 3.5 million files and its repo is 300GB in size. It is not normal. This is google or MS type of problem and not average git user. MS instead changing workflow decided to create GVFS[2]
I didn't say it was normal. Indeed it's uncommon. I said it wasn't pathological.
That's 3x the source line count of Google's entire monorepo. [1]
So if you're using git for source code, 500GB is beyond pathological.
If you're using git for other purposes, then yes you might need something like Annex/LFS/GVFS.
[1] https://m-cacm.acm.org/magazines/2016/7/204032-why-google-st...
git has a terrible UI, granted, but I find the hg UI pretty terrible too. I really hate their approach to branching (tho that is remedied these days with e.g. bookmarks to some extent).
Speaking of data model, I find gits model to be a lot saner. I REALLY hate that hg spams my disk with tons and tons of files within the .hg directory. Ever cloned e.g. the mozilla hg repos? ugh. Bonus points for their fancy name escaping mechanism in those files, which had me run into "path too long" issues on windows boxes a couple of times already.
I would rather like a new option which was designed as a VCS from day one, that is user friendly and fast. Fossil is actually nearly there (I used it for a bit for some private projects)
Hmm? It’s just a directed graph of SHA1s under the hood. Seems pretty simple to me once you understand that. My understanding was that Hg’s data model is actually way more complicated with more pointers.
As everyone else is chiming in, the reason git won was speed. I haven’t used mercurial in a few years but at the time when I was looking to replace SVN, git did everything seconds faster than Hg which made it the clear winner.
It wasn't just speed, it was also that the Linux kernel was developed under git, that Linus created git, and the existence of github, which a lot of people liked.
After a while, it also just developed a critical mass where the attitude of people/companies that weren't using it almost inevitably became one of "everybody else is using git, so we should too".
The Mercurial alternatives like bitbucket just didn't have the same spread, and we got stuck with year after year of teaching new people a difficult interface.
If not for that difference we’d all think of git as that bizarre source control Linus makes the Linux devs use.
GitHub was then mirroring open source projects git repos without asking. (Which I think is fine, but ruffled some feathers back then)
GitHub could have easily been hghub, but they targeted git because it was already winning.
Then again, if I'm working with people in different areas, and want them to have a full reproducible copy of the repo, git it is.
I maintain a "use the right tool for the right job". Sometimes the Cathedral wins out, and other times the Bazaar wins out (NO! not the Bazaar source control!).
SVN seemed like it didn't go far enough to be honest. It wanted to be "atomic CVS", but there were many long-standing issues with SVN.
In this case the reason is most definitely the fact that it used in Linux, easily one of the biggest open source projects ever. I would wager a guess that if Mercurial was created earlier, Git would probably never have been created, let alone be adapted for Linux.
Both have pretty terrible UI but so long as one uses magit, git comes out way on top.
The data models are different and suffer different problems. A main issue with git is that it is stupid about file copies and renames. An issue with hg is that it doesn’t work well with long running forked histories (i.e. like git branches) because it stores the set of revisions of a file as a list of blocks of “complete file” or “diff from previous version in this list”
Both have scaling problems to large repos and algorithm/data structure problems which cause too many operations to be e.g. O(size of history) at least. I suppose this is better than Darcs’ model of “commit on Friday and hopefully it will be done by Monday.” If hg we’re naturally good at scaling than e.g. Facebook wouldn’t be putting so much effort into trying to make it scale (e.g. using inotify instead of looking throughout the tree for changes (which I think shouldn’t count as any vcs gains from this), having a mergeless history, rewriting a ton of hg in rust (git was always partly in C and there is now also libgit2)).
The thing that makes me most sad about hg is the lack of a really good (ie good and emacs-based) ui.
I’m all for different vc systems being developed and I think it would be good to see some real innovation potentially break up the current git-hg hegemony.
I think there are lots of good things about fossil (e.g. using an actual database that is going to scale well and avoid data corruption instead of using a specialised data structure that is hard to change and likely not so corruption resistant or battle tested or scalable but maybe let’s your data structure be “faster” for certain operations)
Another interesting vc system being developed at the moment is pijul which can be simply described as “like darcs but fast and more likely to be correct”. It feels a bit like it’s fitting in with the current trend of CRDTs, although it’s core data structure is not a CRDT as that would imply that all merges have some deterministic resolution (ie merge conflicts do not happen) and that is not the case, instead files are allowed to be merged into a first-class conflicted state which can then be resolved by later patches.
Could you elaborate on this? As far as I know, file copies and renames will still use the same blob, but the tree referencing the blob can reference it using a different path in the case of a rename or reference it more than once in the case of a copy.
If you merge this with a patch that changed foo then hg will do something sensible (ie either merge the changes into bar or give a merge conflict).
Git has no first-class concept of file name changes. Instead it tries to use heuristics to spot renames and sometimes they work and sometimes they won’t. Maybe if you merge a patch renaming foo to bar with one that changes foo the second patch will be applied to bar, but maybe it will behave as if you are merging changes to foo with deletion of foo.
Merging is already hard, dangerous, and non associative. The danger is less that you get lots of annoying merge conflicts than that you don’t get a merge conflict when you should (and therefore you risk accidentally changing the meaning of the merged files without knowing), e.g. if you merge “rename foo to bar” with “delete foo” and git didn’t spot the move then the merge might leave bar untouched when really there should be a conflict between keeping/deleting bar. Having wrong merges happen automatically can be a big risk when software is supposed to be very reliable.
Git has the "mv" command. If you "git mv" a file, why would git have to guess or use heuristics to figure out that the file was renamed?
In fact, after a "git mv foo bar", if you do a "git status", you'll see:
On branch master
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
renamed: foo -> bar
> Git has a rename command git mv, but that is just for convenience. The effect is indistinguishable from removing the file and adding another with different name and the same content.
git diff, merge, and related tools have heuristics for detecting file moves (off by default, turned on with e.g. git diff -M) but they tend to break if a file is both moved and modified in the same commit.
[0] https://git.wiki.kernel.org/index.php/GitFaq#Why_does_Git_no...
If you only push commits to a branch and use GUI tools like Github or Gitea for merging, chances are you're never going to be exposed to anything more complicated than 'add; commit; push'.
Even merging isn't that terrible and while still being painful, git makes it somewhat clear what you want.
The problem is; if you want people to use something else, you need to improve over git in the areas that matter to most people (ie, 'add; commit; push').
Git is terrible but just good enough that improvement over that terrible will be hard to accept for the mainstream.
I have been using git since it was released and have never needed to think about its underlying data model in order to accomplish dev tasks. Also I suspect that any complexity in the data model was quite necessary to implement its api and features in a performant way.
I actually like the git UI. Like many git power users, I've come to terms with a subset of the UI that I know how to use very well. The thinness of Git's abstractions lets me think of complex VCS operations as I do when reading or writing code. If the cognitive load is too high, you can always just use a merge-heavy workflow and stop thinking about the mechanics, but I recommend instead to understand the mechanics.
The data model in Git is hardly more complex than Fossil's or Mercurial's, and it's copy-on-write all the way, which makes it very safe (think ZFS).
mercurial wasn't superior.