Take a look at Fossil too:
https://www.fossil-scm.org/
It's the distributed version control (and more) used by SQLite. Most people have no idea about how cool the SQLite ecosystem is, and how it's used even on avionics!
It's the distributed version control (and more) used by SQLite. Most people have no idea about how cool the SQLite ecosystem is, and how it's used even on avionics!
No, having a commit "fix typo" in the main branch's history is not at all useful and won't ever be. It's noise.
In a work setting it's much better to reduce noise.
The difference is that Fossil does not promote the use of commit-squashing. While it can be done, it takes a little work and knowledge of the system. Consider the premature-merge problem in which a feature branch is merged into trunk before it is ready, and subsequent typo fixes need to be added. To do this in Fossil you first move the errant merge onto a new branch (accomplished by adding a tag to the merge check-in) then fix the typo on the original feature branch, then merge again. So in Fossil it is a multi-step process. Fossil does not have a "rebase" command to do all that in one convenient step. Also, Fossil preserves the original errant check-in on the error branch, rather than just "disappearing" the check-in as Git tends to do.
The difference here is a question of priorities. What is more important to you, an accurate history or a clean history that tells a story? Fossil prioritizes truth over beauty. If you prefer a retouched or "photoshopped" history over an auditable record of what really happened, Fossil might not be the right choice for you.
To put it another way, Fossil can squash commits, but another system might work better for you if commit-squashing is your go-to method of dealing with configuration management problems.
That isn't the normal use case for commit squashing though. Generally, trunk/master/main isn't ever rewritten. Squashing is usually done on feature branches _before_ merging. What does that look like in fossil?
It seems like part of the problem is that fossil is designed for a very different workflow. See https://www.fossil-scm.org/home/doc/trunk/www/fossil-v-git.w.... The autosync, don't commit until it is ready to be merged workflow might work well for a small flat organization, but I'm not sure how that scales to large organizations that have different privilege levels, formal review requirements, and hundreds or thousands of contributors.
In terms of Git and the usual commercial SCM practices, I'm speaking empirically. Everywhere I worked in a team, leaders and managers wanted main branch's history to be a bird's-eye view, to have every commit fully build in CI/CD, and be able to find who introduced a problem (admittedly this requires a little more digging compared to Fossil, though). Squashed commits help when auditing for production breakages, and apparently also helps managers do release management (as well as billing customers sometimes).
Do I have all sorts of minor commits in my own projects? Sure! I actually pondered using fossil for them but alas, learning new tools just never gets enough priority due to busy life. I'd love to learn and use it one day. I'm sick of Git.
But I don't think your analogy with a photoshopped / retouched picture is fair. Squashing PRs into a single commit is not done for aesthetic reasons or for deliberately disappearing information -- a link to the original PR with its branch and all commits in it remain and can be fully audited after all. No information actually disappeared.
I believe a better analogy would be with someone who prefers to have one big photo album that contains smaller albums which in turn contain actual photos of separate life events that are mostly (but not exactly) in chronological order -- as opposed to Fossil's approach which can be likened to a classic big photo album with all semantically unrelated photos put in strict chronological order.
I'll reiterate that my observations and opinions are mostly empirical. And let me say that I don't like Git at all. But the practice I described does help in a classic team of programmers and managers.
I concede that Git and its quirks represent a local maxima that absolutely can be improved upon, but at least to me the jury is still out on what's the better approach -- and I'm not sure a flat history is it.
I frequently use git rebase in interactive mode to rearrange and curate my commits to form whatever narrative I'm aiming for. Commits are semi-independent stories which can be merged, in order, at any rate and still make sense. Each commit makes sense with respect to history, but doesn't care about the future.
I squash and rearrange and fixup commits until they look they way I want, and would want to see if I was looking at a history, and then send them for review.
Whether you merge my branches, or fast-forward and rebase the individual patches, makes little difference to me. But please don't squash my hard work.
Not looking to pick a fight here, mind you, but the Linux kernel is hardly a representative demonstration oh how to consume Git out there in the wild.
The way you describe your usage it already seems you kinda sorta do your own squashed commits, only you want several of them get merged into the main branch, not just one. So you're still rewriting history a bit, no?
The problem with squashing at review time is that it is incompatible with my model. I'd rather teach engineers to not push shitty "fix the thing" or "oops" commits that are ultimately useless for me, the reviewer or reader. Just use git commit --fixup and git rebase --autosquash as $DEITY intended and force push away. It's your world, you'll be deleting the branch after you land it anyways.
The tool works so well when used as intended: a distributed version control system. The centralized model adds so much pain and suffering.
Squashing buys you next to nothing, and costs you the ability to dive into the history in greater detail.
I suppose if your project is truly huge, it becomes worth it to reduce load on your VCS, but beyond that...
The "greater detail" part can cost me a lot of time.
Sure it does, but sometimes that level of detail in history is not helpful. Individual keystrokes are an even finer/"more accurate" representation of history; but who wants that? At some point, having more granular detail becomes noise - the root of the disconnect is that people have a difference in opinion on which level that is: for some (like you), it's at individual commit-level. For others (like me), it's at merge-level: inspecting individual commits is like trying to parse someone's stream-of-consciousness garbage from 2 years ago. I really don't care to know you were "fixing a typo" in a0d353 on 2019-07-15 17:43:32, but your commit is just tripping-up my git-bisect for no good reason.
If the author did something edgy or hard-to-understand with the change-set, I expect to see an explanation why it was done that way as a comment near the code in question, rather than as a sequence of commit-messages, that is the last place I will look - but that's just me
I mean, granted, it's not ideal. I think this is a bit of a problem with the low-level nature of git - Ideally it'd be easier to semantically bundle such sequences of commits such that it's be more reliably dealt with in the broader ecosystem (not every tool supports --first-parent), and in any case, there's nothing forcing you to maintain the first-parent-is-linear-mainline-history; that's just a tradition which, again, many common tools follow. Then of course there's the poor integration with git hosting (such as github) and git - I can blame a file, but I can't easily correlate that with the discussions in the PRs, and whatever correlation there is is purely online, with all the limitations of a single-vendor non-distributed system like that entails.
Ideally this wouldn't even be a tradeoff at all; it would be obvious how to track history both at the small scale and the larger scale (and perhaps even more?), but alas, it's what we have.
Out of curiosity - when you merge via squash, what kind of commit messages do you retain? Do you mostly concatenate the commit messages, or rewrite the whole thing?
Context-specific. For a bigger PR that deals with an extensive refactor I'll prefer to have a descriptive title and hand-curated task list below (so definitely not 1:1 to commit messages). For smaller PRs -- or more focused ones, like those dealing with a single feature or bug -- I'll only leave a descriptive title.
But I usually never leave a list of commit messages. Not because I have no discipline; sometimes some refactoring requires 4-5 steps and all commits have 99% identical messages which is not useful when you aggregate those in a single list of bullet points in the end.
---
> But the problem with this approach is that you're making it impossible to extract the actual changes when you do want them, whereas simply skipping non-merge commits is a minor inconvenience (`--first-parent` tends to cover it).
Again, that's not the issue here. The issue is that when you work on a big project (like many of us do) you get something like 4-7 merged PRs a day; don't pull/fetch for 3 days and you'll get 60+ lines in your terminal when you get to it.
There are people who manage releases and people who chase subtle regressions. Having git bisect narrow it down to a big PR squashed commit is actually a win; it gives them a localized area inside which they can work with other tools (not bisect).
In the end I suppose we can say it's a subjective taste. But I always appreciated the main branch's history to only consist of squashed commits. Again, it gives you a good bird-eye's view.
You'd be surprised. It's only because we understand (and are used to) tool limitations (regarding storage, load, etc) that we don't advocate for that, not because some other way is philosophically better.
I'd absolutely like to have "every single version of the file which ever hit disk locally on my machine in the history".
I understand the rationale but the balance tilts too far into the "too much details" territory for me and that can slow me down while digging.
What I found most productive for myself is that searching for a problematic piece should happen on a two-tiered tree, not a flat list. What I mean is: first find the big squashed PR commit that introduces the problem, then dig in more details inside of it.
Not claiming my way is better but for almost 20 years of career I observed it was good for many others as well, so I am not exactly an aberration either.
To me a very detailed history is mostly a distraction. Sure `git-bisect` works best on such a detailed micro-history but that's a sacrifice I am willing to make. I first use bisect to find the problematic squashed commit and then work on its details until I narrow down the issue.
And while those histories are typically unreadable, it is possible to label (even retroactively) relevant moments in "history"; and in any case just because a consumer-level wordprocessor doesn't export the history in a practical way doesn't mean a technical VCS couldn't do better - it just means we can't take git-as-is or google-docs-as-is and add millions of tiny meaningless "commits" and hope it does anything useful.
Ideally, each commit should be something that you could submit as a stand-alone patch to a mailing list; whether it's a single commit that was perfect from the get-go or fifty that you had to re-order and rewrite twenty times does not matter at all; the final commit message should contain any necessary background information.
It would be needlessly restrictive to prevent users from making intermediate commits if that helps their workflow: I want to be able to use my source-code management tool locally in whichever way I please and what you see publicly does not need to have to have anything to do with my local workflow. Thus, being able to "rewrite history" is a necessary feature.
Indeed they are, hence the need to rewrite the history. Managers, tech leads, or users of your OSS project don't care about the "fix typo" comments. They are interested in a meaningful history that tells a bigger story.
And to be frank, I am interested in the same, mid-term and long-term. While I am grappling with a very interesting problem for a month then yes, I'd love my messy history! But after I nail the problem and introduce the feature I'll absolutely rewrite history so the squashed PR commit simply says "add feature X" or "fix bug Y".
Why would they be looking at the version control system for this? That is not what it's there for.
And many tech leads simply don't have the time to review every single WIP commit. They want meaningful message/description of the big squashed PR commit. If you just post a merged list of all commit messages with 10x "fix stuff" inside you'll be in big trouble the next time around and your work will be inspected very closely.
The practices I am describing to you are reality in many tech companies. Writing code there is not about you at all. And almost nobody will read your code and PR descriptions unless they really have to. Hence it's a professional courtesy to make those as small and meaningful as possible.
Do you usually review every commit? I usually just review the diff between the PR'd branch and master, as does everyone I work with.
Whether you think that "makes no sense" is inconsequential. Many people find it very meaningful.
Let alone the code review issue. In any multi-person project, it is just as important that your commit history be readable by a third party for information as that it be useful for your own personal debugging.
Compromises with human nature must be made. Hence -- we need to be able to rewrite history.
Your academic purism is out of place, dude. Real humans don't work like you say they do. Some do -- most don't.
By definition the actual history of your work includes WIP, so this means your commit history should not reflect the actual history of your work.
Just don't publish WIP crap; fortunately, you can have your cake and eat it too, with git.
The biggest reason git (and any similarly advanced SCM) is superior to non-distributed alternatives like Subversion is that I can use it to manage my own workflow, instead of just as the final off-site backup of whatever I decide to publish. I get to actually use everything git offers for shuffling commits and code around while coding.
Want to switch contexts quickly? git commit the whole thing and just switch a branch.
How about untangling a hairy merge? Do it piecemeal and commit when you're done with each bit; it's trivial to then undo mistakes, redo, combine or reorder stuff and you cannot lose any work by accident because git commits are immutable.
All of these features essentially require history rewriting; sure, you're free to rebrand and not call "store WIP state in repository" a commit even though it is one, but I would consider any SCM without these features nigh useless for most work.
Why?
But it absolutely should when you work in a team. I'll personally scold you if you waste my time with merging a 20 commit PR of which 10 commmits are "fix stuff" or "fix typo" or "oops forgot variable" etc. It won't pass code review.
You are free to disagree. I am only telling you how it is in many companies.
2. Waste of later coders' time when running git blame on a line when trying to figure out the purpose of code
3. Waste of later debuggers' time when they need to decide whether this error is the thing they're looking for, an unrelated error to bisect skip, or an unrelated error in possibly-related code that they need to manually fix and then re-test.
I think representing history correctly is best, and agree that “squashing buys you next to nothing” other than visually pleasant output. Clearer?
My favorite approach is rebase + non-ff merge. Best of both worlds.
You think the disadvantage (destroying history) is more important, but you can't say that it "buys you nothing".
I would say what you're describing is a break down in CI/CD and code review. How is code that is that broken getting into your default branch in the first place?
As to rebases to clean up history (and not just the PR itself)... personally, I don't think that's worth it. My experience with history like this is that it's relevant during review, and then around 95% of it is irrelevant - you may not know which 5 % are relevant beforehand, but it's always some small minority. It's worth cleaning up a PR for review, but not for posterity. And when it comes to review, I like commits like "pay respect to the linter gods" and the like, because they're easy to ignore, whereas if you touch code and reformat even slightly in one commit, it's often harder to skip the boring bits; to the point that I'll even intentionally commit poorly formatted code such that the diff is easy to read and then do the reformat later. Removing clear noise (as in code that changes back and forth and for no good reason) is of course nice, but it's easy to overdo; a few typo commits barely impact review-ability (imho), and rebases can and do introduce bugs - you must have encountered semantic merge conflicts before, and those are 10 times as bad with rebases, because they're generally silent (assuming you don't test each commit post-rebase), but leave the code in a really confusing situation, especially when people fix the final commit in the PR, but not the one were the semantic merge conflict was introduced, and laziness certainly encourages that.
It also depends on how proficient you are with merge conflicts and git chicanery. If you are; then the history is yours to reshape; but not everybody is, and then I'd rather review an honest history with some cruft, rather than a frankenstein history with odd seams and mismatched stuff in a commit that basically exists because "I kept on prodding git till it worked".
All due diligence is done there, not in the main branch.
The main branch only needs to have one big commit saying "merging PR #2169". If you need more details you'll go that PR/branch and get your info.
The "fix typo" commit being in the main branch buys you nothing. It's only useful in its separate branch.
Why not merge?
It gives you better high-level observability and a good bird's-eye view. And again -- if you need more details you can go and check all separate commits in the PR/branch anyway.
And finally, squashed commits are kind of atomic commits. Imagine history of three separate PRs happening at roughly the same time. And now all those commits are interspersed in the history of the main branch.
How is that useful or informative? It's chaos.
EDIT: my bad, I conflated merging with rebasing. Still, I prefer a single squashed commit for most of the reasons above, plus those of the other two commenters (useful git blame output and buildable history).
"Encrypt project's sensitive fields (#1234)"
With the number being a PR or an issue # (which does contain a link to the PR).
I do care about history in branches though. And many others do. I agree that it varies from team to team.
Also, in case it helps you in the future, `blame -wC` is what I use when doing blame; it ignores whitespace changes and tracks changes across files (changes happened before a rename, for example.)
I've come across "fix indentation" or "fix typo" commits where a bug was introduced, like someone accidentally comitted a change (maybe they were debugging something, or just accidentally modified it).
For example: I'm tracing a bug where a value isn't staying cached. I find a line of code DefaultCacheAge=10 (which looks way too short) and git blame shows the last change was modifying that value from 86400. What I do next will be very different if the commit message says "fix indentation" vs "added new foobar feature" or "reduced default cache time for (reason)".
As I replied to @SQLite above, I am not saying this is the optimal state of affairs -- not at all. But it's what is required everywhere I ever worked for 19.5 years (and similar output was desired when we worked with CVS and Subversion and it was harder to achieve there).
But I'll disagree this is lack of discipline. It's not that at all. It's a compromise between programmers, managers/supervisors, CTOs / directors of engineering, and release engineers. They want a bird's-eye view of the project in the main branch.
Sometimes those insights are just as useful as the packaged whole.
You surely aren't going to have 50 contributors all simultaneously working on the same mainline or feature (or have 50 working together at the same time if ever, if you do, I'd say that's poor project management). The reality is a portion of the developers work on this feature in this part of the code base, a few over here, they'll be on their own branches or lines, and everything will be fine.
This scenario where we have 50+ devs all crashing and bumping into each other rarely happens, if ever. I have personally only seen one instance of it happen in kernel development. And even then it was relatively straightforward to sort out.
To go further, in a hypothetical scenario where there is one feature and 50+ open source developers are all vying to push their patches in, there is still going to be one reference point to work off of, and reviewers are going to base everything off that. It's a sequential process, not concurrent.
And since that day, not one looked at Fossil the same way, at least not the same way they looked at git
https://www.mail-archive.com/fossil-users@lists.fossil-scm.o...
I can't tell for sure how much of impact this had on fossil's adoption, its hard to beat git no matter how good you are, but I think it was a bit hit
1) No, it didn't. Please read the part of the thread after Zed's initial panic attack.
2) To the best of our[1] knowledge, fossil itself has never caused a single byte of data loss. When fossil checks in new data, it reads that data back (in the same SQL transaction the data was written in) to ensure than it can read what it wrote with 100% fidelity, so it's nearly impossible to get corrupted data into fossil without going into the db and massaging it by hand. People have lost data by storing their only copy of a repository on failing/failed storage or on a network drive, but no software can protect against hardware failure and nobody in their right mind tries to maintain an active sqlite db over a network drive (plenty of people do it, despite the repeated warnings of anyone who knows anything about sqlite, and they have only themselves to blame when it goes pear shaped). Fossil makes syncing to/from a remote copy absolutely trivial, so any failure to regularly sync copies to a backup is end-user error.
[1] = the fossil developers.
> Let me put it this way. Suppose you’ve got Zed Shaw. No, wait, say you’ve got “a person.” (We’ll call this person “Hannah Montana” for the sake of this exercise.) And you look outside and this young teen sensation is yelling, throwing darts at your house and peeing in your mailbox. For reals. You can see it all. Your mailbox is soaked. Defiled. The flag is up.
> Now, stop and think about this. This is a very tough situation. This young lady has written one of THE premiere web servers in the whole wide world. Totally, insanely RFC complaint. They give it away on the street, but everyone knows its secretly worth like a thousand dollars. And there was nothing in that web server that hinted to these postal urinations.
Still an impressive bug – but the first rule of “my CVS has broken” is “stop running commands, and copy the CVS directory somewhere else”. (I've needed to do this to a .git twice.) Had he done this, he wouldn't've lost work. While he shouldn't've had to, “stop doing everything and take a read-only copy” is the first step when any database containing important data has corrupted.
I finally gave Fossil a serious try last year for a few months, just on my own but I don't think my opinions would change if I tried in a team setting. I still love the idea, but the execution is... ghetto. Serviceable, certainly, but ghetto is the best overarching description I have for it. You have to be willing to look past a lot of things, sure many of them are petty, in order to take it over dedicated polished services for the things it combines. And git+[choice of issue tracker]+[choice of forum]+[choice of wiki]+[choice of project website]+etc. is the real competition against Fossil, not git+nothing, so even if it was better on the pure version control bits it would still be a tough battle. (Not to mention the elephant git+github is a pretty good kitchen sink on its own if you don't want to think about choices and have something serviceable that's also a lot less ghetto.)
I also realized how much I love git's staging area concept once it was gone -- even when I had to use Perforce a lot, at least Perforce has the concept of pending changelists so you have something similar. I've never been a big fan of git's rebase, but it's also brought up a lot as a feature people are unwilling to give up, and I see the appeal. In summary, I think the adoption issue is just that people who do eventually give it a shot find usability issues/missing functionality they aren't willing to put up with.
I think that this is only true for some projects. For some people, the ease of self hosted setup (one executable) and the fact that you can change the documentation, edit code and close bugs offline is a big win that no centralized service can compete with.
For project management I'm less familiar with the options out there but I'd be surprised if there was nothing that gives a really stellar offline experience. I'd give a preemptive win to Fossil on the narrow aspect that your issue tracking changes can be synced and merged automatically with a collaborative server when you come back online, whereas if you stood up your own instance of Trac for instance I'm not sure if they have any support for syncing. If you're working by yourself, though, then there's no problem, Trac and many others work just like Fossil and stand up a local server (or are dedicated standalone programs) and work the same whether you're offline or online. But when I'm working solo I prefer low-tech over anything that resembles Jira (and I don't even really dislike Jira) -- I've played with https://github.com/dspinellis/git-issue as another offline/off-platform option but in my most recent ongoing solo project I'm quite happy with a super low-tech issues text file that has entries like (easy to make with https://github.com/dhruvasagar/vim-table-mode)
+--------------+
| Add thing |
+==============+
| Done whens / |
| other info |
+--------------+
> I just cloned your repo. Everything is working fine. Breath, Zed.
seems like he panicked and made it worse, then rage-quit
I understand both that he is and why he is such a polarizing figure. Just wanted to put my positive anecdote on the pile, since they seem to be a less common when he comes up in online comments.
My personal website, https://dbohdan.com/, is powered by Fossil. A year ago I was shopping for a wiki engine, didn't love any I looked at, and realized I could try something I was already familiar with: Fossil. It did take a few hacks to make it work how I wanted. The wiki lacks category and transclusion features and, at least for now [1], can't generate tables of contents. I've invented a simple notation for tags and generate a "tag page" [2] using a Tcl script [3]. The script runs every time I synchronize my local repository with dbohdan.com. The TOC is generated in IE11-compatible JavaScript in the reader's browser [4]. The redirects are in the Caddyfile (not in the repo). Maybe I'll migrate to a more full-featured wiki later [5], but I am enjoying this setup right now. I am happy I gave Fossil a try.
Fossil also has a built-in forum engine [6]. I am thinking of migrating a forum running on deprecated software to it.
Edit: My favorite music page and sitemap are generated on sync, too. [7] The sitemap uses Fossil's "unversioned content" feature to avoid polluting the timeline (commit history). [8]
-----
[1] In the forum thread https://fossil-scm.org/forum/forumpost/b635dc56cb?t=h DRH talks about implementing a server-side TOC.
[2] The page lists the tags and what pages are tagged with each. Tags on other pages link to their section of the tag page. https://dbohdan.com/wiki/special:tags.
[3] https://dbohdan.com/artifact/8297b54f5d
[4] https://dbohdan.com/artifact/d81bb60a0e
[5] PmWiki seems like a nice lightweight option—an order of magnitude less code than its closest competitor DokuWiki, very stable, and has a better page history view. Caveat: it is written in old school PHP. https://pmwiki.org/.
[6] https://fossil-scm.org/home/doc/trunk/www/forum.wiki
[7] https://dbohdan.com/wiki/music-links with https://dbohdan.com/artifact/053d0ff993, https://dbohdan.com/uv/sitemap.xml with https://dbohdan.com/artifact/c21444f7c9.
Just FYI: we recently improved the internals to be able to add propagating tags to wiki pages[1], so it will eventually be possible to use those to categorize/group your wiki pages. What's missing now is UIs which can make use of that feature. The CLI tag command can make use of them, but that doesn't help your UI much.
> ... and transclusion features
For the wiki it seems unlikely to me that transclusion will ever be a thing. It can hypothetically be done with the embedded docs feature if the fossil binary is built with "th1-docs" support, but, alas, we can't currently support propagating tags on file-level content. (i have an idea how it might be integrated, but figuring out whether or not it internally makes sense requires trying it out (and that doesn't have a high priority).)
As for transclusions, I don't expect Fossil to implement them. While something like https://www.pmwiki.org/wiki/PmWiki/IncludeOtherPages would be cool, it seems probably out of scope for Fossil.
Side note: I've been exploring the ecosystem around Fossil/SQlite as well. I've been working with Pikchr a lot recently as a way to create diagrams that can be version controlled. Because Pikchr is implemented as a single C file, I was able to compile using Emscripten as a WASM file, and embed that file in a single HTML page that gives me a "live editing" experience (basically I call a render method when the text area gets updated). The way these pieces of software are written minimizes dependencies and allows for them to be used in a huge variety of environments. I've really enjoyed working with them.