Stacked Git – manage commits as a stack of patches
stacked-git.github.io
stacked-git.github.io
Basically, having a stack of patches as commits in my local branch. Edits to older commits saved via fixup commits, and occasionally merging in the fixups and/or rearranging the commits with rebase when its convenient to have a different commit on top of the stack or when I'm pushing a patch out from the bottom of the stack.
To compare: I often use the workflow you described. But large PRs, I might rebase, and then rewrite my entire history as a clean series of commits, each incremental and easily reviewable.
Most often, the commits while working and the commits at the end look different, but the final presentation of a series of clean commits makes life easy for reviewers.
But I find most people on github end up reviewing the whole PR as one unless we all are on the same page ahead of time. So despite supporting the functionality it's not all that highly used compared to, say, the way individual patches in a set might be reviewed on the Linux mailing lists during later revisions. I feel like on github it's more common to see asks to break up PRs (maybe just as a result of the projects I've seen? both approaches have valid usecases imo, and there are good reasons to break up PRs which can be done in single commits anyway).
I still feel it's good hygiene even if reviewers don't look :)
On GitHub it tends to push you towards smaller and smaller PRs, which is generally good, however I kinda miss the "full context" of having a larger change on a single PR with well thought out commits
Reordering patches with StGit is trivial with `stg push` and `stg pop`. Arbitrary patches can be combined with `stg squash`. A modification in the work tree can be incorporated into the top-most patch with `stg refresh` or an arbitrary patch in the stack with `stg refresh --patch <patchname>`.
In other words, StGit provides a command vocabulary for performing these kind of patch-stack manipulations in a first-class manner.
But as you note, everything StGit does can be accomplished with plain git--albeit with more or less difficulty depending on the particular stack manipulation goal.
> slightly nicer interface to interactive rebase
More accurate to say StGit is an alternative interface that does some of the same things as interactive rebase (and more). For me, part of StGit's value proposition is that it is not interactive. A single imperative StGit command may accomplish as much as an interactive rebase session.
Workflows using `git rebase --autosquash` feel more similar to StGit because of their non-interactivity.
A sibling comment mentions interactive rebase. I had never thought about it this way but if stacked got gives you a command vocabulary / imperative interface to adjusting your local commits, I would say `git rebase -i` gives you a declarative interface.
I prefer the latter; it's easier for me to prepare a bunch of modifications in an emacs buffer / vim tab and then have git go and do them all at once rather than try to maintain the state of the commits in my head and plan out the commands one at a time to achieve the same result.
Others prefer to work their way through a piece of code for example from top to bottom, making opportunistic edits along the way, even if unrelated to the task. Not having to switch branches can be convenient in that case.
I’m 100% in the latter team. Near the end of a coding session, my `git diff` often shows three or four completely unrelated sets of changes. Some might call that a rather unproductive and annoying way to work. But I really love it that way. My brain just _craves_ to fix that typo or convert that tab to spaces right away, even when I’m supposed to fix a bug right now. I love dealing with those things right away so I get them out of my mind and don’t have to put them on a mental stack — even if that means a little more chaos later when it’s time to bin my edits into different commits.
Arguably, I _could_ switch branches for such edits. But frankly, I’m used to opportunistic editing, and sticking with that approach feels just super convenient.
I just don’t switch branches while coding.
Rebases and merges use diffs behind the scenes, although git stores the commits as snapshots. There is always a conflict of the two mental models while rebasing. With StGit, it's explicitly just patches that you can inspect anytime. I assume that this is the reason why StGit operations make more sense than rebasing.
Another important fact is that you don't need to wait till a feature-branch is complete to edit the history. With StGit, you can craft the ideal commit history along the way while you develop. I treat a stack of patches as multiple staging areas - each with specific purpose/feature and a predefined commit message. That way, you can develop code hunks in whatever order that makes sense, and then check them into (refresh the patch) the appropriate patch. It's even possible to edit or reorder these patches as we progress. Finally, the patches can be committed when the series of features are complete.
Why would you need to wait until it's complete?
I don't feel any need to do that, and I'm constantly rebasing. I guess I must be something about your workflow here.
At any time, I can choose to consolidate a few of my WIP commits into a more (logically) cohesive commit and keep that around for further rebasing.
I must be missing something here, but I'm not sure what.
(If it helps understanding, we work with Gerrit which basically expects a 'list of patches' on top of the branch you want to commit your changes to. So that's already part of our mentality, but we just use plain git.)
For example:
git commit -m "one thing"
# commit hash: deadbeef
git commit -m "other thing"
# commit hash: badf00d
git commit --fixup deadbeef
git commit --fixup badf00d
git rebase -i --autosquash
But I’m not sure I’d take on a new tool dependency just for that. What are other benefits of Stacked Git? Just wondering if I’m missing something.
Generally, it sounds like people are simulated Phab with these tools which is trying to simulate the email patch workflow.
Gitlab-runner can execute any shell commands, so it’s easy to just loop through all the commits and run bazel test.
Gitlab has improved lately and it’s now easier to review each commit in a MR, but you can sense that most of the PM attention is on the single commit/squash everything MR workflow.
I'd say if you wanted to do these kinds of stack manipulations a little bit faster or with a little less cognitive overhead (e.g. named patches vs SHA1 hashes), then investing in StGit might pay off.
I generally would agree that using StGit isn't going to be a slam-dunk win for someone who already has a capable workflow using interactive git rebase.
git commit --fixup :/fooYeah, dance is good analogy. Lot of training, one little mistake and it is ruined.
For example, if you need to introduce a change in commit "one thing" which would require merge conflict resolving in "other thing", with stgit it would be much easier than with fixup/autosquash.
Or another example. You need to put away "other thing" for a while, and work with "one thing" for an hour, and then apply/fix "other thing". Another set of git tricks needed.
> But I’m not sure I’d take on a new tool dependency just for that.
If you are happy with git, nobody forces you to.
By the way, you don't have to use stgit all the time. I usually just use git, but when a change in a branch requires changing of previous commit, I simply do: stg init; stg uncommit; stg pop. It's easy to convert local repo state to and from stgit.
It's a great workflow that makes you look like a genius who rarely makes mistakes. Absolutely vital if you need to maintain a set of patches against a bunch of branched source code (e.g. maintaining an open source driver with variants upstream (linus's plus stable kernels) and also for various distro kernels.) This was my main use case for years, but I still use it even for things that don't fit that pattern, just because it's convenient to do so.
Sure everything that stgit does can be done with git by itself (duh, that's what stgit is doing under the hood), but I find it's a lot easier with stgit.
[1] http://savannah.nongnu.org/projects/quilt [2] https://lwn.net/Articles/13518/
I'd love to add functionality that mimics `git rebase -i`. That is, you would open an editor and be able to select which patches you want on your stack as well as possibly designate patches as 'squash' or 'fix' from your editor. Think of it as `stg sink`, but able to operate on multiple patches at once.
Prior art: this script[1] already performs a re-ordering of commits but in a pretty hacky way. I'd like to productize it!
I'd love to have this new `stg rebase --interactive` be part of the main repo to enjoy the benefits of the existing test suite. My question for you is around how to include the new command with the rest of the tools. Would you want it to integrate with the existing rebase command (`stg rebase --interactive`) or is it something more appropriate for `contrib` (so a new independent command like `stg-rebase-interactive`)?
[1] https://github.com/da-x/misc-gitology/blob/master/stg-rebase...
I think I get the gist of what this interactive script does and how you propose to extend it. A PR for such a script to go into StGit's contrib directory would be non-controversial. Maybe a little higher bar to have this capability as a first-class StGit command.
Would you mind expanding upon this? I'm really curious.
Have you tried Pijul too?
# coding some-task
# critical bug approaches
$ stg refresh; stg pop # save current changes and pop patch
$ stg new -m 'fix of critical bug' # start working on new patch
# fix fix fix
$ stg refresh; git review # commit fix
$ stg push # bring some-task to the table again
# continue to coding some-task
I have also several git worktree's to facilitate working on multiple things at once though, so I don't necessarily need to switch away in my current repo, cause I just go to one of my 4 worktrees that I always have. As I explain this out loud though, it does sound like a very stopgap solution!
Not GP, but I share that sentiment. Let me add my two cents. Here are some of the things that confuse me while doing interactive rebases in git:
1. What happens when I delete a commit? Won't the subsequent commits still hold on to its changes, since git uses snapshots? (Nope, that's not how it works)
2. Why does a rebase conflict happen when we try to squash or reorder a few commits? Yes, we can see the original commits, but we have wrack our brains to understand what trips the algorithm.
Git internally uses snapshots to store commits, but uses diffs for rebasing, merging etc. Knowledge of that fact reduces the confusion a bit, but doesn't relieve the cognitive overload. With StGit, they are clearly just patches that you can inspect any time. The reason for any conflicts or results are immediately apparent when you see the patch - much the same way you know why a program misbehaves when you see the source code.
> Have you tried Pijul too?
I have tried Pijul, but didn't stick with it. It didn't feel mature enough yet. However, I sure am going to keep trying until it feels right. What excites me about Pijul is that the author stresses the advantage of patches over snapshots when it comes to merging and rebasing (both are the same in pijul). If the combination of git and stgit is any indication, pijul is likely to be way more powerful. I am excited about its future!
Pijul excites me too in this regard! I'm trying to use it concurrently with git in local projects, but perhaps stgit would be a good in-between solution.
---
A question for the maintainers...I was enjoying the command `stg publish` to allow myself to keep refreshing patches locally but every so often push to a branch that others non-stg users were pulling and pushing to. The command was removed in 1.0.
I'm curious why it was deprecated and removed and if there is a good replacement flow for working with non-stg users on branches I can't force push to.
I removed it because it was challenging to test and maintain, its semantics are somewhat complex, and it wasn't clear that it provided value to anyone.
If you wouldn't mind making your case on the issue tracker [1], I will reconsider the fate of `stg publish`.
cat ~/bin/stg-update
#!/bin/bash
set -e
current=$(stg top)
git push -f origin HEAD:$current
Definitely cumbersome to use, but it had fair reasons to be cumbersome.
StGit does not support that as I understand.
I've been using StGit for more than 10 years. And I want to thank authors to free me from `git rebase` and `git stash` and branch switching burden. StGit is a so very fun part of my everyday job!
Are tools like git-absorb safe/reliable? Git-absorb is a port of hg absorb:
"Essentially, when your working directory has uncommitted changes on top of draft changesets, you can run `hg absorb` and the uncommitted modifications are automagically folded ("absorbed") into the appropriate draft ancestor changesets. This is essentially doing `hg histedit` + "roll" actions without having to make a commit or manually make history modification rules."
I haven't (yet) wrapped my head around the algorithm. I get that an algorithm can "recollate" a series of commits in a way that yields no commit conflicts, but that's not the same as rearranging and combining commits into a sequence of semantically coherent atomic commits.
---
Also, what a shameless plug I am, have a look at git-crecord to interactively commit files line by line.
Some days I wonder if an interactive rebase oriented workflow would be good enough for me, but manipulating the patch-stack with StGit has become so second-nature that even the small amount of extra friction of `git rebase -i` sends me right back to StGit.
Thank you for maintaining StGit — I really enjoy it as a tool! topgit is fine, but quite heavy. I don't use stg much at work with squash merge workflows, but I love it for personal projects or when I need to work on a series of commits with intricate change sequencing.
Although I imagine you'd still run into problems when you push a commit you've popped and there's a merge conflict.
Git packs, however, don't crash on rebases or commit changes, and can be merged whereas a stack of patches would likely fail in the beginning and then is inefficient to work with if other patches rely on those changes.
Maybe someone can explain why a stack of patches might be better if you need to use git anyways?
You are correct that conflicts can occur when pushing or reordering patches. These conflicts are resolved with the regular git mechanisms. I.e. conflict markers in files, use `git add` to mark conflicts as resolved, and then `stg refresh` to update the patch with conflicts resolved.
Can someone explain why this is useful? This means I can maintain a set of "private" changes to a branch, say, better log messages, have this applied to the working tree, do non related changes in a feature, and commit my work without having to deal with the log message's lines?
But I really don't think this is better than `git add -p`.
https://git-scm.com/book/en/v2/Git-Tools-Interactive-Staging
Does stacked git help with this?
There's no magic in resolving merge conflicts, stgit helps only to some degree.
For example, stgit allows you to not fix all conflicts at once. You have a stack, and you may have first commit in a stack rebased, have a nice usable repo state, and return to rebasing of the second commit tomorrow.
I actually started with quilt, and when I discovered stgit (many years ago), it was a huge relief.
StGit uses push/pop in the sense of pushing and popping patches to/from the patch stack. An alternative such as apply/unapply would be valid and could be aliased, but this ship sailed 16 years ago (StGit has been around almost as long as Git) and so is unlikely to change.