Working with Git Worktrees in Magit | Hacker News Reader

Working with Git Worktrees in Magit

emacsredux.com

147 pointssrijan452 comments

To be honest, AI or not, I just don't like to be in the situation where I need multiple checkouts of the same repository. I do it sometimes anyways, but it's not because I'd like to work like that, rather, because something (bad) happened and it needs fixing, and having an extra checkout would help fixing it.

Some things that the post indicates as the problems solved by worktrees to me feel like the (quite common) problems of development environment setup. For example, OP uses worktrees to prevent different tools from competing over files in the working directory of their development environment... Well, a better way to run tests is to not run them from the development environment at all: deploy them to wherever they are supposed to run, and run them there.

Unfortunately, a lot of tools are designed to run code from "dirty" environment by default, take for instance Python developer's love for "pip install -e" or pytest running code from the source directory by default. But, for one's own sanity, these practices are better avoided. Running from the working directory of the development environment means that you, as a developer, have to remember every detail of the state left by the previous run and reassure yourself that none of those changes matter to your next run... I don't trust myself to remember that.

I can use a worktree to automatically bisect through so many commits that it takes hours. Of course I’m gonna use a worktree or a clone for that rather than let it use “my” worktree. Or I could run tests for a certain setup. Or for another setup. There might be many test setups which test different things.

But I guess to someone those wants indicates “problems” and impurity. “The root problem is that you need to use bisect (i.e. you have bugs).”

A slight limitation of worktrees is that you can only have a branch checked out once in all of your worktrees. I guess that's just how git works.

But when I want to checkout a branch that's already checked out in another worktree, Magit just plain refuses. I wish it would offer me to either:

1. Switch to that worktree instead

2. Detach head in the other worktree and checkout here

Would be such an improvement! No excuses though, I love magit and might as well hack this myself. I'm just bad at elisp.

> I guess that's just how git works.

You can use `--force`:

> If <branch> does exist, it will be checked out in the new worktree, if it’s not checked out anywhere else, otherwise the command will refuse to create the worktree (unless --force is used).

Or "--ignore-other-worktrees" for "git switch".

This leaves things in a weird state though, as modifying the branch from one of the worktrees will effectively change the ref the other is pointing to, but won't update the other's working tree, making it look like it's preparing to revert all the changes that the first worktree made.

(obligatory mention of jj, whose workspaces track what state they were snapshotted at, thus never losing what the true diff of a workspace should be, allowing a "jj workspace update-stale" to safely update the working copy with whatever the other workspaces changed (if you change the working copy and in parallel from another workspace change what that workspace points to, you'll get a saved commit of the working copy changes before being updated, which you'll need to squash/rebase wherever those changes were needed))

I wonder what use case there would be to checkout the same branch in multiple worktrees.

Or you aren’t saying this is a wanted feature, simply that magit should switch to the worktree (like lazitgit behavior)?

> I wonder what use case there would be to checkout the same branch in multiple worktrees.

I've run into it when I've wanted to checkout that branch (to try running code there, or view a file, or rebase it, or whatever) and there's already a worktree checked out.

This is more like treating worktrees as "lightweight clone of the repo checkout out at some dir". (If it were a separate clone, it wouldn't be an issue to have the same branch checked out in two places).

Just use copy-on-write clones. They're way more flexible, faster and easier to reason about.

I don't get spending all this time on tooling that tries to ease working with worktrees when they're just not the right abstraction for the way we're working now.

do these automatically copy over git config and/or installed hooks?

Yep. It's a full copy of the state of the origin

You are not referring to to a specific git feature, right?

You use your filesystem ability to perform a snapshot of your local git repo? Or you do a cp -reflink?

How do you handle the exposure of secrets to agents?

One thing I like about worktrees is that you get a clean copy (with share git objects though), so you have to copy over what the agent will need, not remove what you don’t want the agent to see.

This sounds really cool, can you explain how you do this?

Assuming you're using a filesystem that supports it like ZFS, Btrfs, XFS, etc, it's as simple as:

  cp -R --reflink=always /path/to/source /path/to/dest

On macOS with APFS:

  cp -R -c /path/to/source /path/to/dest

That's it. You get a copy that only stores additional space for metadata, not the files themselves.

AFAIR you can just git clone ../path/to/other/local/repo/.git and it'll use hardlinks, so, basically a copy on write

Doesn’t that set the origin to the local repo instead of upstream?

Also, it’s not really CoW. The immutable object store is hardlinked (since it’s immutable it doesn’t really CoW, but that’s not a very important distinction). But working files are full copies, and so is the rest of .git. With a CoW filesystem you can CoW everything, including node_modules and build artifacts.

I looked up `-c` on macOS. It says it causes cp to use clonefile(2) instead of copyfile. So I looked up clonefile(2). It says:

    NAME
         clonefile – create copy on write clones of files
    
    SYNOPSIS
    ...
    
    LIMITATIONS
         Cloning directories with these functions is strongly discouraged.  Use copyfile(3) to clone directories instead.

But no explanation of why strongly discouraged.

You get other things like mutual exclusion, a clean baseline regardless of whats in the main directory, and potentially in the near future a way to tie into lifecycle hooks for eg creation/deletion [1]. That last part is nice in the new agentic AI era where you might want to setup/teardown local services that are isolated to those directories.

But yeah the filesystem cost is not ideal. I was just thinking about how to exclude my worktrees from timemachine backups automatically.

[1] https://lore.kernel.org/git/7c8b4673-37ac-45fa-ad8c-a1dc09af...

They are the right abstraction for people that do not want to relocate terabytes of their existing data and filesystem structure to migrate to some sort of an esoteric filesystem just so they can let a tool work "properly" according to some tech purists. I don't wanna worry about it at all, it just works behind the scenes.

The problem I have with worktrees is that if I want to test the changes I need to juggle the worktrees.

It works better for me if I have multiple copies of the repo with their own supporting environment like repo-a, repo-b, etc. going to app-a.foo.localhost, app-b.foo.localhost, etc.

Aren't worktrees multiple copies? I usually have multiple worktrees and each one has different ports the various apps and servers talk to so I can easily visit each worktree's instance to test while having them all run.

As the author, I have been using worktrees due to AI agents. And I love Magit but didn’t know it handles git worktrees so well! Thanks for letting me know, will start using magit to manage my worktrees immediately.

Worktrees aren't about filesystem cost. A clean copy that shares objects is what you want when handing repo access to agents and secrets live in the main tree.

Like many of you I also knew what git worktrees were but always found them clunky because you mentally had to manage worktrees AND branches at the same time. That's why I didn't bother using them.

A couple of weeks ago I discovered Worktrunk. That's when my whole workflow flipped 180 degrees: I now always use worktrees, love it, will never go back. Worktrunk was the missing piece for me. Very recommended.

Switch to worktrunk also.

The hooks system is great.

The UX is great (wt list, wt switch —create …)

It is magnificent.

Thanks for the WorkTrunk recommendation!

I just installed it and loving it so far.

One thing I've desperately wanted from git worktrees is to use reflinking in some way to lower the cost of copies.

I just found https://github.com/josharian/git-cow-worktree that doesn't inspire confidence.

You could try proposing a worktree copy command on the mailing list. Worktrees are currently independent and work off of repositories which may have no existing worktree / working copy at all, let alone one it makes sense to start from, so there is bothing to copy but for dodgy heuristics.

I recently open sourced the worktree wrapper we use at my workplace, which uses reflinking in this way.

https://github.com/Applied-Intuition-Open-Source/stol

There are many uses for worktrees for people who are more than meat proxies:

https://stackoverflow.com/questions/31935776/what-would-i-us...

I use grm when I need worktrees, which makes admin a bit easier; https://github.com/hakoerber/git-repo-manager

Worktrees are a must if you work with monorepos or just multiple branches in general. Lack of conventions (or rather the braindead defaults) and having to explicitly type out everything is what hinders adoption the most.

I've solved this by running thin bash script wrappers that keep the workflow SVN-like with a root directory that contains per-branch directories and a hidden bare repo).

I think my layout is similar to yours. What do your scripts do?

My branches end up in a tree structure (no shit!), and I rebase and merge up stream as changes land. I guess it could be more automated, but the only tedious part is remembering to remove old worktrees and prune the old branches

worktree clone: Basically git clone that initializes a worktree root according to this guide: https://dev.to/metal3d/git-worktree-like-a-boss-2j1b

worktree add: `git worktree add` that creates a local new branch and a directory with the same name and sets upstream to match; alternatively checks out an existing remote branch.

worktree rm: Removes worktree directory, first checking it is in porcelain state. Then prunes them and removes local branch pointers if they match the remote ones.

These all are handwritten and probably buggy, but still less prone to errors than typing out `git worktree add -b foo-123-fix-missing-semicolon foo-123-fix-missing-semicolon origin/develop` manually every time. One pass with Claude or Codex would probably do wonders.

I'm currently very much on the jujutsu train as the author also mentioned, i do recommend y'all give it a try

Sorry git worktrees are a waste of time Just keep multiple copies of the repo

I don't understand why worktrees either, but to each their own.

For my purposes, the alternative with just switching branches or if parallelism is needed, full repos and remotes (including local path remotes) is just much more flexible. Clone local, branch it, optionally do anything later: push a branch back to the local repo, push it to a remote, delete local repo done.

The actual code and clean merges is much more significant than worktrees vs full clones

Worktrees are significantly faster to use and checkout than clones. They also use much less disk space, and obviously commits made in one worktree do not require git plumbing to get them visible in another etc.

They don’t use less disk space as long as you account for that correctly: clones within a file system use hardlinks for the object store, so if you du on a clone it’ll be however many megs / gigs S but if you do that encompassing n such clones they will only use a bit more than S for the object stores. And cross file system you can use “--shared” although that will break the clone if you remove the source (then again the same occurs with worktrees). It’s also not significantly faster: on a fairly hefty monorepo (a few gigs, working copy is 1.2G) a worktree takes 4.76s, a clone takes 4.94. The clone’s .git “is” 3.4G, but that falls to 5.8MB when du-ing both the source repository and the clone. Of course if you create every clone from the base repo it’s going to be slow as fuck and eat all your disk, but you can not do that.

The rest I agree with, that objects and branches are shared between worktrees, as well as the repository configuration (e.g. remotes) is why I use worktrees, having to move things around and losing the content when I’d delete the wrong clone is why I stopped using them.

Hardlinks are the wrong abstraction. You modify one copy and it changes on all linked paths. You want reflinks instead.

edit: spellchecker corrected reflinks, this is what I meant.

I believe you're wrong here. If you modify a hardlinked file, the FS will create new inodes under the hood, leaving existing ones intact. So it's like CoW. At least on ext4.

// Didn't double-check, but I've held this belief for over decade now, would be surprised to be proven wrong

Changing a hardlinked file changes it everywhere, unless your specific editor happens to create a new inode. Thus it depends on your editor.

CoW is similar except the new inode is created by the file system itself and is independent of the editor.

No this is completely incorrect. The filesystem doesn’t break a link just if you open it. Your editor or higher level utility is free to do so (by unlinking and copying the data into a new file/inode), but this is not how the filesystem works.

You're about to be very surprised!

  echo old > a
  ln a b
  echo new > a  
  cat a b                   # prints "new" and "new"

Some programs like GNU sed [1] do create new inodes, but that is a property of the application, not the filesystem:

  echo old > a
  ln a b
  sed -i 'c new' a
  cat a b                   # prints "new" and "old"

With reflinks in a CoW filesystem the inodes are different from the start, but share the same underlying blocks:

  echo old > a
  cp --reflink=always a b
  ls -i a b                 # prints different inodes
  echo new > a
  cat a b                   # prints "new" and "old"

[1] https://www.gnu.org/software/sed/manual/html_node/Command_00...

Cool, I love surprises :D

Thanks for examples and especially the doc link. I don't normally use hard links, and for some reason was sure that's how they behave.

Maybe my belief indeed stemmed from observing software which overwrites opened file with a new copy.

The difference here is that git objects don't change. When you do a `git gc`, you do get new objects at new inodes, but they don't get written to the existing ones. Try this out and you'll see local git clones do indeed create hardlinks:

``` cd $(mktemp -d) git init test ( cd test; touch README.md; git add README.md; git commit -m "Test"; ) git clone test/.git test_clone find test_clone/.git/objects test/.git/objects -type f -printf "%i\t%p\n" | sort ```

You'll see the same inodes used in both clones.

Yeah, but this only works for the immutable object store. The rest of the tree is still copied.

CoW and reflinks let you share everything, including the non-immutable paths, and even stuff like node_modules.

Not relevant to comparing clones and worktrees, both create a fresh working copy from the repository data, and object store files are not modified in place (or at all really).

For anyone unfamiliar with this strategy: it's literally just "clone from a /path".

> Tools like Claude Code create a worktree for each task, so several agents (or several sessions of the same agent) can work on the same repo in parallel without stepping on each other

Isn’t this unsafe and can cause corruption? https://github.com/kaeawc/auto-worktree/issues/176

Jujutsu probably fixes this;:

https://gist.github.com/ruvnet/60e5749c934077c7040ab32b54253...

An overspill of AI slop.

Frustrating because JJ does have workspaces which are the comparison you’d actually want to make https://www.jj-vcs.dev/latest/working-copy/#workspaces

Reply on news.ycombinator.com