You can do this. You just need to use --detach.
When you check out a branch, your working directory is expected to be in sync with it. Internally the HEAD points to the branch which then points at the commit. When you make changes on one worktree, in general it doesn't update the other worktrees.
So if you change the reference that the branch is pointing at, when you look inside the other worktree that points to that branch, it has no way of knowing it was initially looking at a different commit and it just thinks that the diff between the current head of that branch and your current working directory is a bunch of uncommitted changes.
If you use --detach or refer directly to the commit instead you get the behavior that is expected which is that it points to the commit itself and not the branch (which then points to the commit).
> Also, git worktree requires me to make a new directory anyway! So why would I not just put a fresh clone in there? To save a few megabytes of the .git index?
The problem is that not all repos are small and often it's easier to just do things in one repo than to need to push everything up and pull it down to move between multiple repos.
Also not all repos are small. The linux repo (which if you are working in the higher spec side of embedded software you probably will be using with multiple different downstreams) is at least a gigabyte in size nowadays. I like having the whole repo when I can because it makes looking for stuff locally easier but I don't want to lug around multiple copies of that repo if I can avoid it.