I'm in a different direction on this, worktrees don't solve the problem for me, this is stuck in 1 agent = 1 task mode. I want a swarm of agents on 1 task.
There's a couple of agent personas i go back to over and over (researcher, designer, critic, implementer, summariser), for most tasks i reuse 90%+ of the same prompt, but implementer has variants, one that's primed with an llms.txt (see answerdotai) for a given library i want to use, another that's configured to use gemini (i prefer its tailwind capabilities) rather than claude etc.
To organise these reusable agents i'm currently test driving langroid, each agent contributes via a sub task.
It's not perfect yet though.