I was running into this too until I started upgrading with

llm install -U llm

instead of

uv tool upgrade llm

(the latter of which is recommended by simonw in the original post)