One solution I haven’t seen recommended much is to have a Claude instruction/skill that explicitly audits the diff of every upgrade, and force this manual audit as part of your upgrade workflow. This seems like it would work pretty reliably.
How does having an AI audit external code help? Can they not be prompt injected to ignore a malicious change?
I guess I am sort of concerned that they are a pretty thin layer and even if you put "DO NOT ALLOW PROMPT INJECTION", it's a bit like saying "make no mistakes". There _is_ a priority between `system` and `user` level messages as I had recalled, so a specifically made tool that has its own system prompt should prevent injection while asking Claude CLI could still allow for prompt injection.
What are your thoughts and experience?
The concern is real and unsolved. I think security researchers have an advantage here because they still can fall back to manual audits if their automated analysis (or scores thereof) is off.