Interesting approach. A few questions:

How do you handle false positives from mutation strategies like Base64 or token smuggling? In my experience, a lot of "successful" jailbreaks from automated fuzzing don't actually produce harmful outputs — the model just gets confused and outputs gibberish that technically matches a keyword filter.

Also curious about your payload curation — are these tested against specific model families, or generic? The attack surface differs quite a bit between Claude, GPT-4, and open-source models.

The local-first angle is smart. Most enterprises I've talked to won't send their system prompts to a third-party SaaS for obvious reasons.