Theres a video of a recent talk Nicolas Carlini gave this past week on Youtube. It’s eye opening. If you don’t believe that LLMs are going to transform the cybersecurity space after watching that I can’t help you.
https://www.youtube.com/watch?v=1sd26pWhfmg
7 minutes in, he shows the SQLI he found in Ghost (the first sev:hi in the history of the project). If I'd remembered better, I would have mentioned in the post:
* it's a blind SQL injection
* Claude Code wrote an exploit for it. Not a POC. An exploit.
What's the distinction? A proof of concept is just something that demonstrates that a bug is possible to exploit, by doing so.
POC generally means “you can demonstrate unintentional behavior”.
“Exploit” means you can gain access or do something malicious.
It’s a fine line. Author’s point is that the LLM was able to demonstrate some malfeasance, not just unintended consequence. That’s a big deal considering that actual malicious intent generally requires more knowhow than raw POC.