To be fair, the limiting factor in remediation is usually finding a reproducible test case which a vulnerability is by necessity. But, I would still bet most systems have plenty of bugs in their bug trackers which are accompanied by a reproducible test case which are still bottlenecked on remediation resources.
This is of course orthogonal to the fact that patching systems that are insecure by design into security has so far been a colossal failure.
Both assumptions being unlikely.
You also end up with a code base you let an AI agent trample until it is satisfied; ballooned in complexity and redudant brittle code.
They attempt to, sure, but it rarely works. Now, with AI, maybe it might, but that's sort of a worse outcome for the specific human involved - "If you're just an intermediary between the AI and me, WTF do I need you for?"
> or moving it to insurance agencies.
They aren't "moving" it to insurance companies, they are amortising the cost of the liability at a small extra cost.
That's a big difference.
Usually thats after the premiums become too high for most people to pay.
And are you thinking this going to start happening at some point or what?
The letters I get every other month telling me I now have free credit monitoring because of a personal info breach seems to suggest otherwise.
Now I used the agent to do a lot of the grunt work in that refactor, but it was still a design decision initiated by me. The chatbot, left unattended, would not have seen that needed to be done. (And when, during my refactor, it tried to fold in the fourth case I had to stop it.)
(And for a lot of code, that's ok - my static site generator is an unholy mess at this point, and I don't much care. But for paid work...)
What if it's a 200% chance, and every fix introduces multiple defects?
If you plot this out, you'll notice that it eventually reaches > 100% and the total number of defects will eventually grow exponentially, as each bugfix eventually introduces more bugs than it fixes. Which is what I've actually observed in 25 years in the software industry. The speed at which new bugs are introduced faster than bugfixes varies by organization and the skill of your software architects - good engineers know how to keep coupling down and limit the space of existing code that a new fix could possibly break. I've seen some startups where they reach this asymptote before bringing the product to market though (needless to say, they failed), and it's pretty common for computer games to become steaming piles of shit close to launch, and I've even seen some Google systems killed and rewritten because it became impossible to make forward progress on them. I call this technical bankruptcy, the end result of technical debt.
"BuT ThE LlM iS pRoBaBlY iNtRoDuCiNg MoRe BuGs ThAn It FiXeS"
This is an absurd take.
Very, very rarely is a bug a mistake. As in, something unintentional that you just fix and boom, done.
No no. Most bugs are intentional, and the bug part is some unintended side effects that is a necessary, but unforseen, consequence of the main effect. So, you can't just "fix" the bug without changing behavior, changing your API, changing garauntees, whatever.
And that's how you get the 1 month 1-liner. Writing the one line is easy. But you have to spend a month debating if you should do it, and what will happen if you do.
I use it all day. I consider it a near-miracle. Yet I correct it multiple times daily.
I stated the LLMs are actively being used in repo's today, to chew through backlog items, and your response is to wonder if I've ever used Claude.
To me it's surprising that someone like you, who appears to have a reading comprehension deficiency, is able to use Claude.
If you find a bug in a web browser, that's no big deal. I've encountered bugs in web browsers all the time.
You figure out how to make a web page that when viewed deletes all the files on the user's hard drive? That's a little different and not something that people discover very often.
Sure, you'll still probably have a long queue of ReDoS bugs, but the only people who think those are security issues are people who enjoy the ego boost if having a cve in their name.
The prediction is: Within the next few months, coding agents will drastically alter both the practice and the economics of exploit development. Frontier model improvement won’t be a slow burn, but rather a step function. Substantial amounts of high-impact vulnerability research (maybe even most of it) will happen simply by pointing an agent at a source tree and typing “find me zero days”.
When the payment for vulns drops i'm wondering where the value is for hackers to run these tools anymore? The LLMs don't do the job for you, testing is still a LOT OF WORK.
Fixing vulnerable code is usually trivial.
In the physical world breaking things is usually easier.
I've never been on a security-specific team, but it's always seemed to me that triggering a bug is, for the median issue, easier than fixing it, and I mentally extend that to security issues. This holds especially true if the "bug" is a question about "what is the correct behavior?", where the "current behavior of the system" is some emergent / underspecified consequence of how different features have evolved over time.
I know this is your career, so I'm wondering what I'm missing here.
As bugs get more complicated, that asymmetry has become less pronounced, but the complexity of the bugs (and their patches) is offset by the increased difficulty of exploiting them, which has become an art all its own.
LLMs sharply tilt that difficulty back to the defender.
> formal verification
Outside of limited specific circumstances, formal verification gives you nothing that tests don't give you, and it makes development slow and iteration a chore. People know about it, and it's not used for lot of reasons.
First type checking is a form of formal verification and it’s used everywhere. Second have you heard of rust? Do you know why it’s becoming an alternative to C++ or C? Entirely because of its type checker or aka formal verification. It is the literal main reason why rust was created.
Have you heard of typescript? It’s essentially a formal verification layer over JavaScript. Everyone uses it now for the front end.
You don’t know what you’re talking about. I recommend you do some research before saying anything on this site.
Also, like ForAllSecure's Mayhem, I think they can differentiate on automatic patching that's reliable and secure. Maybe test generation, too, that does full coverage. They become drive by verification and validation specialists who also fix your stuff for you.
Harnesses, maybe, but prompts?
There's still this belief amongst AI coders that they can command a premium for development because they can write a prompt better than Bob from HR, or Sally from Accounting.
When all you're writing are prompts, your value is less than it was before., because the number of people who can write the prompt is substantially more than the number of people who could program.
Some orgs will be able to do this, some won’t.
Probably because it will be a felony to do so. Or, the threat of a felony at least.
And this is because it is very embarrassing for companies to have society openly discussing how bad their software security is.
We sacrifice national security for the convenience of companies.
We are not allowed to test the security of systems, because that is the responsibility of companies, since they own the system. Also, companies who own the system and are responsible for its security are not liable when it is found to be insecure and they leak half the nations personal data, again.
Are you seeing how this works yet? Let's not have anything like verifiable and testable security interrupt the gravy train to the top. Nor can we expect systems to be secure all the time, be reasonable.
One might think that since we're all in this together and all our data is getting leaked twice a month, we could work together and all be on the lookout for security vulnerabilities and report them responsibly.
But no, the systems belong to companies, and they are solely responsible. But also (and very importantly) they are not responsible and especially they are not financially liable.
>Probably because it will be a felony to do so. Or, the threat of a felony at least.
"my software" implies you own it (ie. your SaaS), so CFAA isn't an issue. I don't think he's implying that vigilante hackers should be hacking gmail just because they have a gmail account.
deliberate vulnerabilities (thanks nsa)