Security auditing in the age of (good enough) AI | Hacker News Reader

Security auditing in the age of (good enough) AI

blog.trailofbits.com

96 pointsaray0716 comments

Good enough AI for security auditing feels like asking for "good enough" brakes. There's just no room for complacency.

Not sure I follow the analogy, so apologies if I'm reading it backwards, but we do in fact have standards for "good enough" brakes without being brake-maxxers.

Similarly here, I'd argue that a verified implementation with correctness proofs, mechanical translation, and easily auditable theorems seems close to good enough? A lot rides on the Claude-built translator, I suppose, but the trusted code for that project seems to be tiny in comparison to what it would have been 2 years ago!

I suspect you two are using the same words for different terms / connotations.

"Good enough" in colloquial speak usually means the minimum required for some particular requirement.

For security, there is usually no exact threshold that differs between insecure and secure. It's a spectrum that involves costs and tradeoffs, which are subjective value judgements.

A SaaS startup in pre-seed mode with no customers will have VASTLY different value judgements than a bank that handles $trillions in assets. Hence they will make very different security choices and "good enough" will mean very different things in their different sectors.

I don't get the analogy. Brakes that are good enough for a Honda Civic driven at regular speeds are not good enough for a fire truck or a race car, but there are in fact standards that are "good enough" for all of those.

People on HN have a tendency to shove in analogies where it doesn't need one. Even worse when most of the time the analogies do not make sense.

You could even say we are like a glass carpet: the color may be attractive but the texture is terrible...

There is an area where an AI audit can add some real value: the huge multitude companies that have never looked at security at all. Small businesses.

This is why we all use ceramics in our daily-drivers, of course.

All security is a negotiation of allocating resources. It always ends with “well, that’s good enough for this.” There is never a “Ahhhh we’re done!” It can always be “more secure”.

It’s over security people

You need to find a new way to do business with the assumption that you’re 100% eventually going to get pwned

So just assume that going forward and you’ll start to rethink your architecture

FHE would be nice, but there's a lot of products that could be E2EE today. Plenty of breaches are just server-side files and dumps, there's no excuse for not using E2EE cloud storage.

Databases are much more tricky, but if we're talking a significant shift, it is as good a plan as any. Clearly server trust is not working.

AI is accelerating both sides for defenders and attackers, but mobile clients are already way ahead of typical desktop. Android's desktop mode exists in the OS now but needs a lot of work and more powerful chips before it can replace desktops.

For security against remote attacks, I'd rank clients roughly: GrapheneOS, iOS/iPadOS, stock Android on Pixels, ChromeOS, macOS, Windows, desktop Linux. Prefer more secure clients, always, but now the client is all that's left to trust for content.

Native clients avoid the web E2EE problem and are easier than ever to build. With AI, you can audit the source as often as you want and reproducible builds mean you know that's what you're running.

All valid.

Overall though the point is any particular hole that you have will be exploited

so what used to be a game of “patch the hole” is going to need to change entirely

Haha, we are ahead of you. We realised we were gonna get owned a long time ago, since business and engineering teams prioritised speed over security. And also that it is genuinely hard to secure everything.

This is why there are so many resilience initiatives in serious organisations. Assume it is gonna happen. Limit blast radius, ensure effective recovery.

... How does that change, from the security advice of 2008?

Pwn2Own was demonstrating no stack as secure back then, too. People were told defense in depth, recovery methods, rapid spin up and lockdown, for the last few decades.

This is a great example of how we can actually be using AI to do things better, not just faster

I'd argue that the difference is often just semantics. In general, I can do a better job on a given task in a week than a day. If I only have a day, but AI lets me do in that day what would have otherwise taken a week, then the result is better while my time expenditure remains constant.

Reply on news.ycombinator.com