Hacker News Reader

Comment by radicalbyte | Hacker News Reader

I don't think that 2 is true though. It's exactly the argument we made against open source software.

Yes these models make exploits easier to exploits. Lets use a construction analogy: they've made all of the defects in our buildings easy to see. We have a choice. We either fix those defects, or we ban the tools which lets us see them.

It's clear to me what we do: we use these new tools. Then we fix the defects. Yeah sure it'll mean some work for us but at the end we're in a much much better position.

Anthropic, Open AI and Grok are arguing for hiding the defects. For making us weaker and more vulnerable. For their own profit.

View on HN

Agreed, and if you look at the public information about various attacks the individual vectors are not particurarly sophisticated. The Huggingface incident for example had agents in a sandbox that was about as useful as a wet paper bag, and the attacks on remote infrastructure were basically enabled by bad sanitation. None of the techniques are novel.

These are not hard problems to fix, nor should anyone find it acceptable to have them be so prevalent. A determined human attacker could easily exploit defects like the agents found. Bad input sanitation, SSRF attacks, exploiting stupidly implemented token verification... all techniques that have been widely known for decades and there should be no excuse for publishing software that is riddled with exploits that enable the use of them.

..and they had basically 0 monitoring of it. Sure if it's a 5-person company then you can forgive them. Only these labs are massive companies who are spending $50B+ a year, it's extremely negligent.

They've also apparently deployed the best-of-the-best in Silicon Valley and this is what they deliver? Really?

One challenge is that we’re still constantly generating defects even with these tools. And you need the next gen model to spot the mistakes of the previous one. The exploits get more and more complicated and involved of course. But now:

* there’s a long tail of software that just won’t get secured or will take a very long time

* exploits have transformed from an expertise problem to a compute time search.

This is very different than any problem faced before. I’m not saying I’m convinced by the “slow down” approach, but I don’t think it’s as simple as you point out.

In the things published in public, they're using existing exploits and hitting systems which have been poorly maintained. It won't end at that, obviously, especially with humans behind the wheel.

Yet I'm convinced that we can create extremely resilient software systems. I've been involved in the entire life-cycle of one. We know how to be very defensive and it's a choice to build "cheap, crappy and disposable" software.

LLMs are turning the needle there and I think that's a good thing.

Reply on news.ycombinator.com