Hacker News Reader

Comment by jermaustin1 | Hacker News Reader

> I understand that such models can be used by malicious actors, but it’s fair to have it publicly available

I feel like there should be some mechanism to prove you own the code/app/site/whatever and it will remove the guardrails from the LLMs allowing them to find and fix these vulnerabilities.

View on HN

This is a "they have guns so we need guns" scenario.

You can't guarantee everyone else will use a neutered model.

Impossible with source code, possible to bypass with app/site

Don't we already do this with services like Let's Encrypt, which is arguably more sensitive? If you had the codebase you could fake it, but it would still provide some amount of protection against abuse.

With Let's Encrypt, all the verification is done on their side with them controlling the connection between themselves and whatever they're trying to verify.

In this case, you can put whatever you want between the harness you're running (or modify the harness itself), and essentially "lie" to the model. Any verification technique would be fairly trivial to bypass, while you continue to run the harness locally.

Yeah, for web apps, you can trick models by simply proxying it and pointing the models to that localhost. They then think they're not working on a live target.

Have personally tested this with Opus and Sol and it works.

Classifiers are tricky though. Here's where open weights will win.

Isn’t this essentially what anthropic is doing, albeit in a manual fashion? They work with code owners to run mythos and find issues.

Only if you're some big corporation with deep pockets. They actually accepted me into their cyber program but Fable's still locked down.

OpenAI now makes it easy to join their verified security program. Took me 5 minutes, and I was able to get GPT to do a full end-to-end pen test

Is this about the new project Blue and Red thing? Or just pre-existing Trusted Access for Cyber program?

I've joined TAC, but still have to dance around it.

Doesn't that require a business account (so, actually, two business accounts since that's the minimum)?

Code owners - no. Ultra wealthy code owners with connections, and a few peasants with popular projects, for public image.

Reply on news.ycombinator.com