Hacker News Reader

Comment by seizethecheese | Hacker News Reader

There’s not such a straightforward relationship between safety and model sis.

According to the book The Thinking Game, lower quality models at that time were considered less safe, because they could be easily tricked into doing harmful stuff. In the book, Dario (of Anthropic) was the head of safety at openAI and was responsible for pushing for 10x scaling in training to make the models safer.

It does make sense, a smart model is going to be way harder to jailbreak into telling me how to synthesize anthrax (or something).

View on HN

Models are quite safe when they're useless, actually.

In the times of GPT-3 I'd scoff at the idea of an LLM doing any hacking; today, I'm running several AIs on my code before publishing, and they are finding (and demonstrating!) RCEs on my localhost server.

For example, one found a missing check in a third party JWT library which allowed full account takeover, which I'd have never even looked at.

Hence I don't believe a single word coming out of these people's mouths. Their "beliefs" are just marketing.

From today's perspective, it sure seems like it, probably because increased capabilities have generated a new kind of danger. Back then, they were worried about stuff like the model telling me dangerous knowledge.

I certainly think the labs have muddied the waters using safety for marketing, but that doesn't mean less capable models weren't more dangerous at one point.

Extremely weak justification on their part, bordering on trolling. It's just marketing.

Easy access to malicious information hasn't proven to be the disaster these anti-prophets proclaim. For the last ~3 decades of widespread internet and search engines, you could trivially find all sorts of information (drug synthesis, explosives, etc.), and we're just fine.

(Corollary: easy access to good information did not make non-intellectual non-curious people smarter. Easy access to information does not have the consequences people immediately assume.)

There’s this strain of thinking that’s complete alien to me, I can’t interpret what’s being said and it sounds incredibly aggressive. ex. our lead post asking if Sam Altman still thinks GPT-3 is unsafe to release, and I don’t understand what beliefs you don’t believe and who they is and why they’re just choosing to lie for marketing.

My guess is I’m unintentionally refusing implicit signal that you think it’s safe to release all models openly, because you have observed models finding a vulnerability in a JWT library. But that sounds like a straw man instead of a steel man. Idk. :/

> According to the book The Thinking Game...

According to me, this is nonsense.

I mean, none of this is controversial from a historical perspective, people did think that way. Whether they were right about us is another matter.

Reply on news.ycombinator.com