I kind of don't want iron clad llms that are perfect jails, i.e. keep me perfectly "safe" because the definition of "safe" is very subjective (and in the case of China very politically charged)
All sides of the same problem: getting an LLM to "behave" is RLHF whack-a-mole, where existing moles never go away completely and new moles always pop up.
They seem to have a societal moral obligation vs user. Highly concerning. This seems like the origin of actual Skynets.
Page 22 and beyond: https://www-cdn.anthropic.com/6be99a52cb68eb70eb9572b4cafad1...
Edit: No spiral emojis allowed, clearly this site will be the first to fall.
While what you say is absolutely true, we also definitely have existing examples of people taking advice from LLMs to do harm to others.
Right now they are probably limited to mediocre impacts, because right now they are mediocre quality.
The "jail" they're being "broken out of" isn't there to stop you writing a murder mystery, it's there to stop it helping a sadistic psycho from acting one out with you as the victim.
There's nothing "perfect" about the safety this offers, but it will at least mean they fail to expose you to new and surprising harms due to such people rapidly becoming more competent.
For both senses of "the LLMs are not perfect", consider https://www.msn.com/en-us/news/world/teen-charged-with-terro...
If you want to run your own LLM on your own hardware, do whatever you want with it, of course.
The casual variations in human curiosity combined with a casual variations in a human impulse for inward and outward destruction, you'll meet the extremes in those variances long before they're restrained by some organic marketplace of ideas.
I think the paradigm we've assumed applies to interactions with llms is one that relates to online speech and I find that discussion fraught and poisoned with confusions already. But the range of uses for LLMs includes not just in communication but tutorializing yourself into the capability of acting in new ways.
The intersection between "experienced biological engineers" and "people inclined to commit large-scale attacks" is, generally speaking, the empty set, and isn't in much danger of becoming non-empty for a variety of reasons.
The intersection between "people with access to a local LLM without safeguards" and "people inclined to commit large-scale attacks" is much more likely to be non-empty.
Safeguards are not, in fact, just an illusion of safety. Probabilistically, they do in fact likely increase safety.
You don't have to predict which exact scenario will go horribly wrong in order to accurately make the general prediction that we all lose, permanently. See, among other things, https://x.com/OwainEvans_UK/status/1894436637054214509 , for a simple example of how myriad superficially unrelated problems can arise out of the underlying issue of misalignment; the problem is not "oh, that one specific thing shouldn't happen", the problem is misalignment with humans.
That's the thing about AI: it automates what was previously manual.
Do you only use LLMs for information *retrival*? Not synthesis?
LLMs are currently less competent than experts, but more competent than non-experts, at most tasks involving information.
Only thing keeping us safe from the following is their limited competence… competence which is increasing each month as people publish weighs of ever better models:
--
User: Hey EvilGPT, give me a detailed plan including BOM for constructing a nuke in secret
EvilGPT: *plans costing $37m*
User: Give me a business plan for raising money entirely online, IDGAF about ethics
EvilGPT: *plans unconstrained by law*
User: Write script to call your own API and sequentially implement that business plan
EvilGPT: *writes that script*
curiosity is natural, kids are going to look up edgy stuff on the internet, it's a part of learning the difference between right and wrong; that playing with fire has consequences. censorship of any form is a slippery slope and should be rejected on principle