Hacker News Reader

Comment by Vecr | Hacker News Reader

Yeah, with this kind of stuff it's just a reputation thing. In future systems it's a problem though, if you accidentally train them on on the wrong thing or the system is smart enough to reconstruct the missing data, the model can start giving detailed bioweapon production plans. Including safety instructions so the would-be terrorist doesn't get infected too early.

View on HN

Wow! What an absolutely pointless and stupid concern to have, that a terrorist is going to use an an LLM to make a bio weapon.

Does literally no one understand that the only information that is inside of an llm model is information that has been fed into the model? I think more than anything this fake AI boom has really demonstrated how unintelligent most people are.

Well, most terrorists are not very smart, at least on an absolute level. I think even near-future models would be able to help them on a basic operations/not screwing up level. As I said, a problem is the reconstruction of information intentionally omitted. Models can't currently reconstruct even the basic knowledge I have, but it's something they should be able to do based on simple inference rules.

I don’t think it’s a very valid concern, but LLMs can already do a limited level of reasoning to create information not directly loaded into them. Sometimes that reasoning is even correct! The idea here would be that even if you didn’t load that kind of information into them, at a certain level of advancement, they can determine the necessary plans from other information that has been loaded into them.

Well yes sometimes a bullshitted solution may be correct! But the person doing the prompting for a bioweapon will end up with yeast sludge anyway.

>Does literally no one understand that the only information that is inside of an llm model is information that has been fed into the model?

Not literally no one, but Sam Altman recently claimed that we'll be able to ask an LLM to "solve all of physics" soon, so plenty of people who should know better do seem to think they're basically magic.

If "plans" are sufficient to result in effective bioweapon production capacity then you've lost the battle already. Fortunately, they're not.

I am sure you can get detailed bioweapons instructions on 4chan already. Aum Shinrikyo figures out sarin in the 90s

Aun didn’t just go to the library and look up “how to make to sarin.”

They had progressional chemists and Russian military personal indoctrinated into the organization.

Reply on news.ycombinator.com