If you teach someone chemistry, that someone probably has sense to not use it for criminal purposes. If you have a track record of teaching future terrorists specifically, you will be shut down. Crucially, before you educate them en masse.
If you’re comparing that to a tool that essentially educates thousands, millions of people, with no KYC, you bet budding terrorists are going to be disproportionately represented within that group.
But you raise an interesting point. It seems the ethics of LLM manufacturing is somewhat different than that of weapons (or other tool) manufacturing due to the ability of the LLM to convey knowledge. Still, I'm unconvinced of your position. Consider that we regulate neither the publishing nor dissemination of chemistry textbooks. Surely an LLM teaching someone chemistry falls into the same general category?
> Surely an LLM teaching someone chemistry falls into the same general category?
I think the scale and the effort might break the analogy. Teaching yourself means a degree of patience and certain personality traits that would be rarer in a malicious person (along the lines of “a sufficiently smart person wouldn’t need to be a criminal to succeed”), with exceptions of course. Being taught by a teacher implies a degree of KYC and care about who you are. Being served on a plate the specific information on how to manufacture something dangerous bypasses those barriers, which I think changes the equation.