But you raise an interesting point. It seems the ethics of LLM manufacturing is somewhat different than that of weapons (or other tool) manufacturing due to the ability of the LLM to convey knowledge. Still, I'm unconvinced of your position. Consider that we regulate neither the publishing nor dissemination of chemistry textbooks. Surely an LLM teaching someone chemistry falls into the same general category?
> Surely an LLM teaching someone chemistry falls into the same general category?
I think the scale and the effort might break the analogy. Teaching yourself means a degree of patience and certain personality traits that would be rarer in a malicious person (along the lines of “a sufficiently smart person wouldn’t need to be a criminal to succeed”), with exceptions of course. Being taught by a teacher implies a degree of KYC and care about who you are. Being served on a plate the specific information on how to manufacture something dangerous bypasses those barriers, which I think changes the equation.