> When someone wielding a non-safeguarded model deletes the money in everyone’s bank account, I look forward to the HN comments claiming it’s an attempt by Anthropic to pull off regulatory capture.
This statement portrays a fundamental misunderstanding of how the infrastructure which powers these systems work. Note: I am not saying there are no risks, I am just saying the risk you are focusing on is the least likely one of all I have seen people be upset by.
Far higher risks one could outline are:
1. Network-connected PLCs for big infrastructure (drinking water, sewage, power, etc) being tampered with.
2. Extremely persistent malware tailored for every permutation of hardware + software.
3. Cyber criminals improve in technical capabilities (phishing sites, scam calling, propaganda campaigns, etc).
But, the cork is out of the bottle on this one. With even basic models you can begin a loop of training specialized models on low cost hardware which can be used to do specific hacking tasks.
I don't know what the best antidote to this is but I doubt that it will be in limiting access to OSS models to people in the USA as all of the threats listed above come from *outside actors*.