What would someone like the Tsarnaev brothers be able to do with the power of an unrestricted LLM? Well-financed cartels? Organized terrorist groups?
Yes, there used to be an uproar about stuff like the anarchists cookbook... and people did attempt some of the things it outlined. The saving grace is that many of the things in that book were just wrong anyway. They likely served as unhelpful misdirection as much or more than they were dangerous. Unfortunately, LLMs are a lot more accurate and helpful.
Censorship is not the answer.
> Model ablation exists and you can get far enough on commodity hardware with a local model.
Yes, but that increases the barrier to entry which is in opposition to the effect I'm talking about: the democratization of applying advanced knowledge and analysis to people who for which this would have been previously a barrier.
If someone is smart enough, they can just read a book themselves and figure out how to apply advanced ideas to their malice. The difference with a commercially-hosted model is that people below that bar can obtain that leverage... which is a much larger group of people.