"It's better and safer for people to remain ignorant" is a terrible take, and surprising to see on HN.
Noone is saying that - that's your reframing of it to suit your point.
AI isn't the canonical source of information, and nothing is being censored here. AI is asked to "do work" (in the loosest sense) that any knowledgeable person is perfectly capable of doing themselves, using canonical sources. If they learn. If anything this is encouraging people not to remain ignorant.
The reverse of this is ignorant people copying & pasting insecure code into production applications without ever learning the hazards & best practices.
Limit the scope of available knowledge and you limit the scope of available thought, right? Being more generous, it looks like a common refrain is more like "you can use a different tool" or "nobody is stopping you from reading a book". And of course, yes this is true. But it's about the broader cultural change. People are going to gravitate to the simplest solution, and that is going to be the huge models provided by companies like Google. My argument is that these tools should guide people towards education, not away from it.
We don't want the "always copy paste" scenario surely. We want the model to guide people towards becoming stronger engineers, not weaker ones.
I don't think that these kind of safety-rails help or work toward this model your suggesting (which is a great & worthy model), but I'm far more pessimistic about the feasibility of such a model - it's becoming increasingly clear to me that the "always copy paste" scenario is the central default whether we like it or not, in which case I do think the safety rails have a very significant net benefit.
On the more optimistic side, while I think AI will always serve a primarily "just do it for me I don't want to think" use-case, I also think people deeply want to & always will learn (just not via AI). So I don't personally see either AI nor any safety rails around it ever impacting that significantly.
When that happens, AI may be the only way for many people to learn some information.
I don't know why anyone would pretend not to recognize the importance of that or attempt to downplay it.
ie, both sides have points, and there's no simple solution :(
The point is for now, trying to make LLM’s safe in reasonable ways has uncontrolled spillover results.
You can’t (today) have much of the first without some wacky amount of the second.
But today is short, and solving AI using AI (independently trained critics, etc.) and the advance of general AI reasoning will improve the situation.
AI provided as SaaS by companies? They don't want the bad press.
I’m not a law abiding citizen just because I don’t know how to commit crimes and I don’t believe anyone is.
It’s not lack of knowledge what’s stopping me from doing bad things and I don’t think people are all trying to do something bad but they can’t because they don’t know how.
This information control bs probably has nothing to do with the security.
> The most common route for nearly pure TATP is H2O2/acetone/HCl in 1:1:0.25 molar ratios, using 30% hydrogen peroxide.
Why are you uncomfortable with an AI language model that can tell you what you can already find for yourself on Google? Or should we start gating Wikipedia access to accredited chemists?
There are 2 things to consider:
The only answer to bad people with power is a greater number of good people with power. Luckily it happens that while most people are not saints, more people are more good than bad. When everyone is empowered equally, there can be no asshole ceo, warlord, mob boss, garden variety murderer, etc.
But let's say that even if that weren't true and the bad outnumbered the good. It actually still doesn't change anything. In that world there is even LESS justification for denying lifesaving empowering knowledge to the innocent. You know who would seek to do that? The bad guys, not the good guys. The criminals and tyrants and ordinary petty authoritarians universally love the idea of controlling information. It's not good company to side yourself with.