Imagine if you have a judge giving out sentences based on astrology books. I don't think anyone would argue the problem would be resolved by banning astrology books from our libraries.
1. Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works.
2. Anything that’s invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.
3. Anything invented after you’re thirty-five is against the natural order of things.
To be fair, I do think there are plenty of uses for LLMs, but with adoption skyrocketing there really are no guardrails against misuse.
And I suspect the same will happen for LLMs, in the end we will just start thinking in "another level of abstraction". We are still in the early days and still have a lot to learn about how to properly use this new toll but I think LLMs are a positive change for society.
In your example you should probably drop the judge, but you should also make a rule saying astrology books aren't a legitimate source of sentence guidance. That's what people are annoyed about re:LLMs. People keep insisting they are a legit source in different situations.
You wouldn't ban them overall, but you do want some kind of society-level taboo against relying on them. You can't just deal with it on the level of people who get fooled into using them.
My example was absurd on purpose, I didn't want to bring an example where people could respond with "well, actually..."
But in the real world that is rarely the case, imagine substituting "astrology book" for "Bible/Quran/..." Would that be considered a legitimate source of sentence guidance? I'm sure people would spend years arguing about that...
As a society we need to understand that LLM hallucination is no different than a bloom filter giving you a false-positive.
Thus being a spectacularly bad tool is itself a cause to assign blame.
Banning books does not ban knowledge. It just makes it a bit more inconvenient. Banning drugs has not stopped people from getting access.
Just kick out the judge who clearly... lacks judgment.