The part that I'm disgusted by is the part where the chatbot is operated by a company rather than open source software.
Makes it seem more like a person than an appliance.
Open source might not be very appropriate for this. (And it wouldn't help for the company to use an open source product, because if they're in charge of the infrastructure, re-configuring it to use the filters is the problem and could still happen even if they use the open source software title.) Basically, open-source is a specific solution to a somewhat narrow problem, and not a cure-all for everything bad that happens in technology.
The best analogy I could make is that if I had a roast chicken, but it had a giant smelly turd smell, and somehow I knew that it was totally clean and edible, it'd still disgust me. Even if the turd smell went away after a few minutes.
It's just weird. There are real humans out there, put down the phone man.
> There are real humans out there, put down the phone man.
was the part of this thread that actually provoked disgust for me. It's akin to telling a person suffering from depression "you should just be happy!" as if they haven't tried.
There are people out there who will never succeed in finding human companionship, it's a fact of life. I see it as a positive that they can at least find some form of companionship, because the alternative is a dark path that can lead to self-loathing, radicalization, and in the worst case scenario, violence.
I wouldn't tell a person with depression to "just be happy", but neither would I allow them to humour the idea of giving up on life.
They certainly won't find companionship by giving up and deciding to mess around with bots instead.
It's more like telling a depressed person to get some sunlight in the morning or whatever. Is it going to fix everything? No, but you do it anyway, because behaviour is sticky.