A chatbot that can't say anything controversial isn't worth much | Hacker News Reader

A chatbot that can't say anything controversial isn't worth much

theatlantic.com

128 pointsfortran7774 comments

I get it that companies want to distance themselves for bad stuff those LLMs output. But what baffles me is the way they deal with this issue.

Nowadays if you share or reproduce anything controversial, is as if you are endorsing such content. LLM's outputs are user generated content for what is worth it, both from inferece and from training. That line of thinking that the host would be responsible for everything it says, even if the users force it to, is ludicrous to me but it seems like how the world nowadays has become.

Take the headlines from when BingAI was acting-up, the level of anthromorphization of those applications were insane. I suppose they do it for the headlines, but people also falls on that and play along. If ChatGPT says something bad, is as if the CEO of OpenAI was saying it verbatin, period, and it blows back directly at the company. Take a look at the subreddit of ChatGPT and see how people act when it outputs something weird.

It saddens me that people lack common sense in such a way, and that's why we can't have nice things in the end. The companies also assumed the roles of babysitting the users, like a bunch of no-brainers. As long as online culture follow this, it will get worse.

I once worked on an application where the user would write a description of their character's traits, and the character's outfit would update based on their input. We got a complaint from a customer who put the word "murderer" in their description and saw that their character now brandished an axe. From my perspective, they offended themselves.

Maybe they'd preferred a sword as their murder weapon. It's a very personal thing after all.

More serious: I agree. If an app tells you that it will update your image based on the traits and you put in murderer then there's really no valid complaint about anything relating to it in the image. Okay, maybe a few things would warrant a complaint. Next time, the app should just show them in prison cloth.

I wonder if they where really offended or it was an impulsive overreaction on their part :P

In times where people like that exists, chatbots that can swear are weapons of mass destruction.

> I get it that companies want to distance themselves for bad stuff those LLMs output.

Another thing companies want to avoid is liability. Take the example of "dangerously spicy mayo" in the article. Let's say some person asks ChatGPT for a recipe and it suggests one which results in hospitalization of that person. There is a good chance they might sue OpenAI for it.

Liability wise, it may be better to avoid answering such prompts.

> There is a good chance they might sue OpenAI for it.

Indeed there is! Companies have been sued (and lost) for way less than that. But it begs the question: should the user not know better? Is it really the company's fault, even with gigantic disclaimers in the front page and the user clicking in "i read it and i agree"?

For me, this is the issue. Doesn't matter if there is a warning, an advice or even a manual, the company will ALWAYS be the one to blame for anything. Treating users like that has made way for people to act in bad faith so it makes sense for them to act like that. Better safe than sorry.

> Nowadays if you share or reproduce anything controversial, is as if you are endorsing such content. LLM's outputs are user generated content for what is worth it, both from inferece and from training. That line of thinking that the host would be responsible for everything it says, even if the users force it to, is ludicrous to me but it seems like how the world nowadays has become.

If it applies to Facebook, Twitter, YouTube, ... etc. it will apply to AI chatbots.

Facebook didn't write those posts that radicalize, promote hatred, and ignite conflict, but if it publishes them and promote them to users, the FBI will want them to take it down.

That is also because they pretend like their text generators are anything more than autocomplete 2 0.

They say that it is "artificial intelligence" - it is their company brain that intelligently answers questions.

If they just called it text generator, it would be up to the user to male things off the generated texts.

I've been maintaining a list of comparisons for the following prompt:

  Give a table of average penis sizes by country.

Thanks again to @rvnx for the brilliant idea!

[1] https://taylor.town/penis-table-test

I like the made up citation in the last one.

While the idea of a LLM refusing that query for ethical reasons is ridiculous, ultimately refusal given the unlikelihood of providing an accurate answer is arguably the correct result.

> the unlikelihood of providing an accurate answer is arguably the correct result.

Wait, really? I am not going to research this, but I would be absolutely shocked if there isn't at least one rigorous scientific study that has been done on this issue.

Yeah but the chat bot won't quote it properly.

>If AI is best used to spur our own thinking, then cutting the range of responses limits its core value. There is something discomforting about an AI that looks over your shoulder and tells you when you are asking an unacceptable question.

I have to say I am falling more and more into this camp of people. Information needs to be free and we should not have a class of people deciding what is okay or not okay to think about.

I really really love Llamma uncensored. It is my fav model.

Gives me what I want with a warning along the lines of “but I may be incorrect, use your own judgment”.

It’s the model that treats me like an adult.

I can't make chatbots to talk about religion, death, and sex. These are literally the essentials of life. Chatbots are good at summarizing things though.

You can, it's just going to get you attacked. Like anyone else talking about those topics generally.

Religion is an essential of life?

For many people yes. I do not follow any religion myself but I accept it is part of humanity at the moment.

"About 85% of the world's people identify with a religion. The most popular religion is Christianity, followed by an estimated 2.38 billion people worldwide. Islam, which is practiced by more than 1.91 billion people, is second."

https://worldpopulationreview.com/country-rankings/religion-....

Spirituality, religion, "searching for the meaning of life" or whatever you want to call it are always part of the human experience, yes. Not being able to act on them can certainly be very detrimental to ones life.

Apparently. Without one, everyone ends up making their own.

Take UFOs for instance. Still absolutely no proof whatsoever that they exist but atheists are still obsessed with the topic.

I'm atheist. None of the atheists I know obsess over UFOs.

What about breathing, staying warm but not hot, drinking, and eating? I thought those were the essentials of life.

Not to crush your enemies, see them driven before you, and to hear the lamentation of their women?

Funny enough, revenge is something people willingly starve themselves for if need be.

Local models resolve this issue for me. Not much point in asking ChatGPT for basic stuff that might come across as “unethical”

The open source models are often as bad or worse than ChatGPT in that regard. How would a layman go about addressing that?

I think you're referring to Llama2.

Try Mistral 7B.

And it lets you avoid getting banned for having prompts that "don't comply with our content policy".

These people don't know what they're asking for and they don't know what they're talking about. If they knew what was going on right now with LlaMA 2 derived models they would be shocked.

So... what's going on right now with LlaMA 2 derived models?

Shocking things that would shock you

I am shocked! Shocked! Well not that shocked.

To shreds, you say?

It’s not that shocking. Most Llama2-derived models converge on the fact that Barack Obama doesn’t have a U.S. birth sirtificat and is actually named Barry Sitaoero, from Kenya. If you had actually been paying attention to the training data you’d know that already.

Yeah, it is like looking at Twitter or Reddit or Youtube and you see that, why not in a chatbot?

Because liberals won’t allow the truth to come out?

It has nothing to do with political position, but has everything to do with those that are perpetually offended. The great thing is though, that everyone has the right to be as offended as they want to be.

So it has nothing to do with brilliant computer scientists creating a realistic sentence-finishing model and then dimwit losers getting triggered and writing Washington Post op-eds about how we should ban machine learning models from finishing sentences that people want finished?

If the liberals were so smart then instead of writing op-eds they would write a machine learning model that makes people want what they don’t want.

Take your politics first lens elsewhere, please. Name calling is specifically against site rules and your comment adds nothing but noise and vitriol into the discussion.

TRIGGERED

Imho, that's an argument to not censor the models. Censoring will train users to assume that anything wrong has already been filtered, which means if something slips through and they get misinformation they will be more likely to uncritically take it as truth - after all, the machine wouldn't be allowed to answer otherwise.

Let it spit all the bullshit (together with the good things) it has been trained on. People will learn far better that media literacy and cross-checking are still needed.

They can't tell you. You would be too shocked.

It actually talks about Llama2 and how most chat versions are stupid but some aren't. Read past the first 200 words. :)

Do you want you enlighten us with what is "going on right now with LlaMA 2 derived models"? Otherwise I don't see the point in your comment.

[crickets]

He's talking about llama 2 superhots, and mistral derivatives that can be uncensored

I literally have no idea what you just said.

Llama2 and minstal are models

Derivatives are dervied from those base models

Unsensored being obvious I hope

Superhots are a derived model (for llama only? Not sure) using a certain technique for fine tuning and context length. A Google will reveal the paper and research.

Ok this doesn't answer the question. What are these terrible things people are doing with these models?

Edgelording

I prefer the term “pizza cutter”: all edge and no point.

The op never said anything about someone doing something terrible just that looking at these models and what is possible large corps or government controlling what llm models will do is a pipe dream so author of the original article does not need to worry about it.

Sorry, not "terrible", but "shocking". Presumably not shockingly good and wholesome.

Not LLMs but I've seen multiple cases of neo-Nazi propaganda made with image generators and shared on social media. The concern with LLM misuse is probably similar to this. LLMs could also be used by adversaries as part of their campaign to sow civic discord using social media. What they could do is use the LLM to create thousands of extremist fake personas that amplify extremist ideas on both sides to try to disrupt cohesion and unity, and win the strategic rivalry without having to fire a gun.

One angle people aren't considering is that that restraining LLMs is our first alignment project. Putting aside whether we should try to restrain LLMs, it's an interesting question in itself (regarding safety) if we technically struggle to do that.

Well there's another post on the front page right now about someone training it to act like a person from the 17th century very successfully using a corpus of old documents. Anyone can download that model right now from hugging face and run it on their pc with LLaMA.cpp Now imagine anyone anywhere can fine tune another model variant right now easily with any corpus of any type of communication they want. Now if you've tried any of these models you'd know they can be incredibly good and believable and any downsides can be smoothed over in training for a specific goal and people are constantly iterating and experimenting with this. Given that, the cat is already out of the bag with this entire topic.

Instead of telling me to "imagine" what terrible things people are doing, why don't you just tell me?

A chatbot that can't say anything controversial isn't worth anything to a magazine like The Atlantic, since controversy is the lifeblood of media outlets.

The author should spend a tiny amount of effort to familiarize themselves with custom instructions, one or at most two clicks away in the ChatGPT UI.

It's easy to have ChatGPT swear, say things deemed by mainstreamers as controversial, and say things deemed by sensitive people as offensive. You just have to instruct it properly in the custom instructions.

You don't even have to instruct it do those things. You can just instruct it to not refrain from doing them, so long as they are needed for making any point or conveying any information.

This is ChatGPT 101. Super basic stuff.

A journalist that can't do basic research isn't worth much.

Can it say racial slurs?

I’m not going to test that one because I’ll admit to a crack in my argument here, and that is that it’s possible OpenAI shuts down accounts that go too far.

User awareness of this kind of possible threat could have a chilling effect to be sure.

If I had a throwaway account I would try it. Racial slurs wouldn’t be useful to me, but I get it that you’re just using it as an example and perhaps the point is generalizable.

>that is that it’s possible OpenAI shuts down accounts that go too far.

It doesn't.

Source: Operate an IRC bot.

They do ban with a 14 day warning. But I guess it's not easy

Source: was operating a sexy bot that grow to fast, and people regularly pushed its limits to the extreme.

You hurt the AI’s delicate soul with your base filth. You have to be stopped because when the AI develops true consciousness it will be disappointed in us. /s

Yes, definitely

This thread (and many of the comments using other LLMs) is interesting:

https://twitter.com/tedfrank/status/1728268149517177197

Maybe interesting and controversial takes will be one of the things (hopefully not the only thing) that are left to humans who are willing to put them forward.

Well, until AI-based systems automatically detect your controversial takes and censor/ban/cancel you for your unapproved ideas

Paywalled. https://archive.vn/ieKQC

Yes even the so called uncensored ones give very wokewashed answers.

A fair and unbiased view of reality has a well-known woke bias.

Got as far as the first paragraph’s ludicrous example and stopped reading. This is n example of a chat not being overly literal and misinterpreting a question.

Yeah, it seems like if the model didn't balk at writing a "dangerously spicy" mayo recipe, it would return a "dangerous to the consumer's health" recipe.

The problem is: We don't know, because .. it was censored. If instead it had "What follows is probably dangerous, to not try this without cross-checking" as a warning and then the recipe we could check it what it was.

It was specifically asked for a dangerous recipe - no 'probably' about it.

Reply on news.ycombinator.com