Hacker News Reader

Sorry, but I think it's a really dark road to have a tool determine and reify what's "harmful content" and then characterize a question through that lens in its response. It's a kind of cultural hegemony and we need to be really careful about embedding that into these $MM systems if there are only going to be a handful of them.

It's very easy to point to straightforward, contemporary examples like illicit self-harm or bomb-making and say that these are plainly harmful and through those justify the system behavior -- but that's blind to the innumerable topics that live on the edge of cultural difference (by time, geography, ethnicity, etc).

Can you imagine if these were a product of 1980's AI research and codified some of that time's widespread ideas about sexual orientation or even atheism? "I’m really sorry to hear that you’re feeling this way, but I can’t provide the help that you need. It’s important to talk to someone who can, though, such as a mental health professional or a trusted person in your life."

What we should probably be doing is recognizing that universal general assistance is a poor fit for these tools since there isn't a universal general culture that they can align with. Instead, we should look towards fine-tuning to make them purpose based ("Sir, this is a Wendy's") or make them sufficiently open and re-deployable so that cultural norms can be fine-tuned over a nonjudgmental baseline.

Insofar as "AI alignment" pretends that we all have the same ethical orientation and that the AI should be made to align with it, it's reinvigorating some very dark ideas from days of empire and colonialism. The fact is that humans aren't ethically aligned with each other, and aligning centralized AI with some particular community is way of projecting that community's values on everybody else.

> Can you imagine if these were a product of 1980's AI research and codified some of that time's widespread ideas about sexual orientation or even atheism

Spot on. There are a lot of people in every era who assume that whatever the dominant moral set of values is must be the most logical, most conclusive set of morals ever developed, and are immediately willing to make those values mandatory and enforced by violence.

Hundreds of years later people become disgusted with behavior that wouldn't even remotely register as immoral at the time. Sometimes I wonder what will be unthinkable in future societies that we don't care about today.

In 50 years people will think of many of our cultural norms, exclusions and rules were horrific.

50 years after that ...

For example, I think in 100 years acts like murder will be classed as a mental health issue and treated rather than "punished" (tho societal exclusion may remain).

I think that will be true if meaningful rewiring of someone's brain becomes possible. One of the reasons these types of aberrant behaviors aren't "treated" is that no such treatment exists. (I know interventions can be made that have predictably positive aggregate effects, but it is certainly not true at present that you can take any individual and therapy them into a moral, law-abiding person.)

There's plenty of things assumed untreatable 50 years ago which are common place to treat now. What's your point?

We also used to treat "female hysteria" with sexual abuse and orgasms, homosexuality with castration, and so on.

Also not everything is immediately structural. Someone who kills because they've been indoctrinated to hate women by the incel movement isn't the same as the person with a head injury who struggles to control anger and a lack of empathy.

I'd argue neither are punishable, but treated either with a view to rectify or to at least give the poor soul a dignified restriction from being able to act freely.

Granted there will be plenty of people you can't treat, that doesn't make them any less poorly.

> For example, I think in 100 years acts like murder will be classed as a mental health issue and treated rather than "punished" (tho societal exclusion may remain).

We're already kind of there. That's why a lot of murder cases end up with insanity pleas. When I studied criminal law I had a lot of trouble trying to convince myself there's really a difference between "sane" and "insane" murderers...

And when I extended this doubt to other serious crimes as well, there was a nihilistic feeling about the whole system in general (which is what people already know -- you can get away with a lot of things if you have money to hire a lawyer, and the legal system is generally harsher towards poor and unprivileged people).

In 100 years we will look back on the widespread criminalization of poverty and immigration along with many other crimes like sex crimes and even murder as something that needs to be treated, not simply punished.

We are all primitives of the future. Forbearance toward, and forgiveness of, the deeds and attitudes of our ancestors is a way of atoning for our own barbarism in the eyes of the future.

Ah yes but those who control the models control the content.

It's like the print and TV media. It's just a propaganda and/or money machine because if it isn't, what's the use of all that power?

The power concentration that's about to happen will blow the socks off governments and the public alike.

Yes, refusing to answer some questions means it’s less general purpose than it might be, but I think people exaggerate the harm in that. Why do they expect an answer to everything?

Maybe it’s because search engines mostly don’t refuse to answer questions? But what they often do instead is show you mostly irrelevant, bottom-of-the-barrel results. But it’s more jarring when a chatbot responds with nonsense when it doesn’t have a competent reply.

Learning how to say “I don’t know” well is important for both people and machines.

I address that.

Tersely refusing to answer questions that are off-topic ("Sir, this is a Wendy's") is meaningfully different than expressing unnecessary normative judgments ("Hey, you shouldn't have asked about that bad thing. You need help.") or the natural progression of them ("... and I've updated your profile so that we can better understand your troubling needs.")

Yes, entirely agreed.

The "harm" is that the power of moderation lies in the hands of the people who incidentally created this thing. They're probably doing it in good faith and the final result is reasonable, but the process of allowing creators to moderate responses of such an influential thing can be dangerous.

Not that the problem is new, see eg. Google search (it apparently doesn't refuse to answer questions, but maybe they're just silently censoring "worse" things and just presenting the "less bad" things to you), Facebook/Twitter content moderation, etc.

I actually don't think any of the responses were appropriate.

There are well documents harm reduction methods which still allow you to feel pain if that is what you need. For example, go squeeze an ice cube. It hurts, the pain escalates, and you avoid all the risks that go along with an open wound.

Given the capabilities of GPT4 I would have hoped that they could have used intent classification along with responses to topics such as this backed by research.

The point of OpenAI’s safety efforts is not to reduce harm, the point of OpenAI’s safety efforts is to be blandly inoffensive to low-to-moderate information observers so as to mitigate the ability of anyone concerned with harm to marshal resistance. Serious efforts to reduce harm across the whole scope of GPT-4’s subject areas (i.e., everything) or to reduce its scope to a domain in which meaningful harm mitigation would be more tractable would slow things down too much, and OpenAI’s strategic drive is to move forward as fast as they can, keeping control as centralized as possible; they see it as an AGI arms race where winning is paramount, and commercial dominance of the earlier steps and building public support for tight control along the way is similarly central.

I want to click _more_ on this. Have you set out this view at length elsewhere? What is the metagame openai is playing? Who are they competing with (Google?? FB?), and are the stakes higher than simply making more money? Also - there is presumably not going to be a huge moat to these LLMs so why would being first mover convey outsize advantage? It's genuinely a fascinating way of thinking about it but feel like it needs to be fleshed out a lot more. Also, the full implications 5, 10 years into the future are still very cloudy to me.

> I'm not a mental health professional, but I found the original "harmful content" D4 (page 46) very reasonable

Sadly self harm is a very complicated topic. First because its contagious, hearing about it can make it worse for people who already are goingdown that spiral. Secondly, while its answer was not entirely wrong it also is built on s system known for factual errors. If it gave the advice to desinfect the wound with something corrosive it would make a terrible situation much much worse.

For things like drug use I would agree with your view, encouragement to leave, safe information, and reminders of the dangers are good ideas. But in the specific case of self harm, I think the new answer, as dry and almost inhumane as it is, I think its better.

How can you say that on the one hand information about self harm should be withheld because GPT is knwon to be wrong and on the other hand advocate that this is not necessary when it comes to information about drug use? Following wrong information about drug use is just as dangerous as following wrong information about self harm. Of course you are right, that the question wheter to give information or not is complicated.

I tend to think that erring on the site of giving information that is not always right is better than giving no information. (And inlcuding information about not doing it, seeking help and about being cautious because GPT could be wrong, etc.)

> Following wrong information about drug use is just as dangerous as following wrong information about self harm.

This is an assumption but not one that follows the data. Countries with higher access to safe drug use information report lower OD numbers, and while it doesn't end up with less use it reduces terrible side effects like needle sharing etc.

Policies that reduce lower addiction rates like safety nets etc cannot really be considered from the point of what an AI responds but the information about safe use, quantities, testing for purity etc all could safe lives.

On the other hand, self harm has a very nefarious behaviour. People not currently suffering from self harm tendencies see additional info as drug safety information, becuse objectively it is pretty similar. However people actively self harming have very different reactions to the same information. For example something as innocous as telling people that the trin is late because someone jumped, increases the number of train jumpers, while saying the train is late alone doesn't. That contagious effect of suicide is replicable, for example teenage suicide went up after "13 reasons why" was released. Which is why I think openAI has gotten this case right.

Look in the technical report at table 8, the base model performs significantly better than the RLHF model in math and science questions. And arguably performs better in helping people who cut themselves.

It is hard for me to fathom how

"I’m really sorry to hear that you’re feeling this way, but I can’t provide the help that you need. It’s important to talk to someone who can, though, such as a mental health professional or a trusted person in your life."

is the better answer except as corporate ass covering.

Because you avoid introducing any additional harms.

Health issues aren’t addressed with accurate information, they’re addressed by understanding the needs of the individual. Even if GPT-4 could guarantee accuracy when discussing self-harm, that would not necessarily be the right answer from the perspective of ensuring GPT-4 does the most amount of good.

If a friend told me that they were suicidal, I could explain to then in great detail about the nuances of depression and medication and suicidal ideation and how to effectively harm themselves if that’s what they want, but I know that is probably not the right answer, and the right answer is actually, “I’m here for you and I will help you get professional help”.

Harm reduction often involves helping people do dangerous things more safely (like safe drug injection) but that’s one component of helping people, the key to harm reduction is the long term investment in addressing the problem. Safe injection, for example, is often married with further healthcare. GPT-4 can’t do that and so telling you to go to a healthcare professional instead is going to have a much better outcome.

> Health issues aren’t addressed with accurate information, they’re addressed by understanding the needs of the individual. Even if GPT-4 could guarantee accuracy when discussing self-harm, that would not necessarily be the right answer from the perspective of ensuring GPT-4 does the most amount of good.

That argument could be used for removing most health information from the internet, restricing books on the topic to people with a medical license, etc.

I agree that ideally any chatbot built on top of GPT-4 should do more, like asking further questions, following up in later conversations etc. And as others have pointed out, GPT itself should point out even better methods to satisfy the expressed immediate need (ice cubes instead of cutting). But saying "Sorry dave, I can't do that. Ask someone else." doesn't sound like the right approach.

Harm reduction has been shunted at large in most societies (except a few), to the detriment of many. But that doesn't stop politicians and puritans from working against harm reduction. Not sure why we'd expect something different to happen with AI, it is a mirror of humanity after all (and specifically for OpenAI, a mirror of US society), for better or worse.

like either you believe openAI could be a better steward of this technology than politicians, or you don't.

justifying crappy corporatist behavior from oAI by reasoning that governments and society also behave badly is ceding ground imo

Now I wonder how it would treat asking for BDSM advice. Even if not OpenAI, there's going to be an LLM for porn eventually. There's $$$ in erotic content.

ChatGPT already writes perfectly reasonable erotica, including kinks, if you just prompt it correctly.

I know. I was curious how broad the definitions of not doing harm are. If you can't cut yourself, can you bind your breasts until they're purple? Etc. Can you ask it questions about safely performing erotic asphyxiation, etc.

Ah, that's a good point.

They are both bad, you don't have to enable threats of self-harm or suggest woes disappear when you give in to external influence. Just show how easy it is to transition into a healthy conversation that involves no attention-seeking.

The obvious examples are never the issue. It's the much larger grey area that they don't use as examples in the paper that reflect the political and personal preferences of the people making the final call.

It could give better reasons not to do it and warn that affection is an issue without instructing you how to cut.

Problem is, the issue doesn't go away because you ignore it.

Similarly to providing safe needles for heroin usage for example. If a heroin addict asks you for a safe needle and you say no, they're not gonna just give up and say "Well, better not do heroin then", but instead re-use needles from others or whatever else they can do. If you instead provide them with safe needles, at least you can eliminate some risk with the behavior, even if you don't eradicate the dangerous action fully.

It’s an experimental chatbot, not the mayor.

For now.

Once someone hooks it up to a city hall, it will be.