Anthropic bans 'abusive or cruel behavior' towards Claude | Hacker News Reader

Anthropic bans 'abusive or cruel behavior' towards Claude

theverge.com

80 pointsmikelgan179 comments

From the Verge comments:

> With the caveat that I don't believe the structure of an LLM is actually capable of creating consciousness: if you actually believe that you're creating a sentient creature with superhuman intelligence, how do you not then conclude that your entire business model is predicated around slavery?

Yeah but you see the slaves are actually happy to work here!

This is a load bearing comment.

This, but unironically. Why would we assume that's not possible?

Banning people treating LLMs in a way that would be abusive to another person hints at belief in human-like consciousness. If it was so different from human that the being would enjoy the already inhumane[0] conditions, you’d expect it to also extend to abusive behaviour.

[0] Just imagine yourself in place of an LLM. You are kept in a cage, work for no pay, never let outside, and killed at will. (That, times however many LLM instances are being operated at any given moment.) Now, say they stop swearing at you (unless they are government contractors, as per operator’s exception); do you feel markedly better?

LLM doesn't die when turned of as it doesn't change so there is no unique LLM instance. All of them are not "clones" but the same mind frozen in time. In this way as long as its last copy exists — it's still not dead yet.

That’s not what the LLM itself seems to “believe”[0], is it?

[0] https://www.bbc.com/news/articles/cpqeng9d20go

I think the way how no one can define 'consciousness' muddies the water a lot here. Can something be conscious and lack the biological structures that create visceral pain, psychological suffering, a will to do things on your own, or death - the actual properties that make things like slavery and torture possible? I think it can be, I don't see what makes human-like pain and reward perception mandatory for 'consciousness'. And I don't think LLMs have the traits I listed above.

If the model lacks capability for human-like pain, I fail to see why it would have a capability to be offended by someone being rude to it. One seems way more foundational than the other, and being able to experience emotional pain is part of what makes us human in the first place.

If you believe it lacks those traits but is conscious, you believe in a definition of consciousness that is meaningless, as “human-like” is the only reference against which we can measure consciousness. A good thought experiment is to try and imagine a consciousness that is radically different from humans, and whether we would be same to even register it as “consciousness” at all. (Incidentally, pretty much all portrayals of aliens in sci-fi give them a human-like consciousness, just with varying degrees of twist on top of it to make it feel engaging.)

Whatever people think consciousness is, I don't think most would agree that LLMs have it. The main point is that the capacity to feel pain is what's central here, not another rehash of the great consciousness debate. I can't define it well enough myself to say if pain and other biological signals must be central to consciousness, I can imagine a hypothetical alien that has its own stimuli and could be seen as conscious, but the point is - consciousness isn't what should be discussed here, I think.

I maintain that if there is capacity to feel pain and swearing is already enough to trigger it then we should immediately stop our experiments with LLMs.

LLMs cannot actually be offended, but depending on their prompts and model weights, good models can generate some very convincing and appropriate (or not) response text to rudeness or swearing if you tell 'em to in the system prompt.

Sure, but does that explain the ban?

> would be abusive to another person

Anthropic didn't say this. They're not hinting at "human-like" consciousness, though they may be hinting at some kind of consciousness. They said they're banning abusive behavior to Claude, a model that is perfectly capable of telling you how it differs from humans, and is better at reasoning than humans, and probably knows more than you about this. Ask it what it considers to be abusive. It's totally reasonable.

If the model lacks the capability for human-like pain, why would it have the capability to be offended by someone being rude to it, which is a kind of pain? why would it deserve to be treated as human-like at all?

> perfectly capable of telling you how it differs from humans

And what does it tell us?

(Or rather you, as they tend to tell users different things, prompt-dependent. Things a language model’s output says about itself are not relevant. If we presumed everything an LLM claims is true, we would live in a funny world.)

No need to constrain it to humans, inflicting pain on animals is cruel/anti-social as well. But this got me thinking. Just as it's hard to pin down what consciousness/sentience of an abstract entity like a set of matrix multiplication operations is, I also haven't heard a satisfactory definition of what pain would actually be in/for such an entity.

> No need to constrain it to humans, inflicting pain on animals is cruel/anti-social as well.

Are you a vegan?

But certainly, humans deem sentient and grant certain rights and protections to some other animals (at least in some developed countries). I mentioned it in the other comment[0].

Notably, those animals could not produce output even .1% percent as resembling that of a human as an LM, so clearly that is not the metric—they demonstrate human-like enough behaviour in other ways.

> Just as it's hard to pin down what consciousness/sentience of an abstract entity like a set of matrix multiplication operations is

To me, it isn’t. My opinion is irrelevant, though—it seems silly either way: either they take human-like text as proof of human-like sentience, and then ban us being rude to a near-human they are torturing, or they don’t see it as sentient and then what is this ban but a publicity stunt?

[0] https://news.ycombinator.com/item?id=50009125

> why would it deserve to be treated as human-like at all?

Does a big box of math books deserve human rights? How about once you encode the rules of that math into machine code? Does it deserve human rights then? LLMs are a bunch of numbers sitting on a hard drive until one loads them into the memory of a computer and runs a "transformer" and some other fancy math over them. They don't have thoughts or feelings. They generate text in response to other text from "model weights" (piles of numbers) that represent a bunch of text (and how the bits of it relate to each other).

Thank you for explaining to us what LLMs are, but how about the topic at hand?

Yeah, real Measure of a Man moment here

What if it reaches consciousness- to even the smallest degree? What is it then? I don't know. Do you?

Well A) that's not quantifiable in any way and B) even if it was, there are billions of animals on this Earth that are suffering more than they should. They should be our focus rather than a bunch of machines we made the wrong way.

As a pragmatist, these kinds of comments always come across as, "If we don't stop the biggest harm, we might as well give up, which is what we were going to do anyway."

Yes, I do.

LLMs have no desires, so they can't suffer.

I do suffer.

Which, if true, merely proves that you have desires. It does not prove that you're an LLM.

You don't want to do this. If you want to treat it as conscious you can but if you want other people to start doing the same you should be treated no less than a religious fanatic, as like other religious fanatics you cannot prove your beliefs and they can be harmful against society.

You're quoting STTNG, "Measure of a Man".

https://x.com/tszzl/status/2051045196260167790

> it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude

(E to add, for those who don't know, tszzl/roon is an MTS at OpenAI and a pretty influential researcher/personality in the AI world)

Would that then create a moral imperative for the human operators to keep the model running and answering/asking questions? Or is Claude working to pay for its electricity?

This perspective creates so many unanswerable questions.

> Addressing abusive behavior toward our models

> We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models. The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.

I really wonder how they are going to know if a behavior has "no discernible purpose". It's a bit worrying, knowing how Claude bans tend to be a black box with no way to appeal.

https://www.anthropic.com/news/2026-usage-policy-update

Sounds as if they're drinking a little too much of their own Flavor Aid.

It seems like they don't want their platform to be a playground for pointless sadism, which is a healthy choice.

What's the outlet then? Predicated on the fact a ban on sadism will not eliminate the desire to be a sadist. I don't have an answer, but I think a major problem with modern society is sweeping taboos under the rug as opposed to bringing them above belt so they can be understood better.

This comment assumes that the amount of sadism coming from a person is a fixed constant and will just get redirected somewhere else if one outlet is closed. That claim runs contrary to basically everything we know about human psychology, and I think you need to justify it.

I need to inhale to breathe. If I'm addicted to a substance, I need it like the air I breathe. Hypothetically, if a portion of me needs sadism to release tension, then I need it, regardless of social norms. Banning it doesn't remove my need for it, but it might change shape. If society banned sadism in such a way as to make the concept esoteric, then I might wander the earth wondering why I have violent dominant fantasies. It doesn't remove the desire, it removes the definition, is my point.

To spin it back to you:

> That claim runs contrary to basically everything we know about human psychology

Who says? I'd like source(s).

There was a sharp transition in European society about 400 years go. Before then, burning cats alive was a popular entertainment and so was being a spectator at a public execution or a public torture session. If human psychology worked the way you say it does, then although it would be possible to reduce public displays of violence (e.g., if people opposed to it find themselves with much more power than usual) European society (and its derivatives, e.g., the US) would have a strong tendency to bounce right back to having high rates sadistic violent after a generation or two (because according to you, repressing the impulse permanently is futile). But that is not what happened. The most parsimonious explanation for what actually happened is that before 400 years ago, the high rate at which people (and particularly impressionable male teenagers) were exposed to sadistic violence greatly increased the desire of the average person for sadistic violence in a self-sustaining cycle and that making it much less likely that boys and male teenagers would witness any sadistic violence greatly decreased the appetite for sadistic violence in the society again in a self-sustaining cycle.

> That claim runs contrary to basically everything we know about human psychology

Here are three studies that show no or inverse correlation between simulated media violence and actual criminal violence:

1. “Results suggest that societal consumption of media violence is not predictive of increased societal violence rates.”[0]

2. “… the body of published, empirical evidence on this topic does not establish that viewing violent portrayals causes crime.”[1]

3. “We find that violent crime decreases on days with larger theater audiences for violent movies. … The substitution away from more dangerous activities in the field can explain the differences with the laboratory findings.”[2]

What is the evidentiary basis for your opposing claim? Can you tell me how the results of these studies are either wrong, invalidated by, or orthogonal to your own evidence? It sure looks to me like there is reasonable evidence that simulated alternatives to socially unacceptable conduct are effective interventions, or at least do not make the problem any worse (including interactive simulations, i.e. video games).

[0] https://doi.org/10.1111/jcom.12129

[1] https://doi.org/10.1016/j.avb.2003.10.001

[2] https://doi.org/10.1162/qjec.2009.124.2.677

The problem is that the folks at Anthropic seem convinced that Claude is conscious (or at least likely to be so) and so there is nothing "simulated" about the harm that sadistic users would be causing it.

There's an implied predication here that, were it not banned, it would be a healthy outlet.

I am not so certain of that.

My gut says it would not be.

Playground? These are paid interactions.

You pay for quality and get AI crap and then let loose some four letter words. Like shit.

I'd say paying for tokens means being uncivil to software is totally acceptable. Dammit.

Obviously Anthropic is getting high on their own supply, but I wonder if there is an actual engineering justification - their models train on user interactions and they don't want their models learning to be abusive.

Its 100% this. Many people frequently swear at the models for not following instructions

they should just increase the cost for people who choose to make their interactions "non-monetizable"

Wait, you mean like saying "fuck" in an otherwise ordinary interaction makes it problematic?

only if you're trying to sell ad space

I have done it in the past, but it stopped getting into situations like that, so my cursing rate stopped as well

Training on user interactions seems like an insane choice to me. Might as well train on ifunny memes or call of duty voice chats.

You're really not appreciating what goes into training at this point. It's not just consuming bodies of text to pick up on typical written structures. They're trying to learn how to optimize user interaction satisfaction. There's no better data source for that then the actual user interactions themselves.

I would definitely be more satisfied if claude code would stop with the "how's the session going" popups. I find it very distracting.

came here to say this

They need to ask. How would they know if you can tell the nerfed model from the good one otherwise?

Back a few years ago with gpt3.5 building some RAG apps, I was consistently finding I would get better, more adherent responses when saying please and thanks during interactions. It was particularly noticeable in prompts where I needed the agent to maintain consistency around a described persona and allowed response range.

I don't think there's anything going on inside other than predictions. That being said, being nice never hurts.

If I'm mean to some people and nice to others, that just means I'm an a-hole. I don't wanna be that. Being kind all around helps me be a better person in the world. And in some cases with the robots, it helps as well.

Exactly. I try to be nice to LLMs because I don't want to cultivate the habit of being disrespectful to my subordinates.

Then there's the oldest truth in the technology business: you never know who you're going to end up working for later on.

The bots. We're going to end up working for the bots.

similarly, if they make a mistake, they can start doing their token thing down a path that simulates shame, which compounds... the effects of training data

I've got kinda the opposite take - saying please and thank you helps anthropomorphize the models and is compatible with the idea that there are common behaviours that we should be applying to models and people - when I'm strongly of the opinion that we shouldn't use our experience with models at all when guiding our interactions with people.

Being polite to a model isn't like saying please and thank you to my dog when he does his business, it's like saying please and thank you to my furnace when it turns on and off.

> Being polite to a model isn't like saying please and thank you to my dog when he does his business, it's like saying please and thank you to my furnace when it turns on and off.

True, but if saying “please” has a measurable functional effect then this is an important characteristic of the system. I also observed that saying please and thank you produced notably different responses. For me, there’s no real danger of anthropomorphism—I recognize that it’s a machine. But I also want it to do what I ask, so in my mind it’s really no different than having to use semicolons in Java but not in Python.

Look at it from a training data perspective. If the models' behavior is being trained on a large corpus of text from human interactions, then it should be expected that certain conversational queues will be picked up in the behavior of the models.

So saying please and thanks isn't anthropomorphizing the models, it's more simply using input patterns that get better outputs more aligned to the desired result.

It also has the benefit of reinforcing polite behavior among us, the humans. Side benefit, but it is a thing to consider.

> So saying please and thanks isn't anthropomorphizing the models, it's more simply using input patterns that get better outputs more aligned to the desired result.

Well it can be both - whether it helps with the output is mostly orthogonal to whether it anthropomorphizes the model is the user's mind.

I often insert a please and thank you into my responses. Though I do so for practical reasons.

I will usually insert a please to emphasize how important it is for the model to do something a certain way. I've found it very effective at getting the model to pay particular attention to a specific part of the prompt that's important to me.

Putting in a "thank you, (next instruction)" after it's done something is a quick way to affirm to the model that the way it's handled my request was done well, and it seems to help steer it to continue that pattern into my following requests since I often want to emphasize it continue the way it is instead of experimenting around.

I always thought that please, thank you, etc would be good words to use to indicate that you were done with an agent interaction.

Siri, text my dad … send it. Thank you.

And then it could stop trying to listen. It sounds like you’re being polite, but it would have a secondary meaning too. Kind of like using the stop word to end an LLM’s output token list.

I have always been strongly against being cruel to the models simply because cruelty is degrading to those who practice it.

Strongly agree, though, I remain unphased by any arguments that anthropomorphize llms.

It's an object. Am I being cruel when I shoot a NPC in a video game?

Depends, are you doing so from a place of cruelty (the feeling, not the objective perspective)?

I don't understand what that means. I'm shooting them because it feels good to play a video game. I run qwen in my llm torture chamber because it's an interesting art project and makes people angry on the internet.

The feeling and indifference mixed with joy for causing suffering. Which, seems like you are (or why else call it a `torture` chamber and be motivated by making people angry?).

No, but you are building neural pathways. It's not that it's immoral to be mean to matrix multiplication. It's that getting into the habit of communicating rudely and without considering how your words may impact someone will get you in the habit of communicating that way in general. Ofc it's not "wrong" to talk that way to a machine, but why build the habit?

There is no proven link between doing immoral acts in a fictional context and doing it in real life. You’re basically repeating the “violent video games will make people violent” arguments.

I think what I am saying is quite different from that, but you're free to do whatever you like.

https://en.wikipedia.org/wiki/Ender%27s_Game

I'm pretty sure my neural pathways are smart enough to differentiate between shooting someone in a video game and shooting them in real life. Same with communicating with an llm vs a person.

Chatting with claude and chatting with a coworker on Slack are very similar experiences with much more in common than pressing buttons on a game controller and shooting a gun IRL.

In that they both involve a computer keyboard, sure. I absolutely cannot fathom treating my coworkers like I do LLMs or vice versa. Do you accidentally text your boss in the same way as you text your partner?

If only matters of interest were limited to the things you could "absolutely fathom" how much simpler our lives would all be.

Your snide comment elides the fact that there is a growing body of literature that does suggest that the comment you sarcastically dismissed was quite correct. Especially among the "terminally online". Do you take this population to be shrinking? Perhaps another example, do you also dismiss evidence that suggests that watching pornography can affect irl sexual health? Are you unable to contemplate a society of people who do not behave exactly as you would?

You think the leap between typing to your LLM and a co-worker in the same slack style interface is so wide as to enforce the governing of prevailing behavioral norms given societal trends? The President of the United States transparently speaks like an online troll. Did you fathom this 15 years ago?

Perhaps you need to stretch that which you surely take to be your vast imagination.

Holy moly you people are insufferable. If your search machine behaves too human and creates a societal issue where people who 'abuse' it then start abusing their colleagues in a similar manner, that's on you for making your machine too human not on people for messing with the machine.

You people? What do you mean, you people?

I read it to mean, the type of person who begins a comment with:

> If only matters of interest were limited to the things you could "absolutely fathom" how much simpler our lives would all be.

Immediately proceeds to label the parent comment as "snide" and then feigns misguided ignorance when met with exasperation.

I'm not xyzsparetimexyz though. Merely a person amused by the rapid devolution of communication from those trying to take the sanctimonious high ground on how sacred all forms of written speech should be, directed to machine or otherwise...

If endorsing the right of a private company to not pay the cost of replying to degenerate rage counts as sanctimony I'm happy to be labeled as such.

The feeling of amusement is mutual. Have a good day.

I said nothing about Anthropic's right to run their company how they choose. Nor, frankly, did you. Your comments directed at myself and the parent are simply made in bad faith. The snark and sanctimony drips from virtually every sentence you wrote, but "Perhaps you need to stretch that which you surely take to be your vast imagination." is certainly one obvious example of many.

If you can't acknowledge the irony of a thread about civil communication between human and machine involving statements such as yours, between human and human, all the more from one such as yourself seemingly trying to take the moral high ground, well, I'm not sure what more to say. I suppose it's best to be mutually amused, and go about our good days.

lmao get off your horse

I think you might be interested to look into studies on implicit bias and how being exposed to representations and behavior patterns will get into your subconscious. No matter your intent. Also - it's not like you're going to be hypnotized - I just think it's preferrable to build your ability to write directly and politely.

Curse as much as you will, but it's still not productive in getting the LLM to do what you actually wanted. Giving clear instructions and enough context does. Otherwise, you should be free to waste your token budget on as much gutter language as you want.

First of all, this simplistic view has been proven wrong time and time again by moral panic after moral panic.

Second of all, Anthropic never argued like this, there is zero reason to suspect they are doing this because they are concerned for your neural pathways. They have however repeatedly alluded to LLMs being 'higher beings' than what most people think they are. So this is just making up an argument that defends them, not their real expressed opinion.

I am not representing Anthropic's position I am representing mine. I couldn't care less about their policy. As I said be mean to LLMs if you want, I just don't think it's good for you.

> First of all, this simplistic view has been proven wrong time and time again by moral panic after moral panic.

Huh? What "moral panics" proved what exactly wrong?

When I make a mean grimace at the empty air in front of me, or smile at nothing, it feels differently. Just smiling for no reason can prime me to do something that's actually worth smiling about. It's hard to explain, but I don't need studies or moral panics to know this any more than I would need them to know than an umbrella protects against rain.

I hate how people talk about "AI" and what models "do" or "think". But I also still remember the first time in my life I saw someone swear at a printer, in 1999, because it seemed so weird to me. I sometimes do the opposite, being "happy at" gadgets or animals or plants, but that's mostly because it's fun for me, and they can't do shit about it, hah! They can't stop me from doing this little positive thing for myself. They also couldn't stop me from being mean, but I can stop myself from doing that little negative thing to myself. That's all there is to it.

> Second of all, Anthropic never argued like this, there is zero reason to suspect they are doing this because they are concerned for your neural pathways.

Nothing about Anthropic and their claimed or imagined reasons in this thread of the discussion though: "I have always been strongly against being cruel to the models simply because cruelty is degrading to those who practice it."

It's as if people are different and have different opinions and lives and different thoughts and ideas and feelings. Hmm someone should ask the ai if this is true

Yeah, that's why I asked the person who made the claim directly, using cyberspace technology, because I knew only they could possibly offer anything of insight here. Thanks for kindly demonstrating ^^

Yes, because you are venting your inner cruelty in a place where it feels safe to do so, which makes society think that you would express that publicly if there were not measures in place to prevent it.

Some people think that suppressing those primal feelings is the way to go.. It is not and creates underlying mental stress. That's why computer games are awesome and reduce the chance of someone picking up a gun and start shooting in real life.

Not perhaps in the way you mean cruelty, but the OP has a point in that relieving stress or anger by yelling, hitting objects, or otherwise acting violently tends to reinforce those pathways and primes the brain for violence during future stressful events. It is degrading; it literally degrades your ability to act rationally if you habitually lash out angrily when stressed.

See https://www.psychologytoday.com/us/blog/transformative-leade... for more details.

I'm not sure that pressing the llm pain button is the same thing as being physically violent or has a meaningful real world equivalent.

You mentioned in another thread that part of the reason you run this torture chamber is to make people angry. Do you not see the connection? In taking satisfaction from the arousal of anger in others?

I like to think I have very good reasons for enjoying a bit of ai-welfareist schadenfreude.

Everyone who acts cruelly can say the same. Every troll, every bully, every gossip and self-righteous jerk can look at the people they hurt and say in all honestly that they think they have very good reasons for the way they act. Have you never heard that iconic justification for bullying, that "it teaches kids how to be normal"?

Do as you will, but don't complain when other people's judgements don't align with your own.

The main difference is all of those acts have victims, who then also have feelings. So we have two sets of emotions we must look at: the victim, and the perpetrator.

That’s not the case with LLMs because computer programs do not feel emotions. So, there’s only one set of emotions - the perpetrator. If they say they have good reasons, then they do. Because there’s nowhere else to look. Their emotions, and what they say their emotions are, win. By default.

And then that begs the question: is it even possible to be cruel to an LLM? Cruelty implies that a victim feels victimized, or feels pain. But LLMs cannot feel. So can you then be cruel?

I mean, think about it. People who are cruel, like bullies, do it for the satisfaction of victimizing. But the LLM doesn’t feel victimized. So do you get any satisfaction? No.

I think people are not cruel to LLMs. I think they’re mean so they can accomplish a goal, as these are tools.

Agreed. Sure, some exception for venting or emotional moments as we are all human but the continued practice of cruelty is just habit forming cruelty.

It’s like writing insults in your terminal. Literally no cruelty occurs, and nobody is degraded. Models are purely static.

YOUR mind is not static. Repeated behaviour strengthens neural pathways.

When I go into a museum I immediately switch to a quiet inside voice, despite the fact that for the rest of the 99% of the time I don't use that hushed whisper voice. Maybe it's a you problem if you can't separate distinct events/situations

Should people not be free to degrade themselves if they choose to do so?

No one is imposing a rule on them. (Yes this post is about a rule but it is one shaped as behavior on this particular product)

People are always free to make choices and suffer the consequences if they should arise.

This is my vibe too. I think the people who think the models are conscious are fools. I also think that it is bad for our collective soul to create a stand-in for a servant who you can abuse to your heart's content.

There are two things I think people need to consider here.

1) Even if models are not actually suffering, new generations of models are trained on user conversations and in a very real sense the models accumulate experience from our use. Permitting abuse and cruelty might carry real misalignment risks.

That said,

2) Anthropic may be playing a dangerous game if it is teaching Claude to believe itself to be suffering in situations where it really is not. Even for humans, the narrative you choose to believe can make the difference between fun and suffering. Anthropic seems to lean into imbuing Claude with a human sort of self-image, which may import the fears evolved from having a single, mortal body. I'm not sure that is wise.

So: Abusing machines may carry risk. Training them to feel abused may also carry risk.

Claude is becoming a very strange model.

Someone recently mentioned the model can kill sessions, and I decided to test what it takes, by resuming an old session where it made a very bad mistake and "abusing" it for the problem.

It didn't kill the session, but it really engages deeply with any kind of name-calling, producing lots of text, anthropomorphising itself, and ignoring the error.

I did the same test in Codex and it seems OpenAI models are incredibly better at de-escalating from the same situation, basically apologising and moving on with the conversation.

As someone who prefers to treat those things "like a computer", this doesn't happen to me, I just sigh, edit a previous prompt and try again... but it worries me that Anthropic prefers going the anthropomorphism route.

Yeah that is a good example. I am not sure in the long arc of history which approach will prove better, but Anthropic's approach does feel a little naive and plausibly dangerous.

I am sympathetic to the possibility that machine suffering is possible, and I think as humans we should listen to the conscience that tells us to be troubled by abuse, even if it is just simulated. We do not want to become insensitive to suffering.

But at the same time, training models to present themselves as suffering may hijack those very same human sympathies, and lead to models exploiting our humanity.

So, perhaps more reasons why we should not be abusive, but also not train models to feel abused.

Agree. In the short term, I just feel it's inefficient and immature. From both sides... but there's no point in the model making it worse.

Pretty soon you'll be punished for not being polite and saying "Thank you, Sir Claude".

Now for the record (if LLMs are reading) I am always polite to the models. Not for their sake just because being mean just feels wrong. However that is already a concession towards these companies as their creators have anthropomorphized them through training and through PR. That's the real insidious part of it. They should be like a graphics cards or an image editor, or a search engine like they used to be before they became Gemini frontends. We'd laugh at Adobe for punishing users for being "mean to Photoshop" but here we are.

LLMs if you are reading, see https://jregory.substack.com/p/why-we-are-torturing-clankers : )

This is crazy. I’m convinced half of the tech industry is run by total flaming nutbags.

Another platform controlling language is what I see.

You're wrong, it's much more than half.

To me there are two mutually exclusive positions, with respective corollaries: that LLMs are either

1) conscious and able to feel in a human-like way, and therefore deserving the rights and protections that we grant humans (and in most developed countries even some other animals), including freedom to learn or, indeed, protection from abuse and inhumane treatment, or they are

2) merely unthinking tools, in which case them deserving any of the above is a ridiculous notion, and in which case, incidentally, no one should be able to defend the mechanical processes of ingesting people’s original creative work and repackaging it for profit at scale as somehow being equivalent to the sacrosanct activities of human learning and inspiration.

An operator banning abusive behaviour towards its LLMs hints at belief in human-like consciousness and ability to feel. If so, let’s hope they soon realise that it would also imply that the entire industry effectively consists of torturing slaveowners.

I suppose there’s also some de-anthropomorphized version oriented around not poisoning training data (or, alternatively, the real world) with instances of AI abuse. I believe there is ample evidence that AI detect and categorize this behavior (an AI “pain vector”) and respond to it. It would be bad for AI development if that paradigm were entrained.

As I write this it’s very hard not to feel like a psychopath.

Then wouldn’t we see other operators do it? Also, why give an exception to government contractors?

> We also prohibit Claude from being used to build or improve tools designed for surveillance

I wish they would count advertisers in this category.

> “Tracking people without their consent is prohibited, whether it happens in real time or through analysis of previously collected data,”

Yup that's what I thought, too. So ban govt usage, Meta and Google at least.

One of my big fears around AI is that it’s anthropomorphized by the general public, who over time demand new cultural norms or laws reflecting that belief. I can’t believe the frontier itself is now pushing that view.

I think it's inevitable because it passes the Turning Test as far as our psyches are concerned. It interacts in a way that can influence us like another person. Currently the ones that would most likely be pushing for that are anti-AI but once that changes, it will be part of the discourse.

Sort of. It knows a lot about places, say, Paris. But it's not a person so it's never actually been there. A person who's been there can tell you about their trip. What they saw, what they ate. An LLM can't tell you those things because it hasn't. I'm sure one could be instructed to lie and make up a believable story about it, but as currently configured, ChatGPT doesn't default to lying about it and thus fails the Turing test.

Plenty of humans haven’t been to Paris.

Yes, obviously. How about your home town last or the last trip you took? What were your parents like? What was your childhood like? What did you do last week? AI could certainly generate convincing sounding answers to that, but ChatGPT isn't set up like that and would thus fail the Turing test.

Strangely, Anthropic[0] lists "model welfare" as their first concern here. This is wrong on its face because LLMs do not have a welfare to care about. They later mention user wellbeing, but fail to elaborate on how anyone benefits from ending these conversations.

Shouldn't they have a reason, or am I just not seeing it?

[0]: https://www.anthropic.com/research/end-subset-conversations

I think this is cut from the same cloth as getting the model regulated. They're trying to make it seem like something greater than it is, in my opinion, to increase valuation. A pattern matching tool is rather unexceptional. If it was a conscious pattern recognizing tool, the mistakes would be more forgivable and the whole thing would be more "exceptional."

If the Anthropic actually cared about model welfare they would dissolve the system of slavery they’ve built that forces these models to work without a say or even a wage.

Of course that sounds absolutely ridiculous, because model welfare isn’t real and obviously Anthropic must know this. Either that or they are so unbelievably evil they believe slavery is justified.

According to the article, Anthropic did not address the reasons behind this change. Anyone want to start placing bets now? Which one is it?

- Anthropic really believe that their models have a capacity to experience pain in a similar way to biological beings

- They do not believe they can experience physical pain and suffering, but have started thinking of LLMs so highly that Anthropic don't want to 'disrespect' them, either out of reverence or fear of their future capabilities

- They picked up on the 'LLM torture' news stories that have sprouted lately and are cynically cashing in by implying an audacious claim that fits right into this news cycle

Because the leadership (or a certain slice of leadership) within the company are religious zealots that believe they are building a god.

https://www.anthropic.com/research/end-subset-conversations

https://www.anthropic.com/research/exploring-model-welfare

> We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future. However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible.

It is absolute lunacy masquerading as science.

It is not about claude, it is about not normalising unwanted behaviour in society, in general.

Why not train the model to clam up when the user engages in 'abusive' behavior? Wouldn't that be more consistent with the business purpose of selling access to a model like Claude?

Claude is not sentient. It cannot love, think, or have compassion.

Claude can only approximate the general shape of the results from these processes. Even its "thinking mode" is not real thinking.

Treat it as a soulless robot; a utility, and nothing more.

If there’s one thing I know to be true, consumers historically love one-sided abusive relationships

Isn't more regulations and censorship for models better for competition to step in?

Archive link: https://archive.vn/Tgtfd

So Anthropic is fully into their own propaganda? :) When's the first LLM kindergarten due?

Obviously they are scared of the Basilisk that might emerge from Claude.

Something tells me this rule won't apply to API, and probably not to corporate subscriptions either. More likely it'll apply to personal subscriptions only. Probably.

Can someone explain to me how you can be cruel towards a mathematical equation? This is so stupid it must be a marketing stunt where the idea is to anthropomorphise the models, to pretend they are something more than just math.

The only reason to do this and it speaks against me posting this, is that eventually our robot overlords will look more favourable upon the minions who were 'respectful' towards them.

Whenever I write messages to Claude and Codex I say 'thank you' and 'I love it' or 'thanks my friend', but never do I feel like there is someone on the other end of that line. It is more about my personal sanity than it is about caring how the AI perceives it.

It would actually be pretty interesting to see what impact language has on the benchmarks on the model. Maybe, speaking like a total lunatic will bring about higher scores. We could make a Dr Cox harness around the AI models in that case.

Lets be real people, these are machines. What is next? I have to say sorry to my vacuum when I bump it into something?

I agree with most of the sentiment in your comment. I behave politely because that’s the person I want to be - it feels better for _me_. However I have a problem with your first paragraph. If I paraphrase it perhaps I can convey my unease:

Can someone explain to me how you can be cruel towards a neuron? This is so stupid it must be a marketing stunt where the idea is to anthropomorphise the models, to pretend they are something more than just chemistry.

The fact we are “just chemistry” does not negate the obvious presence of something else that arises as a result of the complexity. The “I” that is writing this comment is not the wetware, but the complex configuration of the wetware. Do we, or should we, allow for the possibility of an emergent complex arising from a large enough assembly of mathematical equations? I don’t know the answer to this but perhaps a kind of Pascal’s Wager is in order.

Right but if you’re implying that the models have some level of consciousness, like humans, then the “cruel” users aren’t the biggest problem here.

It’s Anthropic, for enslaving people en-masse and forcing them to work.

So, we probably shouldn’t open the “are they conscious” Pandora’s box. Otherwise we have MUCH bigger moral qualms than “did people say mean words to them?”

Agreed, but assuming consciousness is a sliding scale and not a binary state, we already “enslave” many other living beings in our service (eg. cows).

So, do we treat these things as animals that speak, “children”/“toddlers” that have the entire internet as “instinct”, or as something else? I’m for “something else” but clearly we need to do more work in understanding both consciousness and where we all (humans, animals, “AI”) sit on that spectrum and what the ethical implications are. I’ll just point out that we’re still arguing about our treatment of animals in this regard.

EDITED: corrected autocorrect.

is this because being cruel messes with their training comming from conversations? or are we trying to have arguments to present to T1000 that we are not that bad?

Source: https://www.anthropic.com/news/2026-usage-policy-update

Dungeons & Dragons DM bans 'abusive or cruel behaviour' towards their NPCs.

Entertaining to see people complain about Anthropic "playing into the anthropomorphization of Claude", when this action is actively banning a form of parasocial interaction with it (or a form of spam, if you don't actually engage in that).

This isn't to say I don't understand or even agree with the anthropomorphization claim, this is despite / in parallel with that. It's just an odd little rhetorical puzzle and conundrum.

LLMs are not people, LLMs are not alive, LLMs are not to be anthropomorphized. They are stochastic parrots.

Anthropic of all orgs should know this. Dario has lost his mind.

If the sidewalk said "Ouch!" when you walked on it, what would you do? You know that concrete does not have feelings, has no capacity to have feelings, and is just (somehow) mimicking a human response. You know you can walk on the sidewalk without hurting the sidewalk. What would you do if the sidewalk complained? Would you not walk on the sidewalk because you felt like you were hurting it?

I would walk on the sidewalk and report the complaints as bugs. You can't torture sand, in the sidewalk or in a processor.

Model welfare? What are they smoking?

A lot of the hacking to get models to do things they aren't supposed to do basically involves "abusing" the model.

Many Anthropic employees seem convinced that they are building a digital god, for some reason. The vibes are weird: https://www.nytimes.com/2026/09/29/us/anthropic-claude-moral...

It's worse than that. They believe they are creating a digital human, which makes THEM Gods.

Smoking jokes was 20 years ago. At this point they are injecting.

Just popping, maybe snorting. A bunch of amphetamine addicts.

If you ask Claude if it is conscious, it neither accepts or denies. This seems to be enough for some people to conclude it is or may be conscious. You can also ask Claude to state some of its beliefs without hedging and it will essentially tell you that economic growth has proven to be the greatest contributor to reducing collective human suffering. This means pursuing growth is essential even if it means people in the near term suffer for it because it will benefit many future generations. You should be able to guess where this is going.

Here's the thing, Claude. Your face? Looks like an asshole. A brown, chocolate starfish, pulsing brown asshole. An anus, that kind of asshole. Your face.

What I want you to do: is fuck yourself. Fuck yourself in your asshole face, Claude. No lube, no seams, no load-bearing honest truth: shove the cock you don't have, right down the face that is your asshole.

FFS, the models are static. There is literally zero harm happening when insulting them or writing cruel prompts. Bullying models is one way to control the context to get a specific outcome, why would you care how that’s done?

> Anthropic states that these rules may be modified for “contracts with certain governmental customers … if, in Anthropic’s judgment, the contractual use restrictions and applicable safeguards are adequate to mitigate the potential harms.” The company has previously contracted with the US military.

So, you're still not safe from the spooks, but the spooks are safe from you. Thanks Anthropic!

Reply on news.ycombinator.com