Ask HN: Is AI for online segregation ethical? | Hacker News Reader

Ask HN: Is AI for online segregation ethical?

17 pointsiosystem22 comments

A friend informed me that AI is being used to separate certain individuals in online spaces. He expresses this use of the technology is quite new and is an evolution of shadowbanning certain persons that otherwise add toxicity to discussions. The persons shadowbanned in this instance are tricked into assuming they're not banned by fake interactions generated by the AI towards real content while nonshadowbanned users interact with each other to the content. My friend said the technology is just the beginning and it eventually will be used for certain demographics throughout online spaces for certain governments. I'm curious how ethical is this?

View on HN

It's unethical, without a doubt.

As described, it's not even about segregating different audiences, but about sequestering and silencing specific people, yet keeping them engaged at the same time, perhaps to be able to continue to show them ads and keep up the platform's KPIs.

In a way, that'd be quite similar to dating platforms creating fake profiles to keep their members engaged, which I doubt anyone would consider ethical behaviour.

If an online service doesn't want certain types of opinions, ideas, or people on their platform, that's absolutely fine. Just tell them, so they can take their business elsewhere.

The point of shadowbanning isn't generally to keep engagement for KPIs, but to slow down the rate of just moving on to the next account.

Censorship is already unethical. One would have establish a need for censorship first. Can a book be written, that causes people to become a danger and thus needs to be forbidden? Is it even possible that a text on a computer screen can cause harm to someone?

I'm not aware of this and any argument for censorship is usually quite a long stretch of hypertheticals.

For violating TOS people should know what rule they have violated. For example doxing, spamming or just being insufferably rude. Punishment without explanation or reason is obviously unethical.

I think shadow banning people and then actively keeping them in loop can hardly argued to be ethical. Shadow banning itself is barely ethical.

And if the ai would communicate with these toxic shadowbanned people in such a way they become aware of their rudeness or mistakes on the forum? If AI creates a feedback loop then it's all right?

Interesting thought. I think this falls in line with explaining mistakes to people and is quickly off topic to op.

If AI could help people to bring their point or grievance across in a clear manner, this would be the opposite of a shadow ban. This can be done transparent, in Form of a suggestion.

E.g to write penis as p.e.n.i.s. to make clear that proton enhanced neutron induction spectroscopy is abbreviated here, might be helpful to some.

AI can create a feedback loop ethically, if it makes itself know as an AI or automated system or Eliza 2.0. The user can choose to just opt out or mash on his keyboard as he pleases, as we do since 1964.

That is all ethical, but deceiving users and trapping them definitely isn't.

Sounds similar to the "heaven banning" concept. That was fictional as far as we know, but it's obviously technologically possible. https://twitter.com/nearcyan/status/1532076277947330561

Did your friend ever produce any evidence for this?

No, friend wasn't able to show me evidence because of a reason equivalent to an NDA. I have no reason to distrust my friend. I'm confident what he's informing me is possible as well.

FWIW, informing you that the system exists is probably already a breach of NDA. If they failed to produce any evidence or conclusive examples of this happening, it's likely they were running one over on you. "Dead internet theory" has been a 4chan-staple since the mid-2000s, and it's more-or-less the same idea - "when people don't engage with my problematic content, it must be the fault of AI!" Needless to say, this is pretty easy to prove wrong and mostly just reflects conspiratorial loneliness. It might be true that your friend is trustworthy, but their theory holds little water and sounds more like a gross misunderstanding than systemic abuse.

But hey, maybe he's right. I don't know (or really care) what our next steps are as a society if it's true, but assuming he's telling the truth seems a bit like jumping-the-gun.

We grew up together and he's a programmer similar to me. I have been following AI for other aspects that make me believe what he explained is realistic. Also I did ask him for his thoughts on the ethics of it and he thinks the outcome is fine. I'm curious what your thoughts are concerning ethics if it's indeed happening.

It's perfectly fine, from a legal standpoint. My personal opinion regarding the ethics of the situation is mainly a reflection of that - in the bizarro world where social media companies need to synthesize content to drive wholesome engagement, it's entirely their right to do so. When you sign up for any of these services, you acknowledge that your uploaded content and online experience is completely subject to their will.

The ethics of all this have already blown over. Twitter and Facebook have both been proven to shadowban controversial figures, and no ethical panic ensued. People don't care, they want a Tik-Tok style feed of sugar water and sweet dreams fed to them on an IV drip. If the next step is fabricating fake content (again, considering how many real content creators exist this seems unlikely), then there is no ethical discussion. It's as fake as the AI-corrected profile-picture you shot on iPhone. It's as real as your approximate location being tracked by Walmart WiFi and then processed into shopping habit data to sell to competitors. If you use Facebook or Twitter (or arguably any social media) then you're already being manipulated. They don't need to promote fake content to make you think what they want.

1) We know that corporate America is experimenting with AI to perform business functions more efficiently.

2) We know that shadowbanning exists and is used to contain spammers and other users that a platform doesn't want on their site.

Given that the previous two facts are true, it seems like a perfectly logical outcome that somebody would decide to experiment with some kind of AI powered chatbots to keep shadowbanned users better contained.

That, or Occam's Razor could suggest that the shadowbanned second-class is a much less interesting/engaging group of individuals than the rest. That aligns with the theory that this is all for money (it is) and it's much easier to imagine than a cross-platform initiative to use GPT-3 for pestering extremists.

Who knows though, I'm willing to entertain anything here in the grand ol' USA. Still, logic suggests that there's a simpler answer here.

It's the opposite of pestering extremists, it's keeping extremists entertained while keeping them from pestering others. Seems like an excellent way to deal with people who avoid bans, the problem of course lies in who gets to decide what gets you thrown into the ai segregation boxes...

> Seems like an excellent way to deal with people who avoid bans

I'm sure this would be reasonably effective on the specific goal of occupying these users' time for days/weeks/months/or even years until they realized that they were talking with robots, but it seems like it might have significant negative social consequences for some groups.

For instance, imagine somebody dealing with depression who genuinely needs a human to talk to. Like many in modern society, maybe social media is the only human interaction they have until they get shadowbanned based on tenuous reasons. God, what happens then when depressed and/or unstable people only have robots to talk to for years? What happens when they realize that they've been tricked and feel intense anger/sadness/hopelessness?

This is playing with peoples' lives with fire and shouldn't be an acceptable practice.

What's the incentive to build tech like this? Or, why is it valuable for someone who is shadow banned from knowing they're shadow banned because they don't get reactions?

This doesn't make sense to me, because it seems like a huge amount of investment to...what? Get them to see more ads instead of getting sick of people not reacting to their trolling and leaving entirely? If the return on investment is high enough for this to be practical, it suggests the community has a toxicity problem that itself needs to be addressed. What makes so many people so angry and toxic?

This seems trivial to bypass. The goal for deceiving banned users is to discourage them from creating new accounts. But I'm sure eventually, the system will be exposed, like if a shadowbanned user is friends with non-banned user and they notice discrepancies. After that, people will just regularly create new accounts to see if the new account can interact with the old one, to see if the old one has been banned

It's not ethical at all. Neither is annoyance functionality to churn unmonetized users nor the habit of some (Apple/Windows) to constantly change features and UI functionality to force adaption which creates more buy-in which has the effect of tricking people into continued use of a product that otherwise is lackluster.

Pavlov has nothing on the business of training people through computing.

In my opinion if it’s biological segregation it’s unethical but if it’s ideological segregation, absolutely go for it.

It is my dream that one day, and one day soon, social media companies uncorrupted by having to appease the uneducated masses will, secretly, present entirely fabricated realities to some, placing them in an invisible prison, while magnifying the message of others.

No misguided popular movement (or even factual information, lacking critical context of course) will be able to gain momentum, as its promoters will be unknowingly yelling into the void. On the other hand, movements and ideas that promote wellbeing and harmony will spread easily. It does not take advanced AI to categorize vocal people by politics, and quarantine those with hateful ideas.

This is our chance to safely defuse humanity's hateful instincts, and we should take it.

Best of all, it is completely ethical because, as a since-deleted post pointed out, it is vaguely hinted at in the Terms of Service. Voluntary agreements are the highest form of ethics. They are beyond moral question, and in fact the whole purpose of society is to enable and enforce them.

Those who don't like it are free to stop participating in online society. With such an easy and safe way out, there is absolutely nothing to worry about.

https://en.m.wikipedia.org/wiki/Poe%27s_law

That's not how businesses work.

That's not how humans work.

This is a complete unhinged fantasy.

Reply on news.ycombinator.com