How Shadow Banning Can Silently Shift Opinion Online
insights.som.yale.edu
insights.som.yale.edu
All the talk about privacy and data collection were mostly to muddy the waters by bringing in all tech to the conversion, and divert attention from solely tiktok ("Everyone does it so why focus on tiktok?!"). It's still a threat, I'm not trying to minimize it, but tiktok was banned primarily because of who controls the content delivered, not who is collecting the data.
To make things worse, for 99% of people things might look normal but for 1% it may show a large skew. The net can have memorized this.
Obviously, this becomes an arms race. The usual strategy is to add random checks, friction to contribution and severe-enough punishments for offenders (lifetime RL bans, etc.)
Expert opinions welcome - I'm a tourist.
In whatever form it takes such that a regulator can scrutinise it. If that seems arduous or impacts the social media business model, too bad. Nothing of value will have been lost. I am not typically a person who favours over regulation, even my views on AI are likely too liberal for HN, but on the matter of social media impacting children, I'm all in for regulation.
So they also have to give up all their user data?
Meaning if I record my own inputs, I should be able to recreate the suggestions I see using the published software. This doesn't mean the inputs of other users are exposed to me, but it does imply that "the algorithm" can only use data that is either publicly exposed or is my own inputs.
That seems pretty restricting given that even the most basic recommendation would try to cluster users based on interests and use that to recommend stuff (eg. Amazon's "customers who viewed this item also bought..." from decades ago).
All that would be very hard/impossible to recreate in order to test the algorithm.
Enough researchers, at enough institutions with sufficiently diverse motives, that if multiple of them identify a problem and agree about it, Joe can make a reasonably confident assessment that there's probably something there.
They will make less ad money and engagement will go down because we won't be training children to become 5 second scrollers, but this doesn't seem like an issue to me.
How would new content discovery happen in your proposed world?
Also a part of HN's "algorithm" is literally the mods manually adjusting things, and you can't publish that.
> How would new content discovery happen in your proposed world?
Seconding other posts—entirely fine, like it did before that kind of garbage.
Content discovery was word of mouth or a separate page dedicated to popular stuff.
There is no sarcasm; does anyone genuinely only browse sites like hackernews by viewing the 'new' page? That is what the comment is advocating, that the only view should be the new page, with submissions listed by post date only.
If discovery is by a separate page dedicated to popular stuff, how is that any better? Whatever 'algorithm' is used to create that page would have the same problems that people are complaining about; why would putting it on a different page change anything? That is sort of how things are currently; you can view a feed by 'latest first' or by the 'algorithm'. You can choose either one, but most people prefer the algorithm version because it curates.
People WANT curation. You can argue with the particular type of curation, but I think the concept itself is pretty universally desired.
Look at Instagram, Twitter, or Facebook, I believe there is a desire for a less algorithmic approach on those sites, hence all the uproar about the algorithms and shadow banning. Reverse chronological ordering from people you follow was the standard before, and a feature before revenue mattered as much. You curated your own feed by following people, not by a black box
> Social media are interactive technologies that facilitate the creation, sharing and aggregation of content, ideas, interests, and other forms of expression through virtual communities and networks.
Hacknews is one virtual community. The whole point of visiting the site is to aggregate content and read and make comments about that content.
Reddit and slashdot are also both social media sites.
An ad hominem attack doesn't seem necessary to prove your point (speaking as someone who actually is old enough to remember).
There are definitely people in the tech field who have only (or mostly) experienced algorithmic based feeds.
If there's a need to provably ensure kids are not being influenced by algorithm, I think my solution works. Making social media less engaging for children is something I'm fine with. Adults can have whatever experience they want.
If you publish your algorithm, you are giving interested parties the instruction manual on how to game it; spammers and propagandists can use that algorithm to make sure their content gets to the top.
Letting people see into it could make it easier for people to learn how to game the system but it could also make the system harder to game in the first place.
The engagement ranking done to keep you hooked is probably not that interesting.
X/Twitter does now publish their algorithm, and it hasn't made it any less of a shithole.
[1] https://www.axios.com/local/salt-lake-city/2024/05/06/senato...
I think those are two sides of the same coin. If you read the whole quote in context[1], it seems equally likely that he was trying to imply that tiktok (the platform, not just the userbase) has a pro-palestinian bias, possibly due to the Chinese government. If it's the users who pro-palestinian, it would make little sense to ban the platform, because those users would still be pro-palestinian and migrate elsewhere. Reddit has a similar pro-palestinian bias, but it hasn't attracted similar scrutiny, presumably because no one suspects it of being controlled by the chinese government.
[1] https://www.state.gov/secretary-antony-j-blinken-at-mccain-i... (search for "SENATOR ROMNEY: A small parenthetical point, which is some wonder why")
Instagram apologises for adding ‘terrorist’ to some Palestinian users profiles bios [1]
Facebook violated rights of Palestinian users, report finds [2]
Activists: Google, Apple ‘Erasing Palestine’ from Geographic Apps [3]
[1] https://www.standard.co.uk/news/world/instagram-meta-palesti...
[2] https://apnews.com/article/technology-middle-east-israel-858...
[2] https://themedialine.org/by-region/activists-google-apple-er...
If that was the case, chinese state media ( where the chinese government actually has direct control over the content ) would be banned. But they are not banned.
Does it? And what does that even mean? Are there any other "content-neutral" and "fair" algorithms that you're aware of at other social media companies?
> we have no reason to believe it is unbiased.
Not only is that not how argument works, it's a bizarre bar that only tiktok has to leap to do business. We have every reason to believe that any and all commercial algorithms for selecting and prioritizing content to display to users are not "content-neutral," nor "fair" for every possible definition of fair.
Was this actually proven in specific incidents?
Is this the only media outlet this applies to?
No - and certainly not on any appreciable or meaningful scale.
The bill's proponents claim that intelligence agencies have found proof of this, and that the proof is classified and therefore cannot be shown to the public. (If you're over the age of 30, you're almost certainly getting early 2000s flashbacks reading that sentence).
> Is this the only media outlet this applies to?
No. The ban applies to TikTok, but as pointed above, not to Chinese (or Russian) state media, which do directly control the content shown to users.
(I'm sure it is; there have been plenty of people disappeared https://www.theguardian.com/world/2021/nov/19/china-disappea... , but I'd just like to have a little evidence rather than do an Iraq war again)
In the US, the government has control over the content too, it just so happens that it mostly concerns itself with combatting certain moral vices (from prostitution to bootleg fireworks) - although to be fair, it also lightly pressured big tech into combatting political "misinformation".
In China, the same mechanisms exist, except the controls on speech are much tighter and the punishments more severe. Every other tech company operating there has a story of officials showing up with demands or threatening employees. I worked at two places where this happened. In addition, most businesses operate as formal or quasi-formal public-private partnership, where the CCP has officials within the corporate management structure. Again, ask anyone who tried to start a foreign-owned business in China.
You can make a reasonable argument that having an oppressive and geopolitically adversarial foreign government effectively in control of the most popular youth social media platform in the country is not great. Or you can make a reasonable free-trade argument. Free trade works only if all participants follow the rules; if Western social media can't operate on their market, they don't get to operate on ours.
There are things to dislike about the US government, but I really don't get the contrarian takes on TikTok. China is objectively worse and its government's legitimacy and lasting power is built in the bodies of tens of millions of victims. We're not getting along right now. This doesn't require cynicism to explain.
How is the Chinese government's control over TikTok content any more "direct" than the US government's control over Facebook content?
Let that sink in. (not in a musk way)
As for the US, we have documented evidence of US corporations cooperating with the US government for spying and propaganda efforts.
The US has 4x as many prisoners per capita than China, and Assange is still in captivity.
Perhaps that's the real reason. But if so, why can't the US government just require Tiktok to push content that aligns with US interests? Say for every 3 videos a user watches, they are then required to watch a US army recruitment ad, or an VOA anti-China propaganda clip, or Uncle Sam telling them that there's a high chance they just watched foreign propaganda (think tobacco package warning messages). I'm pretty sure that Tiktok would find this acceptable as the cost of doing business, and that China wouldn't care — plenty of anti-China content already exists on Tiktok.
Of course I'm not seriously suggesting this. But I'm saying that if control over content is the real motivating concern, why wouldn't a solution like this be considered viable?
The suggestion doesn't restrict how much anti-US propaganda gets pushed.
I don't think TikTok or the US would find it acceptable. It is too heavy handed
It's less heavy handed than banning outright, though.
> The suggestion doesn't restrict how much anti-US propaganda gets pushed.
Good point. Let me refine my suggestion so as to require that TikTok prohibit all anti-US content: sophisticated AI algorithms would be put in place to ensure that content that doesn't align with US interests is not allowed. This will make the suggestion even more heavy-handed, but note that, once again, it's still less heavy-handed than banning outright.
Instead, it relies on US citizens embedded in US companies to push US ideals in social media. When it does presumably conduct social engineering attacks, it does it through bots and buying adds on platforms.
Last, I don't think the US government expecting to have to actually ban Tiktok. I think it is expecting to win, and get to have it's cake and eat it too. This would be the least heavy handed option.
Counterpoint: The law to ban tiktok was languishing after the US government got complete access to tiktok moderation, only still pushed by local anti-China grandstanders who ran on it when the government was busy extorting tiktok in order to get that access. The reason it passed now was to suppress the speech of Americans talking about the leveling of Gaza.
> All the talk about privacy and data collection were mostly to muddy the waters by bringing in all tech to the conversion, and divert attention from solely tiktok
Because an anti-tiktok law is a bill of attainder and completely unconstitutional (as I was once downvoted into oblivion for mentioning.) Generalizing the bill passes that constitutional hurdle, and if upheld allows the state to end-run around the 1st Amendment and ban speech at will. The bill would be useless if it didn't silence Americans, and all people will do after a tiktok ban is migrate to another speech platform. It's important to be able to silence speech everywhere.
> tiktok was banned primarily because of who controls the content delivered, not who is collecting the data.
The US doesn't care about tiktok, and has full access to ban whatever it likes. The US cares about silencing you.
Last I checked, Elon Musk hadn't sent hundreds of Uighur women to rape camps to be "reeducated". He hasn't harvested organs from undesirables while they were still alive (for freshness) all that much either. I don't praise the United States government as saints and superheroes or anything like that, but to be unwilling to admit that there are still stark differences between the two countries probably means you wore a Che t-shirt in college and might even be wearing one now if your company permits casual dress.
I think that the parent point cedes this point, and argues that Musk's benefits, incentives, and morality are more favorable to that of the Chinese government.
An American civil war is high, if not at the top, of China's wish list. Yeah, they would feel the crunch, but would be able to cement unrivaled global power if the US unraveled.
So what? Why are Elon Musk and Sundar Piachi allowed direct control of what they show me but Shou Zi Chew isn’t? Is it due to governmental involvement? Imagine if the BBC broadcast direct to Americans!
Or are we, somehow, collectively, of the opinion China is an enemy of some variety? If so, what variety? And what weird peacetime rivalry are we having?
I don't think it's good that cabal of media and tech executives has so much concentrated control over public discourse. But there is a huge qualitative difference between them and a hostile foreign power.
If TikTok can change public opinion then they must have speech right? How do they have speech without speaking? They have speech by filtering and weighing things a specific way. If you can speak without speaking then how does Section 230 come into play? If platforms can speak and have opinions and affect public opinion then whey do they get Section 230 protections? Should the algorithms be open sourced and transparent (or banned) since they have so much power?
It raises a lot of questions that we have put off answering. I think the (distant) relevance of Section 230 makes big tech scared to talk about it since that's their cash-cow.
Section 230 exists specifically to allow platforms to "have opinions and affect public opinion." Liability resides with the accounts making the content, not the platform. It would be impossible to have any degree of free expression on the internet if platforms were legally liable for everything posted there, or if they could be sued over every moderation decision they make.
>It raises a lot of questions that we have put off answering.
We haven't put off answering anything. We have the answers. Platforms are allowed to moderate content. They're allowed to filter and present content in whatever way they want. They're allowed to have a bias. There is no gray area here, no matter how badly people want there to be one.
I know people think that government regulating social media means some kind of laissez-faire free for all where all legal speech will be allowed without consequence (which would be even less moderation than you find on 4chan) but that is very much not how it would go.
Of course, a person in a bubble like that might actually become more radicalized. And if a person discovers he's been had by looking at, say, his friend's Reddit version, he might become angry enough to do something, angrier even than if he had just been told "no", because now he's been made a fool.
also, why would the AIs give only praise? they could also be mean and evil; would that be "hellbanning"?
Oh wait, that’s what the internet already is.
is that proven? how did they find about that?
is it reddit doing it, or is it other users?
EDIT: ok so that's a fake screenshot, the users says it is... sorry but this is misleading.
he says hypothetical, and then makes a fake screenshot with the NYT on it, that's really disingenuine.
twitter gonna twitter
>although the above article is fake (hence my phrasing as a 'hypothetical' and the article being dated in the future in 2024!), it's feasible to build this as a weekend project at this point, so it won't remain fake for very long now!
[edit] tried a totally different internet connection with different local network. Same result.
Yeah, it could definitely happen.
If we view social media as simply a private service, then these companies have the right to do whatever they want. Businesses have the right to refuse service for any reason.
But if we view social media as a utility similar to telephones, then free expression should be protected. People can say whatever they want over phone and text.
I lean towards the latter. Social media has become the new public square/agora/forum. Yes, its still owned by private enterprise, but it's basically the new town hall at this point.
Utilities are regulated monopolies. In exchange for a monopoly status, they need to abide by some pretty strict standards (like regulated pricing and service maintenance), and are forced to work with entities that they would normally avoid.
Things like open APIs, algorithm access (that could be classified, but should be accessible to folks that are accountable to the consumers, etc.).
Not a simple structure. What we have now, is pretty haphazard, and has grown organically. The current utility/public infrastructure setup is the result of hundreds of years of disasters, ripoffs, successes, and learning.
Not sure if we have to go through it all again, or if there is a possibility of taking learning from current municipal structure, and apply it here.
Free expression isn’t something that can be realistically protected when a social media platform has an explicit purpose, code of conduct, and/or ideals.
For example, an LGBTQ social media network has to be able to ban anti-LGBTQ content and users. Hacker News needs to be able to remove non-tech industry-related off-topic posts. /r/pcgaming needs to be able to remove posts about mobile gaming. Spam and unwanted self-promotion is often removed by moderators of social media outlets. Moderation is key to a workable social media experience.
There’s also a lot of 100% legal free expression that platforms have to be able to choose not to host. Pornography is an common example.
The thing that makes this topic difficult is that social media isn’t really pure person-to-person speech (e.g. like a person on soapbox) and it also isn’t really pure curated media (e.g. a television channel or news website published by an editor). It’s like a hybrid of the concepts.
I've moderated some large subreddits in the past, and am a strong free speech supporter, but I'm also actually perfectly happy not dealing with people spamming the n-word in every forum they can find because they're legally entitled to. "Congress shall make no law" really does mean congress. Unlike telephones, if the forums are open to the public, then so some decorum is needed.
Have people used telephones recently? The network is so infiltrated by scam artists (and the tooling to remove them from the network is so toothless) that it's functionally one giant attack vector against the most vulnerable telephone users these days. The fact that Google is trumpeting adding AI to a phone to detect when you are getting scammed is indicative of how badly the network has been poisoned by the lack of anyone's ability to regulate access.
We already know what an unfettered free-expression network looks like. It looks like telephone and email. It's a scammer's playground.
The EFF recommendation is anti-trust enforcement to insure these services don't hold American communications hostage. We must attack their moats to enable smaller competitors to have at more level playing field.
And leave the rest alone. Gore/graphic content -> filter with a toggle for those that want to view it. Bullying -> does it cross the line into an actual law being broken, if so, then ban it. Makes sensitive people nervous, uncomfortable and angry -> leave it be, but if you really wanna go far stick a sentiment analysis thingie on it and allow people to selectively filter it below their own threshold. There is so much that can be done simply, without breaching anyone's freedoms and without teetering into Orwellian 1984 conspiracy territory about wide-scale mind control and opinion shifting.
The problem comes when people naturally express themselves, it makes <insert group here> uncomfortable and it inconveniently bubbles up the rankings. E.g. I bet you right now that 20% of the entire american internet population simply doesn't like Biden and would mock him for his, probably innocent, old-age antics. But if that opinion were to "manifest" into some sort of catchy "Biden is old and sucks" hashtag or meme that needs reporting, well that's uncomfortable for TPTB, or the elites, or the media or whatever you wanna call it, and must be stopped. The internet is the ultimate direct-democracy tool we have right now, and it's being manipulated precisely because it breaks "representative" democracy.
Another uncomfortable topic: trans people in sports. It will absolutely blow up in one direction if people were organically allowed to organize and find others with similar viewpoints. Instead, they're banned, scared into shutting up, shadow-banned, or worse cancelled and doxxed.
Why can’t I start my own social media website about cars where any topics that aren’t related to car culture and discussion are not allowed?
Your line is illegality, but my line as the owner of the social media network might be something else. Is private ownership banned now?
We've crossed the boundary of pure private ownership of anything many years ago. Heck, I don't even have private ownership of my own body, so a say in a highly influential component of our society is a small price to pay to keep the evil at bay.
There is a huge amount of discussion of the problem of males in female sports on Twitter, plus also on smaller forums like Mumsnet and Ovarit. Some of it amongst leading athletes and sports science researchers. People of similar viewpoints are organising, despite the attempts to shut them down on other forums.
It's working too: various sports bodies have made changes within the past couple of years that walk back their previous broken policies of male inclusion. It does seem that the needle is slowly but surely moving back towards fairness for female athletes. Though there is still a lot of work to be done.
IF its the "new town hall" then were gonna have a fuck ton of rules. You're not "Free" to scream fire in a crowded building, you cant make up crisis actors and expect NOT to get sued to death....
The protection of freedom of expression exists. No one is stoping you from hosting your own platform based on what ever nonsense you make up. I dont have to give you a box to stand on, no one does. Thats not how it ever worked.
They actually are. CloudFlare will block you, Visa will refuse payments, Google will delist you, Apple app store will delist your app, and you'll be subject to endless legal harassment from activists if your site allows people to communicate substantially out of line with the San Francisco progressive bloc.
> A system of government marked by centralization of authority under a dictator, a capitalist economy subject to stringent governmental controls, violent suppression of the opposition, and typically a policy of belligerent nationalism and racism.
https://www.wordnik.com/words/fascism
When the government is colluding with private industry to suppress opposing views on social media, you really have to wonder if the word "violent" is even needed in this definition anymore.
I just find this to be such a large detail that it's hard to gloss over. What are the assumptions of this model in the simulation? How accurate is the model? How did you evaluate that accuracy? How sensitive are the results to the model's parameters?
> the network might choose to show the content of a moderate user to a relatively right-leaning connection ... but block that same content from the timeline of a left-leaning connection
That's not shadowbanning because the tweet is still being shown to some people. That's normal algorithmic sorting with down ranking. A true shadowban would result in a tweet not being shown to anybody. What they describe is more like de-listing from the global timeline but still being shown to followers. Now you're just talking about search bubbles.
What's strange is they describe this as "neutral".
> At first blush, the banning appears to impact every user more or less equally.
How is that equal? What the viewer sees depends on their cohort. It's the same kind of manipulation that we've known about for years. The unique issue with a shadowban is how it affects the creator. Specifically a shadowban is intended to counteract ban evasion using multiple accounts. If you're going to make a conclusion about whether or not shadowbanning should be allowed it should be done by comparing silent versus explicit bans.
Personally I don’t think the platforms being private matters - at their scale the big social media platforms are public squares, and should be regulated like that. Or maybe treated like a phone carrier or utility, where even if they’re private they’re subject to regulation that forces them to provide service to all customers.
Malls and shopping centers in California are treated as public squares for 1st amendment purposes. It's unclear why Facebook et al aren't given the same treatment, at least in California. (Note this is California, not Federal, law.)
https://firstamendment.mtsu.edu/article/pruneyard-shopping-c...
The amount of accurate information you find online is probably worse than what you find talking to people IRL, but we seem to insist on treating it as a truth source instead of a game people are playing with each other.
Now you can consume nearly unlimited opinions in a day so whatever it is we think about the accuracy of information online, it's shaping impressionable minds faster than biology and society can cope.
It's bad enough that 'news' sources are now totally politically biased. Social media platforms, search engines, entertainment providers, etc. are in on it too.
Kudos to anyone who works to keep a level perspective. It's not easy.
At one time I moderated two large and growing forums - wordpress.org/.com - and there was an ability to shadowban.
You post to a forum, as a mod I see your post and I ban you (for whatever reason) You understandably get angry You make a new account, post again and react angrily to the ban in your post I have no option but to ban you again We end up in a circle This is not good for you or me.
If though I shadowban your anger does not rise, you do not make lots of new accounts and I do not create more work for us both.
For me the key here is that shadowbanning is okay for an established user who one day went off the rails. Let them rant, check their later posts, and remove that shadowban when the calm has returned. But you must check. For new users, just ban, block and use tools to autoblock going forward.
I was a mod on a growing subreddit for almost 10 years. Sometimes a shadow-banned user would post. As a mod we could see the post. We would let the post be published but would also reply along the lines of "You are shadowbanned by the reddit admins. This is how you check [give info] and this is how you appeal [give info]."
Blanket shadowbanning is a bad thing. Such bans must be reviewed.
I'm the only one I know smart enough to appreciate my own wit. If I had realized you other people were reading my comments all along, it might've made me shy.
This space has a published agenda and those who don't toe that line can go elsewhere; it's a very big Internet out there.