Hacker News Reader

Comment by philjohn | Hacker News Reader

They've also said there will be more harmful (but legal) content on there as they'll no longer automatically look for it, but require it to be reported before taking action.

As someone who worked on harmful content, specifically suicide and self injury, this is just nuts - they were raked over the coals in both the UK by an inquest into the suicide of a teenage user who rabbit holed on this harmful content, and also with the parents of teenagers who took their lives, who Zuck turned around and apologised to as his latest senate hearing.

There is research that shows exposure to suicide and self injury content increases suicidal ideation.

I'm hoping that there is some nuance that has been missed from the article, but if not, this would seem like a slam dunk for both the UK and EU regulators to take them to task on.

View on HN

This exactly mirrors my thoughts, although I don't work in your field. One quote:

"For example, in December 2024, we removed millions of pieces of content every day. While these actions account for less than 1% of content produced every day, we think one to two out of every 10 of these actions may have been mistakes (i.e., the content may not have actually violated our policies)."

That is first order data and it's interesting. However, before making policy decisions, I would want the second order data: what is the human cost of those mistakes, and what percentage of policy-violating content will not be removed as a result of these changes? Finally, what's the cost of not removing that percentage?

For that matter, by talking about the percentage of active mistakes without saying how many policy violations are currently missed, you're framing the debate in a certain direction.

Indeed.

The human cost of a piece of content being taken down depends on the piece of content, and the reason behind posting it.

In the case of someone posting about recovery from self injury and including a photo of their healed self-harm scars, having that taken down by mistake would be more harmful than someone who posted a cartoon depiction of suicide for the lolz.

Yes.

My personal belief, for whatever that's worth, is that communication and speech are one of the most powerful tools any of us have. Talking can change minds, move societies, arouse emotions, and in general makes a difference. This is true no matter the format (text, voice, etc.).

That means that restricting communication should not be a casual activity. Free speech is a good ideal for a reason.

It also means that, if you believe in the primacy of free speech, you are obligated to consider the implications of that belief. Speech has effects. In my adult life, since 1990, we have seen a major change in the ease of communication. IMHO, society hasn't been able to fully adjust to that change -- or rather, that huge suite of changes. I sincerely do not know what a healthy society using the Internet looks like; I don't think we're in one now. All of these arguments (on all sides, mine included) are hampered by our lack of perspective.

Which is why we should research this carefully - and the research thus far points to consumption or graphic or even borderline depictions of suicide, self injury and eating disorder content (eg thinspo) being bad for mental health in at least teens.

Meta seem to be making the case for those who would see social media banned for people under the age of 18. To enforce that properly would require needing ID, and that then opens a whole can of civil liberty issues.

The social "science" research in this area is junk with small effect sizes, unclear causality, and multiple uncontrolled variables. People who claim to be following the science in this area are generally being disingenuous and picking results that support their preferred ideology.

The ideology of ... not doing something that could make adolescent (and adult) mental health worse, to the point of suicide?

Yeah, making that my ideology is a hill I'm willing to die on, sorry.

Forcing the entire world to conform to your idea of "child-safe" has negative consequences, too.

Can you share the negative consequences of not allowing, and not promoting, graphic images of self harm and suicide on a social media network please?

It gives a lot of unearned power to those who decide what constitutes "promoting," "graphic," "self harm," and "social media," for one thing.

If you or I happen to agree with the people who wield that power, rest assured it's only a temporary coincidence.

Given how easy it is to take things out of context, I'm not so sure that the original context really makes a difference.

There's more people online than any of us has heartbeats, and the n^2 number of user-user pairs generates detrimental effects that track any positive effects.

Much better, I think, for each of us to have a small and private personal social network, not to hand everything over to a foreign* company trying to project its social norms worldwide.

* Facebook claims about 3 billion active users, so for 89%-93.5%** of its users, the fact that Facebook is American makes them foreign.

** https://thesocialshepherd.com/blog/facebook-statistics#:~:te....

> we think one to two out of every 10 of these actions may have been mistakes

May have been a mistake? Reminds me of RTO and the subjective feeling of being more productive in the office. They have the feeling they may have made mistakes and base their new policy on that feeling.

I think what they are saying there is the press release interpretation of experiments showing a false positive rate of 10-20%, with error bars wide enough that stating a percentage gives too many significant figures. But the definition of FP is necessarily fuzzy; if you can perfectly identify them as FP at scale then you have built a better classifier and you no longer have the FP problem. So any statement about FP rates necessarily needs to be couched in uncertainty.

I don't think it's malicious wordsmithing where they are mis-representing the internal data, though I don't have the data to confirm.

The human cost can't be quantified in any meaningfully precise way on either side. The calculations are necessarily based on so many assumptions as to become entirely subjective. Ultimately the decisions will be made based on politics and business priorities, not any objective calculation of human cost.

> However, before making policy decisions, I would want the second order data:

I think this the wrong lens. The correct lens is: if they don't voluntarily make this change, will they be forced to?

The incoming administration seems committed to banning "censorship", so I believe making a cost/benefit analysis is something of a false choice.

E.g. see https://www.youtube.com/watch?v=xJfUXVOoFBo

That ignores the regulations in the EU, and the UK (coming into force this year), and also the huge volume of lawsuits they are facing in the US. Does everyone remember Zuck turning around to apologise to the parents in that senate hearing? Those parents must feel this is a slap in the face.

This is a decision for the US market first and foremost. The lawsuits you mention are sadly irrelevant to the decision-making; again, if you are about to be forced to make this change by Trump, the results of some cost/benefit study will not sway his reasoning. His decision is already made.

FWIW I would not be surprised if the bluster about championing free speech abroad gets quietly forgotten; we’ll see. They explicitly state they will comply with laws, which in EU likely means continuing to moderate (more not less over time, given the regulatory trends).

I mean, we can tell it's not about free speech because it specifically allows you to target gay and trans people, calling them mentally ill, but not religious people.

> There is research that shows exposure to suicide and self injury content increases suicidal ideation.

Yes. However, I find this obsession with harm-based value judgment to the exclusion of all other considerations ethically problematic, to put it mildly. Ethics does not reduce solely to considerations of harm.

Would you mind expanding on that please, what are the ethically problematic things you are trying to balance against this?

Freedom of expression comes to mind. If someone had a friend commit suicide, should they not be able to discuss their experience in public?

Absolutely they should, and when I worked there that was known as "protecting voice", that content has always been explicitly allowed because it is free expression, even if reading it can be difficult for some people. The same with someone posting images of healed scars because they've been overcoming their self harm.

The content I'm talking about is graphic photos of suicide and self injury, fresh, blood soaked cuts, bodies hanging, graphic depictions of eating disorder (that goes beyond "thinspo", which is more borderline, and so downranked and not recommended rather than removed).

It's the latter that we believed (based on the advice of experts who we relied on for guidance) is harmful when consumed in large quantities.

I reject the implicit assumption that Zuckerberg has an ethical duty to reduce indirect harm from hosting content, and I also reject the implicit assumption that ethical calculations ("balancing" harm) are a meaningful way to reason about ethics. I think both of these assumptions are ethically problematic. The only reason Zuckerberg apologized to parents is because other people would have punished him had he not, which does not imply sound ethics or that he agrees with what he said.

I also think normalizing an infrastructure whose sole purpose is to suppress speech is ethically problematic. Rights are rarely expanded once taken.

You're right that people post abhorrent stuff on such sites, but I prefer approaches like filters over suppression. I also think that if reducing indirect harm is important to certain people (like reducing suicide), then there are known effective ways that don't require coercive power, like the public campaigns against smoking and MADD which have both had significant impacts.

In what way is not recommending content, or demoting it in feeds, coercive?

I'm sorry, but maximalist free speech positions fail in the face of reality.

Counterpoint: censorship inherently harms everyone. People I follow on Youtube have repeatedly had their ability to discuss topics such as suicide seriously interfered with. It actually gets in the way of factual reporting when a suicide occurs in the community and of discussing the facts of the situation so that people can learn from it and possibly prevent future deaths.

Not to mention, people just straight up have a right to talk about these things. It is not moral to hold one person responsible for an unintended and not reasonably foreseeable reaction to the discussion. And joking about these topics is legitimately therapeutic for some.

I'm not talking about that here - and that always fell under protecting voice - if mistakes were made they should have been reversed on appeal. e.g. imagery of healed scars in the context of recovery, discussions of struggles with mental health, suicidal ideation etc.

I'm talking about graphic images of self harm, suicide, eating disorders. And at some point you have to weigh the maximalist interpretation of free speech "you have to host whatever I want, as long as it's not illegal" with "promoting this stuff causes active harm, no".

>And at some point you have to weigh the maximalist interpretation of free speech "you have to host whatever I want, as long as it's not illegal" with "promoting this stuff causes active harm, no".

The burden of proof is on you to demonstrate that it causes such harm.

I don't generally think people should be held responsible for the unintended reaction to their speech of a small minority of the audience.

Having a piece of content removed, or demoted and not recommended, is being held responsible?

Also as per the inquest into the death of Molly Russel found based on the preponderance of evidence, exposure to this kind of graphic content was largely the causative agent in her suicide.

What would the bar you require be, is there a bar?

I've already seen disturbing stuff on X since Elon took over that I never would have seen when it was twitter. They don't even show the warning "this might be harmful content" on images and videos anymore. The X algo seems to go haywire every couple of days and dumps a bunch of this crap in my feed until I block 20+ bluecheck accounts showing this crap.

I believe it's only going to get worse going forward as they all adopt these policies.

It’s okay to leave X.

My profile is largely unused, I follow no one, and like 1/3 times I open up the front page I get straight holocaust denial threads suggested. Completely insane.

> So, we’re going to continue to focus these systems on tackling illegal and high-severity violations, like terrorism, child sexual exploitation, drugs, fraud and scams.

I don't think this is exhaustive, and I think SSI (suicide/self-injury) + ED/etc. stuff is considered high-severity.

Fingers crossed.

“Think of the children” isn’t really a good argument for censoring completely legal political discourse, which is what has been happening.

They are admitting that there has been a global push against free speech on these platforms.

>There is research that shows exposure to suicide and self injury content increases suicidal ideation.

I mean do you really need research to show this link? Of course it does.

We are okay with slapping an “R” rating on movies and allowing parents to be the ones who decide what content their kids can see. Why can’t we decide that parents also need to be the ones to stop their kids from consuming bad content on social media?

Automatically demoting, not recommending and adding "mark as disturbing" screens is what's going away - which is akin to the "R" rating.

But at this point, I'm siding with the "no social media for adolescents" people more and more.

Reply on news.ycombinator.com