Hacker News Reader

Comment by sd9 | Hacker News Reader

I don’t have any experience with this, so take this with a pinch of salt.

What seems novel about moderation is the frequency that you confront disturbing things. I imagine companies like Meta have such good automated moderation that what remains to be viewed by a human is practically a firehose of almost certainly disturbing shit. And as soon as you’re done with one post, the next is right there. I doubt moderators spend more than 30 seconds on the average image, which is an awful lot of stuff to see in one day.

A doctor just isn’t exposed to that sort of imagery at the same rate.

View on HN

> I imagine companies like Meta have such good automated moderation that what remains to be viewed by a human is practically a firehose of almost certainly disturbing shit.

On the contrary I would expect that it would be the edge cases that they were shown - why loop in a content moderator if you an be sure that it is prohibited ont he platform without exposing a content moderator?

In this light, it might make sense why they sue: They are there more as a political org so that facebook can say: "We employ 140 moderators in Kenya alone!" while they do indifferent work that facebook already can filter out.

Even if 1% of images are disturbing that’s multiple per hour, let anyone across months.

US workman’s comp covers PTSD acquired on the job, and these kinds of jobs are rife with it.

Content moderation also involves reading text, so you’d imagine that there’s a benefit to having people who can label data and provide ground truth in any language you’re moderating.

Even with images, you can have different policies in different places or the cultural context can be relevant somehow (eg. some country makes you ban blasphemy).

Also, I have heard of outsourcing to Kenya just to save cost. Living is cheaper there so you can hire a desk worker for less. Don’t know where the insistence you’d only hire Kenyans for political reasons comes from.

> They are there more as a political org so that facebook can say: "We employ 140 moderators in Kenya alone!" while they do indifferent work that facebook already can filter out.

Why assume they're just token diversity hires who don't do useful work..?

Have you ever built an automated content moderation system before? Let me tell you something about them if not: no matter how good your automated moderation tool, it is pretty much always trivial for someone with familiarity with its inputs and outputs to come up with an input it mis-predicts embarrassingly badly. And you know what makes the biggest difference.. is humans specifying the labels.

I don't assume diversity hires, I assume that these people work for the Kenyan part of Facebook and that Facebook employs an equivalent workforce elsewhere.

I am also not saying that content moderation should catch everything.

What I am saying is that the content moderation teams should ideally decide on the edge cases as they are hard for automated system.

In turn that also means that these people ought not to be exposed to too hardcore material - as that is easier to classify.

Lastly I say that if that is not the case - then they are probably not there to carry out a function but to fill a political role.

Also a doctor is paid $$$$$ and it mostly is a vocational job

Content moderator is a min wage job with bad working hours, no psychological support, and you spend your day looking at rape, child porn, torture and executions.

>Also a doctor is paid $$$$$

>Content moderator is a min wage job

So it's purely a monetary dispute?

>bad working hours, no psychological support, and you spend your day looking at rape, child porn, torture and executions.

Many other jobs have the same issues, though admittedly with less frequency, but where do you draw the line?

> but where do you draw the line?

How about grouping the jobs into two categories: A) Causes PTSD and B) Doesn't cause PTSD

If a job as a constantly high percentage of people ending up with PTSD, then they aren't equipped well enough to handle it, by the company who employs them.

>How about grouping the jobs into two categories: A) Causes PTSD and B) Doesn't cause PTSD

I fail to see how this addresses my previous questions of "it's purely a monetary dispute?" and "where do you draw the line?". If a job "Causes PTSD" (whatever that means), then what? Are you entitled to hazard pay? Does this work out in the end to a higher minimum wage for certain jobs? Moreover, we don't have similar classifications for other hazards, some of which are arguably worse. For instance, dying is probably worse than getting PTSD, but the most dangerous jobs have pay that's well below the national median wage[1][2]. Should workers in those jobs be able to sue for redress as well?

[1] https://www.ishn.com/articles/112748-top-25-most-dangerous-j...

[2] https://www.bls.gov/oes/current/oes_nat.htm

What could a company provide a police officer with to prevent PTSD from witnessing a brutal child abuse case? A number of sources i found estimate the top of the range to be ~30% of police officers may be suffering from it

[1] https://www.policepac.org/uploads/1/2/3/0/123060500/the_effe...

You can’t prevent it but you can help deal with it later.

> So it's purely a monetary dispute?

I wouldn't say purely, but substantially yes. PTSD has costs. The article says some out; therapy, medication, mental, physical, and social health issues. Some of these money can directly cover, whereas others can only be kinda sorta justified with high enough pay.

I think a sustainable moderation industry would try hard to attract the kinds of people who are able to perform this job without too much negative impacts, and quickly relieve those who try but are not well suited, and pay for some therapy.

Also doctors are very frequently able to do something about it. Being powerless is a huge factor in mental illness.

“I would imagine that companies like Meta have such good automated moderation that what remains to be viewed by a human is practically a firehose of almost certainly disturbing shit.”

This doesn’t make sense to me. Their automated content moderation is so good that it’s unable to detect “almost certainly disturbing shit”? What kind of amazing automation only works with subtleties but not certainties?

I assumed that, at the margin, Meta would prioritise reducing false negatives. In other words, they would prefer that as many legitimate posts are published as possible.

So the things that are flagged for human review would be on the boundary, but trend more towards disturbing than legitimate, on the grounds that the human in the loop is there to try and publish as many posts as possible, which means sifting through a lot of disturbing stuff that the AI is not sure about.

There’s also the question of training the models - the classifiers may need labelled disturbing data. But possibly not these days.

However, yes, I expect the absolute most disturbing shit to never be seen by a human.

—

Again, literally no experience, just a guy on the internet pressing buttons on a keyboard.

>In other words, they would prefer that as many legitimate posts are published as possible.

They'd prefer that as many posts are published, but they probably also don't mind some posts being removed if it meant saving a buck. When canada and australia implemented a "link tax", they were happy to ban all news content to avoid paying it.

Yes, Meta are economically incentivised to reduce the number of human reviews (assuming the cost of improving the model is worthwhile).

This probably means fewer human reviewers reviewing a firehose, not the same number of human reviewers reviewing content at a slower rate.

I’d think the higher density/frequency of disturbing content would cause people to be desensitized.

I never seen blood or gore in my life and find seeing it shocking.

But I’d imagine gore is a weekly situation for surgeons.

I agree. But that might be comorbid with PTSD. It’s probably not good for you to be _that_ desensitised to this sort of thing.

I also feel like there’s something intangible regarding intent that makes moderation different from being a doctor. It’s hard for me to put into words, but doctors see gore because they can hopefully do something to help the individual involved. Moderators see gore but are powerless to help the individual, they can only prevent others from seeing the gore.

It's also the type of gore that matters. Some of the worst stuff I've seen wasn't the worst because of the visuals, but because of the audio. Hearing people begging for their life while being executed surely would feel different to even a surgeon who might be used to digging around in people's bodies.

There are many common situations where professionals are helpless, like people that needs to clean up dead bodies after an accident.

Imagine if this becomes a specialized, remote job where one tele-operates the brain and blood scrubbing robot all workday long, accident, after accident after accident. I am sure they'd get PTSD too, airey, sometime it's just oil and coolant, but there's still a lot of body-tissue involved.

I'd really like to see more data on this. I really think (most) people would be desensitized and not become hyper sensitive to this content.

Another fascet to this is the moderators willfully agreed to review this content and had full autonomy to leave the job at any point.

I'm sure they consented to taking the job but is it informed consent? Can you really know what you will be exposed to? And unless unemployment insurance is given to quitters, autonomy to leave a job at any point is overestimated... Sure you can decide if you want to leave but how long it takes may not be in everyone's control if they first need to find another job.

Desensitization is only one stage of it. It's not permanent & requires dissociation from reality/humanity on some level. But that stuff is likely to come back and haunt one in some way. If not, it's likely a symptom of something deeper going on.

My guess is that's why it's after bulldozing hundreds of Palestinians, instead of 1 or 10s of them, that Israeli soldiers report PTSD.

If you haven't watched enough videos of the ongoing genocides in the world to realize this, it'll be a challenge to have a realistic take on this article.

I watch surgery videos sometimes, out of fascination. It's not gore to me - sure it's flesh and blood but there is a person whose life is going to be probably significantly better afterwards. They are also not in pain.

I exposed myself to actual gore vids in the aughts and teens... That stuff still sticks with me in a bad way.

Context matters a lot.

> They are also not in pain.

My understanding is that during surgery, your body is most definitely in pain. Your body still reacts as it would to any damage, but anesthetics block the pain signals from reaching the brain.

But there is a difference between someone making an effort healing someone else vs content with implications that something really disturbing happened that makes you lose faith in humanity.

Should the Uvalde police sue the school for putting them through that trauma?

> I imagine companies like Meta have such good automated moderation

I imagine that they have a system that is somewhere between shitty and none functional. This is the company that will more often than not flag marketplace posts as "Selling animal", either completely at random or because the pretty obvious "from an animal free home" phrase is used.

If they can't get this basic text parsing correct, how can you expect them to correctly flag images with any real sense of accuracy?

Reply on news.ycombinator.com