What makes moderation different? and how should it be handled so that it reduces harm and risks? surely banning social media or not moderating content aren't options. AI helps to some extent but doesn't solve the issue entirely.
What seems novel about moderation is the frequency that you confront disturbing things. I imagine companies like Meta have such good automated moderation that what remains to be viewed by a human is practically a firehose of almost certainly disturbing shit. And as soon as you’re done with one post, the next is right there. I doubt moderators spend more than 30 seconds on the average image, which is an awful lot of stuff to see in one day.
A doctor just isn’t exposed to that sort of imagery at the same rate.
This doesn’t make sense to me. Their automated content moderation is so good that it’s unable to detect “almost certainly disturbing shit”? What kind of amazing automation only works with subtleties but not certainties?
So the things that are flagged for human review would be on the boundary, but trend more towards disturbing than legitimate, on the grounds that the human in the loop is there to try and publish as many posts as possible, which means sifting through a lot of disturbing stuff that the AI is not sure about.
There’s also the question of training the models - the classifiers may need labelled disturbing data. But possibly not these days.
However, yes, I expect the absolute most disturbing shit to never be seen by a human.
—
Again, literally no experience, just a guy on the internet pressing buttons on a keyboard.
They'd prefer that as many posts are published, but they probably also don't mind some posts being removed if it meant saving a buck. When canada and australia implemented a "link tax", they were happy to ban all news content to avoid paying it.
This probably means fewer human reviewers reviewing a firehose, not the same number of human reviewers reviewing content at a slower rate.
I never seen blood or gore in my life and find seeing it shocking.
But I’d imagine gore is a weekly situation for surgeons.
I also feel like there’s something intangible regarding intent that makes moderation different from being a doctor. It’s hard for me to put into words, but doctors see gore because they can hopefully do something to help the individual involved. Moderators see gore but are powerless to help the individual, they can only prevent others from seeing the gore.
Another fascet to this is the moderators willfully agreed to review this content and had full autonomy to leave the job at any point.
My guess is that's why it's after bulldozing hundreds of Palestinians, instead of 1 or 10s of them, that Israeli soldiers report PTSD.
If you haven't watched enough videos of the ongoing genocides in the world to realize this, it'll be a challenge to have a realistic take on this article.
I exposed myself to actual gore vids in the aughts and teens... That stuff still sticks with me in a bad way.
Context matters a lot.
My understanding is that during surgery, your body is most definitely in pain. Your body still reacts as it would to any damage, but anesthetics block the pain signals from reaching the brain.
Content moderator is a min wage job with bad working hours, no psychological support, and you spend your day looking at rape, child porn, torture and executions.
>Content moderator is a min wage job
So it's purely a monetary dispute?
>bad working hours, no psychological support, and you spend your day looking at rape, child porn, torture and executions.
Many other jobs have the same issues, though admittedly with less frequency, but where do you draw the line?
How about grouping the jobs into two categories: A) Causes PTSD and B) Doesn't cause PTSD
If a job as a constantly high percentage of people ending up with PTSD, then they aren't equipped well enough to handle it, by the company who employs them.
I fail to see how this addresses my previous questions of "it's purely a monetary dispute?" and "where do you draw the line?". If a job "Causes PTSD" (whatever that means), then what? Are you entitled to hazard pay? Does this work out in the end to a higher minimum wage for certain jobs? Moreover, we don't have similar classifications for other hazards, some of which are arguably worse. For instance, dying is probably worse than getting PTSD, but the most dangerous jobs have pay that's well below the national median wage[1][2]. Should workers in those jobs be able to sue for redress as well?
[1] https://www.ishn.com/articles/112748-top-25-most-dangerous-j...
[1] https://www.policepac.org/uploads/1/2/3/0/123060500/the_effe...
I wouldn't say purely, but substantially yes. PTSD has costs. The article says some out; therapy, medication, mental, physical, and social health issues. Some of these money can directly cover, whereas others can only be kinda sorta justified with high enough pay.
I think a sustainable moderation industry would try hard to attract the kinds of people who are able to perform this job without too much negative impacts, and quickly relieve those who try but are not well suited, and pay for some therapy.
On the contrary I would expect that it would be the edge cases that they were shown - why loop in a content moderator if you an be sure that it is prohibited ont he platform without exposing a content moderator?
In this light, it might make sense why they sue: They are there more as a political org so that facebook can say: "We employ 140 moderators in Kenya alone!" while they do indifferent work that facebook already can filter out.
US workman’s comp covers PTSD acquired on the job, and these kinds of jobs are rife with it.
Even with images, you can have different policies in different places or the cultural context can be relevant somehow (eg. some country makes you ban blasphemy).
Also, I have heard of outsourcing to Kenya just to save cost. Living is cheaper there so you can hire a desk worker for less. Don’t know where the insistence you’d only hire Kenyans for political reasons comes from.
Why assume they're just token diversity hires who don't do useful work..?
Have you ever built an automated content moderation system before? Let me tell you something about them if not: no matter how good your automated moderation tool, it is pretty much always trivial for someone with familiarity with its inputs and outputs to come up with an input it mis-predicts embarrassingly badly. And you know what makes the biggest difference.. is humans specifying the labels.
I am also not saying that content moderation should catch everything.
What I am saying is that the content moderation teams should ideally decide on the edge cases as they are hard for automated system.
In turn that also means that these people ought not to be exposed to too hardcore material - as that is easier to classify.
Lastly I say that if that is not the case - then they are probably not there to carry out a function but to fill a political role.
I imagine that they have a system that is somewhere between shitty and none functional. This is the company that will more often than not flag marketplace posts as "Selling animal", either completely at random or because the pretty obvious "from an animal free home" phrase is used.
If they can't get this basic text parsing correct, how can you expect them to correctly flag images with any real sense of accuracy?
Not in the sense that matters here: the caller doesn't pay (unless the call is frivolous), leading to more calls that are preemptive, overly cautious or for non-live-threatening cases. That behind the scenes people and equipment are paid for and a whole structure to do that exists isn't really relevant here
> Do we think paramedics in Germany are more susceptible to PTSD
No, we think that there are far more paramedics than ICU or ER staff, and helping them in small ways is pretty easy. For ICU and ER staff you would obviously need other measures, like staffing those places with people less likely to get PTSD or giving them regular counseling by a staff therapist (I don't know how this is actually handled, just that the problem is very different than the issue of paramedics)
You probably DO NOT want to read it.
There's a very good reason moderators are employed in far-away countries, where people are unlikely to have the resources to gain redress for the problems they have to deal with as a result.
The difference? The reputable hotlines care a lot more about their employees' mental health, with mandatory breaks, free counseling, full healthcare benefits (including provisions for preventative mental health care like talk therapy).
Another important difference is that suicide hotlines are decoupled from the profit motive. As more and more users sign up to use a social network, it gets more profitable and more and more load needs to be borne by the human moderation team. But suicide and mental health risk is (roughly) constant (or slowly increasing with societal trends, not product trends).
There's also less of an incentive to minimize human moderation cost. In large companies, some directors view mod teams as a cost center that takes away from other ventures. In an organization dedicated only to suicide hotline response, a large share of the income (typically fundraising or donations) goes directly into the service itself.
Doctors and lawyers can’t afford that sort of option, but they tend to embrace alcoholism at higher rates and collect ex-wives.
Moderation may be worse in some ways. All day, every day, you see depravity at scale. You see things that shouldn’t be seen. Some of it you can stop, some you cannot due to the nature of the rules.
I think banning social media isn’t an answer, but demanding change to the algorithms to reduce the engagement to high risk content is key.
My guess is even standing at the ambulance drive in of a big hospital, you'll not see as much horrors in a day as these people see in 30 minutes.
They are obviously not identical scenarios. They have similarities and they also have differences.
Being a doctor and helping people through horrific things is unlike helplessly watching them happen.
IIRC, PTSD is far more common among people with sleep disorders, and it’s believed that the lack of good sleep is preventing upsetting memories from being processed.
Casual Facebook viewers may stumble across something disturbing on it, but they certainly don't get PTSD at the rate of the poor moderators. Likewise the professionals usually have their own professional exposure levels to messed up stuff. Meanwhile child pornography investigation departments who have to catalogue the evidence are notorious for suffering poor mental health even with heavy measures taken.
There is already the 'blacklist hash' approach to known bad images which can help reduce exposure. So they don't all need to be exposed to say the same cartel brutal execution video, the bot takes care of it. I don't know anything about Facebook's internal practices but I would presume they already are doing this or similar with their tech hiring base.
Dilution is likely the answer for how to make it more palitible and less traumatizing. Keep the really bad stuff exposure at similar proportions to what other careers experience. Not having 'report button' and 'checking popular content' as separate tasks and teams would probably help a little bit. I suspect the moderators wouldn't be as traumatized if they just had to click through trending posts all day. A dillution approach would still have to deal with the logistical trade-offs for what could be viable. Increasing the moderation payroll a hundred-fold and making them work at effectively 1% efficiency would make for better moderator experiences, but Facebook would be understandably reluctant to go from 5% revenue content moderation budget to 50% revenue content moderation.
That's bullshit. Ever talked to a paramedic or firefighter?
A content moderator for Facebook will invariably see more depravity and more frequently than a doctor or police officer. And likely see far less support provided by their employers to emotionally deal with it too.
This results in a circumstance where employees don’t have the time nor the tools to process.
Content moderators see all of that.
And it used to be priests who had to deal with all the nasty confessions.
Why not? What good has social media done that can't be accomplished in some other way, when weighed against the clear downsides?
That's an honest question, I'm probably missing lots.
I think not. The gatekeepers of the old media pretty much died for a reason, that they seriously sucked at their job. Open social media and everyone having a camera in their pocket is what allowed us to basically disprove UFO sightings and prove routine police abuse of power.