Facebook moderators break NDAs to expose working conditions
theverge.com
theverge.com
Facebook is enormously valuable. They made something like $15B in net income in the last four quarters.
Content moderators are a necessary condition for that profit. If kiddie porn, gore and animal cruelty flooded the network, it would cease to be a destination visited by people that advertisers will pay to reach.
And yet, there are two sets of entry-level knowledge workers at Facebook: engineers ($150k/year, benefits, upward career trajectory) and content moderators ($30k/year, no benefits, likely going to acquire mental illnesses).
I understand the arguments about supply and demand of labour, but I'd have more respect for Facebook if they demonstrated awareness of this issue. The article talks about moderators re-evaluating the same piece of distressing content that they've already flagged. Why? I suspect because the moderator is cheap, and so Facebook isn't putting in the effort to ensure that every judgment needs to be made the minimum number of times.
More so than salary, I suspect Facebook considers the moderator cheap in terms of reputation risk. By outsourcing to contractors located offsite from main campus, engineers aren't thinking daily about the absolutely horrible stuff moderators are seeing, and so the one group doesn't impact Facebook's ability to hire engineers. This is a guess - can anyone at Facebook speak to whether engineers are aware of the working conditions of moderators, and agitate to improve their lot?
This problem applies to other forms of sites and content as well. The app store gets hundreds of submissions in the game category per day. Hosting and distributing that content is easy. Only a tiny fraction of those games are going to be played and rated enough times to show up in a recommendation engine. The bulk of the incoming content stream isn't being matched to interested people at all (sorting by interesting is the same curation issue as filtering by offensive).
Literally every content platform is going to have some problem similar to Facebook. We can blame Facebook, but the reality is no one has a good solution. Not even me.
I don't think it's a problem any more than weather is a "problem" that required the invention of the roof. As far as user-generated content goes, it's easy to invent the hose, but the faucet is much more complex.
It's irresponsible to provide only acceleration ("reach," "virality," etc.) for information without also implementing brakes. Moderation has been a known fix for decades, but people like Zuck -- in disposition and inexperience -- won't launch anything human-powered, and moderation requires a human eye. Fuck us, right?
This is just Facebook being cheap.
We can take a step backwards and eliminate Facebook, Insta, etc from our societies and enact legislation that will govern the second generation of social networks.
They do this because acknowledging and dealing with bad users would also mean impacting fake users that make them money with ad fraud, growth metrics and ginning up engagement.
End of the day, if you operate a social platform, and you have people uploading child pornography and animal torture video, you should do everything in your power to make the user go away, for good.
I have no bussiness solution, but the solution to the latter is to simply ignore/block people who post stuff you don't like.
Maybe the best solution is for the moderaters to unionize.
The deck is stacked against employee organization. If I were conspiracy-minded, I might even say this was by design.
If there's something wrong with this reasoning, please elaborate.
That assumes perfect knowledge. As mentioned in the article, these moderators often do not understand what they are getting into until long after they have committed to the job. Once you have quit a previous job, moved, setup your new life, only then do you realize your new "brand consultant" job means watching kittens die all day.
Jobs like this also open persons to realities that we normal keep tucked away deep in our brains. We all know that cops don't investigate most reported crimes, but that doesn't come up very often for us. A facebook moderator sees the crimes, sees the lack of response by the police or anyone else, every minute. Repeatedly, every minute. Nobody can appreciate what this does to ones' mental state until you have been there.
C. Dickens: A Christmas Carol. In Prose. Being a Ghost Story of Christmas, 1843
That is not how the labor market works... At all.
"Worth the pay" is overridden by "absolutely need this money for me or my kids to survive"
Tell that to the people who pick your fruits and veggies.
Now we have gov externalities trying to legislate something that shouldn’t have existed?
I’m not saying any of this is write or wrong, I’m just at a loss how people get mad at Uber when the model shouldn’t have made it. Where did the market dynamics go wrong here?
I don't understand how this
isn't a problem perfectly
solved by the market.
Most people depend on their meagre income quite greatly, it's a bold thing to say someone could just 'quit' their job and spend 9 months looking for another one, especially if they don't have some pretty hard skills.
So in the realm of talent, yes, it would solve itself, which is why devs et. al. often get 'free lunch' among other things.
For most working people, it's much harder.
"All possibilities are fully encapsulated in very simple dynamics."
Yes, of course, but the incredible power asymmetry between FB and not hugely skilled workers is the dynamic that drives all of this.
anecdotal, friend who streams on twitch also mods for a few streamers and they even have issues when a stream is in subscriber mode only. simply because anonymity and distance from those being affected empower people to do bad things
edit : I am really surprised there is/are not companies which would spring up to provide these services seeing how most of these activities are required by law.
however before bemoaning what they are paid, just go look at your local 911 operators who are government employees. just because we think it should be paid more doesn't mean others do.
It seems like there are no easy solutions for this and it's really frustrating: Societal norms are just completely shredded right now.
I think for most people that I'd want to be moderating content (e.g. not sociopaths) making them richer isn't going to do anything to deal with the real reason this is worse than a call centre - the content itself. I'd rather see facebook put more effort into protecting moderators from the content they're viewing. I realise this is a non-trivial problem, but here's a few ideas of things they could do which may help:
* Once a video is confirmed bad, fingerprint it and block future uploads.
* Provide colour-inversion buttons to help reduce visual impact
* Rotate people between areas, so they aren't looking at the same class of bad content constantly
* Use ML training to detect which content is likely to be the most severe and for this content, reduce minimum viewing interval to ~ 5 seconds, and ensure that a single individual doesn't get too many of these.
* Flag accounts who post bad content too often, and shadow-ban them so their posts are not visible by anyone else (this could go in stages, starting off by blocking re-shares of their posts or something)
Too often? If we're talking about content that is traumatic to view (extreme violence, child porn, etc) then I think one time is enough for a flag, no?
That said, I do agree that shadow banning is the best option. Let them spin their wheels for a few weeks posting content before they realize no one is seeing it.
To add to this, some degree of pixelation would help without impacting the accuracy of assessments.
I was thinking also about blurring the picture/video.
Neither the text nor the image themselves would be ban worthy but in combination it is boot-worthy. Extrapolating has Scunthorpe problem issues.
The only partial mitigation that comes to mind would be if a small circle of your friends had to "approve" your video before people outside your circle could see it. That would put them on the hook for sharing bad things too. That might even reduce the psychological load on the moderators.
Also in terms of the reputation of Facebook: any issue with moderation failures and Facebook can easily says that it's the contractor's fault
I want to believe the content moderators only end up seeing the 1% of content that is dubious, or brand new content (not published and hashed earlier). I don't understand in that regard why the one video mentioned in the article (about the iguana) wasn't marked as "ok" in the systems, or marked as "on hold for law enforcement" so the moderators didn't have to see it again.
They are passively aware but mostly don't care. The engineers who express frustration are in the vast minority. Most engineers are just super happy about their own situations, very focused on PSCs, and try to keep to their own stuff. Any internal frustrations are channelled towards appropriate feedback channels where the anger is watered down.
Their profit is not proof they are valuable, because they are willfully abdicating responsibility for the damage they cause, and, further, involving themselves in government debates over their own regulation.
Advertising businesses are inherently suspect, and when one is addictive and potentially enabling election fraud etc, we need to be extra careful in declaring its value.
Anyone participating in the debate should have to disclose if they are a daily user or if their company advertises on any social network. (I'm clean)
Edit: my opinion of course is that FB, Insta, etc (any advertising product with push notifications and/or infinite scroll) ought to be illegal. In 50 or 100 years we'll all recognize that social networks are like internet cigarettes.
Well cigarettes are still not illegal, so we have some ways to go...
Why do you think they don’t get benefits? Facebook contracts out to another company. The company they contract with still is probably providing some benefits
This leads me to believe that there is a qualitative difference in the calibre of benefits Cognizant provides versus those that Facebook provides.
Further, since the job has high turnover and can be expected to put a high toll on an employee's wellbeing, it's also important to me to think about what happens to their benefits if/when they quit. In the US model of employer-linked healthcare, this tends to result in losing your benefits when you need them most. By contrast, other organizations that do similar moderation jobs seem to offer psychological health care treatment even after the employee leaves the company. See https://www.theguardian.com/news/2017/may/25/facebook-modera... for more info.
It is a very politically correct statement, which, as it usually happens with politically correct statements, really says nothing. I mean, what exactly do you propose they should do about it?
I am the last person to defend Facebook, but, seriously, they are running a business. That is, essentially, finding a way to do/own/produce some stuff while spending less on it than you receive for it. And cost management is all about supply and demand.
After all, it's not like Facebook wants to pay engineers $150k/year and for them to be happy. It needs good engineers and the only way to get them is to keep them happy, or else they stop coming. Engineers are not really special, they are not even exactly more necessary than cleaners, and loaders, and content moderators. The only difference is, demand for them is higher than supply.
And that would be tragic in a sense, if everyone would be randomly assigned a role and if you draw an "engineer" you are lucky, but if you draw a "content moderator" you are not. Or if we were talking about some factory in some giant village in Vietnam, where no matter how smart you are and no matter what you do, you'll be working much harder job in conditions far, far worse than described for $500/year and you really don't have any choice.
But we are not. No, this is Florida, $30K/year and these are people choosing the job of content moderator over a job of an engineer, either because they don't want to or cannot do otherwise. They while many of them may have stories that can be seen as tragic (because who hasn't, really?), they are no more victims than anyone else born into living a life on the Earth. And thus, hardly entitled to anything.
Let me be clear: I really hate Facebook and I find companies with a modus operandi like Congizant's distasteful to say the least. And I can see how Cognizant can be held accountable for giving somebody a false hope, luring into doing the job while promising something else. But what exactly do you want for Facebook to do? They want a service as cheap as possible, they get asked a given price, they agree. That's it. It won't save their reputation after an article like this (because, well, public relations...), but it seems just fair.
If you go work at Facebook as a engineer chances are you aren't too bothered by ethical questions
-Public (https://www.facebook.com/communitystandards) and private guidelines
- A "Known Question" document (15k words)
- Facebook internal Workplace, with real-time (and sometime contradictory) decisions, for breaking news events
from https://www.theverge.com/2019/2/25/18229714/cognizant-facebo...
The second one is pretty important. Consider that sales, sports, and entertainment work similarly, where you can make a case that you should pay more to get the best and workers (or their agents) can use this to good effect.
> Cognizant also offers a 24/7 hotline, full healthcare benefits, and other wellness programs.
My experience with the US health care system is that there is a wide gulf in health benefits. When I worked for Microsoft in the late 2000s(as an employee, not as "dash trash", as contractors were called), I paid some very small premium like $10-20/month for a plan with 0 deductible, 0 copay, and pretty much every doctor was in network and covered at 100%. When I dislocated my shoulder, some orthopedic surgeon who used to consult for an NBA team looked at it for me.
I understand that that was considered quite good insurance and that most people in the USA don't get that level of care, much less at that price.
If you read Glassdoor reviews of Cognizant, people complain about the insurance offerings available to employees. https://www.homebaseiowa.gov/sites/homebaseiowa.gov/files/do... looks like it might be what they offered 2 years ago. It doesn't look very good to me. If I was earning $15/hour, I think I'd pretty much buy the cheapest plan and only use it if I was actively bleeding to death.
I'll also link to https://www.theguardian.com/news/2017/may/25/facebook-modera... again - it describes how other companies that moderate distressing content approach this. It includes the company paying for psychological treatment, including after the employee leaves the company.
I bet this can be done with ML and IA instead human power. Yet, FB uses their human cattle to censor conservative voices.
IA and ML work in bulk. But they misclassify things that a human easily can discern.
Think Tesla VS Google. Google wants a self-driving car, and that is not working yet. Tesla wants a car that helps you to drive and that is good enough.
It's order of magnitude easier to be almost always correct that to be correct all the time. And you only need one kiddie porn video in your Facebook feed to start to worry about Facebook.
Where are the moderators living ?
Or the opposite. People flock to the internet to see this stuff, the material that isn't on TV or even dedicated commercial websites. They get to view it in the privacy of their homes, a secret between them an their trusted "friends" on facebook. Facebook knows that if it really cracks down, if cops bust down doors over this stuff, that people will go elsewhere to share horrible material. So facebook does a poor job of moderation, one just good enough to avoid new regulation but not so good as to actually make a difference.
Advertisers care about eyeballs. They may claim to not want to be associated with X or Y content, but in reality they would rather be in front of an audience than not.
There's a whole section of adtech dedicated to brand safety.
I believe it's over the top[1] but it's something marketing folks do care about.
Also the good old pop up ads were created for that reason, aeons ago.
[[1] https://www.campaignlive.co.uk/article/advertisers-embrace-n...
People who work at Facebook should be pushing for change. But they're numb to the schpiel. They're cushy and looked after and don't want to create a fuss.
Rosen doesn't care. Zuck doesn't care. Sheryl doesn't care. What DO they care about? Perception. Sit in any high-up integrity meeting and you'll see the only thing they seem to talk about is how "x" would be received by users at scale. There's no comment as to the ethics or corporate responsibility. You can be talking about something pretty out there like how human rights intersect with takedown decisions and all you've got is a bunch of people umming-and-ahhing about lossy metrics and how Zuck wants this or that so we better hurry up. Or how awwesome it'll look on our PSCs if we ship this thing.
Broken company.
This is utterly nightmarish, given how costly to one's life bedbugs are. (clearing out a home, including the replacing of all mattesses/couches, and bagging or hot-cleaning all clothing, sheets, towels, rugs...)
"A manager saw that she was not feeling well, and brought a trash can to her desk so she could vomit in it. So she did."
This particular manager put their employees in danger of catching illness, especially given what appears to be the open office floorplan where airborne sicknesses can travel the entire room. I'm shocked and apalled, and this is the stuff I'm comfortable quoting from the article to be shocked and apalled by. The other stuff has convinced me to help inform friends and family to get off facebook rather than passively clean myself of it only.
Once an employee has used their allotment, they receive a write up if they take off again no matter how ill. Too many write ups and you're fired. Managers have no discretion in this process, so they can only mitigate the impact by doing something like bringing the trash can over or buying hospital masks.
Not saying it’s acceptable! But this isn’t a facebook problem specifically.
How stupid are we as a nation that we don't mandate more humane/non-stupid policies, even for people in environments where coming to work unhealthy can often result in death?
It says a great deal about how broken the United States' job market and social safety net are. If minimum wage were $15, they could find another job that paid their basic living expenses. If health care weren't left up to your employer, they wouldn't be out of luck while looking for a different job. If there were any alternative, they wouldn't stay in this hellscape.
They stay because this is the best deal they could find. Think about the kind of society that makes that the case.
If you're making $30K you certainly don't have any kind of buffer to hold you over between jobs, and don't have skills that are attractive to employers. Even basic things like interviewing are much harder. An engineer or data scientist can pretty much disappear from their desk for an entire day and no one will ask questions, you don't need time off for an interview, and if you do you likely have tons of leave. Someone making 30k most likely has to give notice of vacation many weeks in advance, if they even have leave benefits at all. And because your skills are less in demand your interview success rate is going to be much lower so you'll need even more time to interview.
It's likely very hard for these people to find new jobs even if they hate their current one, they are probably happy just to have income.
These people are literally trading dollars for their mental health. That is the choice they are making.
They aren't hustling because they want some extra cash. They're trading their mental health for the basic ability to be a living, eating human being.
Why is this necessarily true? Couldn't it be the case that minimum wage were $15 but they still couldn't find any other job?
Edit: I may have misspoken when I said "surplus". What I was referring to is the very low unemployment rate we currently have, despite large numbers of people - with "jobs" - continuing to live in near-poverty.
> Why is this necessarily true?
It costs money to take a surveying class, or buy tools to practice your skills. It also takes having money saved so if there is a crisis, car repairs, etc you don’t have to constantly work overtime to keep your bills paid.
Please reach out to a friend who is making less than a “living wage” in their area and ask them about career development and what they need. I appreciate your question.
... then low-skilled candidates would NOT be able to find any job at all, because if low skilled candidate's expertise does not support $15/hour salary, and employers who could, potentially, pay less (e.g. $10/hour) are legally prohibited from employing people at that lower rate -- then there are no other jobs available to that low skilled candidate at all. Which means forced and hopeless unemployment.
These people do not work for Facebook, and we don't know the nature of the contract in play. Are they paying per person, or a lump sum for some capacity at some accuracy rate. If Cognizant automated all of this would it be accepted under the contract?
Anyways, I don't want to shift focus away from Facebook so much as wanting to recognize the contracted call mpanies like Cognizant (which is what the whole article is about btw, with some comments referring to Facebook). Accenture and Cognizant really shouldn't escape the scrutiny just for being overshadowed by a bigger name.
In the article, one of the contractors says that Cognizant puts up a "dog-and-pony show" whenever FB executives visit. Again, it's ultimately up to FB to decide how much they want to push past the facade.
I don't think I could do that job for very long - let alone in a badly run, high pressure environment with low wages.
That one paragraph about organs was enough to ruin my day, and it was just text. I'm surprised such a "rash of videos" wasn't in the news somewhere.
He sought out the on-site counselor for support, but found him unhelpful.
“He just flat-out told me: ‘I don’t really know how to help you guys,’”
I'm left wondering if it was a clip from a horror movie or something. Shitty to see, but not necessarily newsworthy like "live children chopped up for organs" would be.
I'm no fan of Facebook, but in their defense, this is sort of the point.
In fact it's their very realness that makes them especially horrifying.
Some of the people interviewed were complaining primarily about the bad working conditions (and their complaints are valid as far as I care). I would wager these people are not as bothered by the content (though I'm sure they don't like it) as the ones who's primary complaints are about the content. They could probably do the job with less burnout if the rest of the job was made to suck less (i.e. it wasn't a shitty call center style job with all the accompanying baggage).
Edit: Why am I getting down-voted? Can people legitimately not fathom that some people would not be seriously bothered by seeing this content? People post violent content to Facebook. Shock and gore sites exist because some people actively seek out(!!) the kind of content that these moderators are being exposed to. It stands to reason that the subset of the population that at least finds that content not mentally damaging is substantially larger than the group that seeks it out.
My wife, who has worked in a busy McDonald's for 5 years, says this sort of moderation is far, far worse than anything she had to deal with. And she's had to deal with human waste, violence, and direct verbal abuse from customers.
Your point about making the working environment better is a valid one, but think it is overshadowed by your assertion that the horrific content is acceptable to "many people".
Your views do resonate with some people on Reddit though. I remember saying that I didn't like the pained yelps, limping, and whining of Dogmeat in Fallout 4, and I was attacked and ridiculed for that. I know I wouldn't last more than a minute at this Facebook moderation job, it would scar me for life. I don't think that means I'm "wrong", any more than your views are "right".
There are also people who are habituated with cutting into living bodies (surgeons) but the difference is they are highly paid.
It's a bit unfair to equivocate and call this kind of work a good option for some people, when the vast majority of people who are in these jobs probably will be psychologically harmed by it² and are doing it because they have few other options.
It is relegated to an underclass, like all dangerous and undesirable work.
1: Statistically at least. 2: See also: https://www.researchgate.net/publication/228141419_A_Slaught...
Of course the real answer is that this sort of site is a bad idea and shouldn't exist. Wide-open signup and public visibility of content, or ease of sharing it with strangers. Bad combo, don't care how much money it's making them (and other, similar sites).
Oh, dear god, no - have you ever used Reddit? This is what happens when you outsource moderation to the sorts of users who enjoy moderating other people.
------------------------------------------------------------
Hello user, you have been randomly selected to help moderate.
Does this depict pedophilia? Y/N
[ Image here ]
This image was flagged by other users as depicting horrible gore and death, do you agree Y/N?
[ Image here ]
This post may contain hate speech, do you agree? Y/N
[ Embedded Nazi Screed post ]
Thank you for making Facebook a safer place for everybody!
------------------------------------------------------------
"Facebook made me look at child porn" is probably a headline Facebook would prefer not to have going around.
Would that be expensive at the scale required by Facebook? Very.
Overall not possible. On Slashdot everything is basically public, on Facebook it's largely private/restricted and I'm sure from past discussions that the worst stuff is in private groups or clusters of friends. They could do much more aggressive banning of users (and detecting new accounts designed to circumvent bans) for sharing such content or being in groups focused around such content, but that might hurt their most important metrics.
(stolen from Scott Adams.)
I actually wonder now how much of their workforce would fit in that demographic.
Granted, not everyone does that, someone nibbling at the frontend or sitting deep in some distributed DB generally does not care much about data quality.
It’s management 101
[Clarity: Am not a Facebook employee.]
Now, as before, no-one wants to see how the sausage gets made. Especially those selling it.
Can't kill demand or bear the visceral truth. So instead we'll pretend the seedy underbelly doesn't exist. Paper over dissonance with ethical codes and platitudes.
Not new. Just a context shift in production of sustenance for the collective, insatiable gaping maw.
I also believe this is one of the reasons where Facebook's real name policy comes in - it discourages people from posting the worst of it. Animal brutality could just be kids fooling around on their phone, accidental, but produced child pornography is not, and the people making that shit know really well how they shouldn't put it on sites that require them to use their real names.
Facebook is really broad, so are also online news magazines which are sometimes full of horrible comments (only text of course). Usenet more or less ceased to exist because of that. So in reality this is not a Facebook problem but an online community problem - it just looks like a FB problem because that's the major online community.
I wonder if this could be solved with a 3rd party Facebook integration/startup :-). Anyway, Facebook also has more problems with that, another one being undesired unsolicited contacts which fall into the same category of not so nice stuff about Facebook. People don't see the meat production but the bad smell is really close which is why many leave.
They would hate it so much, that would make better tooling. But now they don't have to know about it, just send images to the moderation team over and over like they are robots.
I think if FB engineers had to actually interact with the bottom-bucket of the content on a regular basis, Facebook would have stricter rules, and perhaps even prior-restraint filters for image and video uploads.
While putting themselves in the moderator's shoes may prove helpful they aren't comparable in skillsets. If they were given a command line interface for instance most mods would be puzzled. "Produced for self" and "produces for content moderators" are differing needs.
Using the things you produce generally helps motivate ironing out bugs and/or improving the product overall.
End result: engineers paid $250k, moderators unemployed making $0k.
At least that way they get a bit of a break from all the other horrible stuff.
The article says staff were constantly reminded of how easily replaced they were, which is a euphemism for “you’re lucky we gave you a job.”
Facebook and Cognizant are majorly dropping the ball but US government and legislation strongly enables that. As do individual states.
From a European perspective this is a sad article to read about work conditions in “the greatest nation in the world.”
Cognizant Technology Solutions
Tampa, Florida • This is an exempt position, requiring day, evening, weekend, and holiday shifts, as this delivery center is operational 24/7 and 365 days a year.
Cognizant is seeking a team of strong Team Leads to manage a team of social media content moderators for a global social media organization.
The Team Lead will be responsible not only for managing day to day operations of the team, people management, performance management, but also help the client determine gaps in processes, identifying innovative ways to solve problems upstream and scale our operations.
Ideal candidates will be comfortable understanding social media, have an appetite for research and gathering data insights, a high level of comfort working with cross-functional partners, and a strong analytical mindset. Successful team members have a passion for business success, strong attention to detail, analytical problem-solving abilities keeping a high level of team motivation and keen eyes for operational inefficiencies.
Responsibilities: • Provide mentorship, guidance and career development to members of your team • Lead a high-performing team through an exciting transition to build problem solving, critical thinking, analytical and technical capabilities which will enable the department to develop deeper, more scalable solutions • Team management responsibilities for a market team, whilst also serving as a cross-functional and a global liaison in developed areas of expertise • Establish team goals and work with direct reports on strategies for executing, measuring progress and sharing results • Deliver projects involving quantitative analysis, industry research, and strategy development, working directly with global cross-functional teams to problem solve analytical approaches and develop solutions • Identify actionable insights, suggest recommendations, and influence team strategy through effective communication • Advocate for users within their market, partnering with global and cross-functional teams to develop global solutions
bindsym $mod+backslash exec "xcalib -i -a"
which inverts all colors (btw, I have to `killall redshift` because of a bug). When colors are inverted, I feel almost undisturbed at any gruesome content (for me it's like seeing screenshots of Quake 2 with buggy GPU drivers), yet I can still easily recognise whether the content is disturbing. And when it's a video, I play it at ≥2x speed so that it wouldn't feel realistic for my brain.
I wonder whether those moderators use these two lifehacks.
Not sure how you get around that.
The chief information security officer in my organization came from our state police where he headed the internet crimes against children division. People on his team had to watch videos like the ones described in the article. Despite having all hardened officers on his team, team members regularly cried at their desks while watching videos. His team members had to be taken off-duty for weeks every six months to recover. They had mandatory counseling even when off-duty.
I would think FB and other large companies could look at the measures taken by police forces with similarly disturbing jobs as a guideline for the contractors.
Should employers really be asking about mental health history during the hiring process?
There should be some disclosure of potential mental health risks in the job description. no? i.e. Screening for this at a later stage in the hiring process would be reasonable (No idea on the legality, however)
My decision to delete my account from this toxic spider web of a platform is affirmed regularly by the frequent revelations of their immoral behavior.
Seems like it should be pretty "easy" to spot similar videos and audio (even after some modifications) given how great dataset all those moderators are providing.
I'm pretty sure there are some pretty smart folks working at fb, so it would mean that given accuracy standards it's still cheaper for them to hire humans to do the job.
Ban the user and the video, if the same video is posted again, autoban it. If the user is law-enforcent, allow them to see the details of all related content that was banned.
There are neat things called if-statemets that could do that.
However, I just also described how Facebook could easily be used in a dystopian police state fashion. I am fairly sure they aren't allowed to proactively report crimes. And that law enforcement would be overwhelmed by the sheer amount of reports coming in all at once.
Especially screams in the audio should be fairly easy to find.
Then block those videos by default, with a manual appeals process that sends it to a moderator, combined with a big warning that submitting videos against the TOS will get you suspended. Of course this could lead to people submitting videos without audio, but this will always be a cat and mouse game.
Or is FB under legal duty to review potentially criminal content?
Even if it could (and parallel posters argue it cannot), there is a meta-problem: "Who is allowed to post violent content?". Violence is contextual, and also legality of posting it is contextual. Same goes for other objectionable materials (nudity, extreme ideologies etc.).
Point in case, if all users were subject to the same criteria, the big names in media would quickly get banned. And/or would enter war path with Facebook, accusing it of heavy-handed censorship against the freedom of the press.
Same for reportage of popular events with any degree of nudity. A news piece about a Femen protest, a Pride event, a World Naked Bike Ride, maybe even the No Pants Subway Ride would cause a ban.
Likewise, anybody documenting & discussing historical events, and anybody documenting & discussing present-day uprising, revolutions, civil wars, persecutions, etc., would quickly get banned, essentially sweeping a lot of abuse under the rug. Probably even discussion & documentation of domestic violence would have this problem.
As a lighter aside, cartoonish violence (video games & movies) could also easily fall prey to automated violence take-downs.
All in all, Facebook really really wants to give certain users (mostly press & historians) broad, sweeping exceptions to the general rules.
I thought the US had finally come to peace with nudity, but that was extremely wrong for every side of the political spectrum as the latest developments clearly show.
Nope. It's just not that easy. Why do you think the biggest companies on the planet struggle so mightily to reliably and un-controversially figure out what content to block.
Facebook doesn't have a blanket policy of removing violent content, for wholly legitimate reasons. Many users in Mexico very strongly want videos of cartel murders to stay on the site, because they feel that they are important reportage of something that the mainstream media is unwilling or unable to report. Many users in Syria very strongly want videos of IS murders to remain on the site, because they want the world to see the suffering that IS has wrought on their country.
If Facebook just delete all violent content, they're robbing the victims of violence of their voice - it doesn't feel like Facebook are protecting users of their platform, it feels like Facebook are complicit in covering up the crimes of drug cartels and terrorists.
It's not enough to identify that a video is violent - they need to know context. Is this particular video in this particular context a cry for help or a celebration of evil? Will removing this video silence the propaganda of a murderer, or will it help to conceal their crimes? Does this video show the death of an unidentifiable person in an ungoverned warzone, or is it vital evidence that urgently needs to be forwarded to the relevant authorities? For now, only well-trained human beings are capable of making that call.
This sort of dilemma points to the fundamental impossibility of moderating Facebook to the satisfaction of everyone. Some users complain about pornography, so Facebook make a policy to take down the porn. Other users complain that these policies are sexist and protest outside Facebook's offices to free the nipple. Facebook end up with several pages of rules that allow their army of moderators to consistently distinguish between pornographic and empowering nipples, but nobody's really happy with the outcome.
Facebook are doing a really crappy job of looking after their moderators, but that issue is fixable if we apply enough pressure. There are many other problems with moderating Facebook that are far less tractable and have far wider consequences.
I guess it depends how reliably it could determine between ordinary screaming (especially with children who scream at just about anything that makes them happy or unhappy), and 'terrible things are happening' screaming. Even most humans would probably struggle to tell the difference just based on a short audio clip of a scream, which is what your algo would have to work with.
Rollercoaster videos would be banned, as well as haunted houses, neither of which are terribly offensive.
No. Already filters fuck up way too much and restrict perfectly legal content.
Would users actually have a problem with that though? People have the freedom to share whatever they want, but that freedom doesn't necessarily have to include sharing it on Facebook. Facebook could decide to tell their users "We're filtering out the bad stuff much more aggressively now and unfortunately that might block your content by mistake." I think a lot of users would be fine with that.
As long as this exists and actually leads to human review, I'd much rather wait a while for my videos to be posted if it means easing the burden on 15.000 people having to watch highly damaging content.
On the one hand, of course you don't want to see this and of course you want it removed from your social media feed.
On the other hand, if it's hidden, it horrifies fewer people, and instead of rising social pressure to take action against it, it can fester in the dark.
So if we manage to replace a significant chunk of centralized, moderated social media with decentralized, unmoderated alternatives, many of us will be exposed to more of this kind of evil. But as a result more of us will be aware of it and motivated to fight it.
I'd rather participate in unmoderated media, even at a greater risk of being assaulted by this kind of crap. But at this extreme I can sympathize with those that want straight up censorship, even if sunlight is a better long term disinfectant.
“I think Facebook needs to shut down,” he said.
Cognizant Technology Solutions Woodlands2 7725 Woodland Center Blvd, Tampa, FL 33614
There's another big story here, it may get more horrific, from browsing their phone directory most of their "employees" appear to be people of south Asian nationality. Given the circumstances and their predatory behavior, I imagine they are also taking advantage of H1Bs.
Posting violent, hateful, evil stuff is as old as the internet itself. There will always need to be moderators. The question is how these people are treated.
Do you really think it is _impossible_ for a company as big and valuable as FB to treat these moderators well? Do they not have the money to pay them what they deserve, to give them sick leave, and to offer even basic mental health services?
Imagine if half or even a quarter the money and effort that has gone into their crypto debut had gone into helping build better tools and services for their moderation team.
I'd definitely believe that that is the case, as there's little economic incentive to play the long term game with its uncertain payout if you can just churn through low-wage content moderators instead.
Users of Facebook, but more realistically, employees of Facebook are the ones best situated to pressure Facebook to change that behavior.
in general response to this article there was a thread on twitter with some good links https://twitter.com/max_read/status/1141336189800239107
This moderator labor is (relatively) cheap. It also allows FB to point to a third party if something goes wrong.
Look at their statements in the article. It's easier for them to distance themselves and point to "bad actor" contractors than it is if these were direct FB employees.
The only reasonable explanation I can imagine is that Facebook is doing everything they can to avoid having to implement a "Content ID" system like YouTube. Now why exactly they don't want to do that can only be speculated.
Dear Lord, what a horrible thing.
It seems to me that they put companies like Cognizant between them and the exploited workers to deflect criticism as "we didn't know, we've been told by our partners that everything is great".
And on a technical level: if true, how can it be that FB needs the same videos and pictures to be moderated over and over? Are they not using any form of content id? Is developing such a system more expensive than running content moderation sites and then swiftly firing the burnt out moderators?
At what point do you become complicit if you're working for Facebook?
There's also the aspect of if we make it search out for distinct frames (like "scene change" type things), there's the possibility that some safe things could become auto-moderated which would be another controversy. Perhaps if they could locate start/stop times for inappropriate elements of the source video and then check for videos containing any of those in a later uploaded video, but then instead of asking human moderators to discern "is this video good or bad" we're now forcing them to get timestamps of when bad elements take place which would force them to really watch the video.
Dunno, I'm of the belief that it's just not an easily solved problem. I hope I'm wrong and they can toss money at the problem to fix it, but I genuinely don't know how they could.
I assume that most of these videos aren't manipulated by professionals to escape a ban by looking completely different (which actually would make it a different video), but that they are mirrored, cut, have extra material appended or, as you suggested, have their resolution changed. I'd guess that videos are easier to ID than, say, image data, because you have plenty of material to work with.
Are there well-organized troll armies attacking Facebook? Because trying to sneak a video in that shows an animal being abused and doing sophisticated changes to evade detection sounds weird on a platform where there isn't even (as far as I know) monetary gain connected to posting a video (as it might be on YT or for Email spam).
I know that a lot of people here are probably bothered by the fact that the workplace sounds like impersonal zoo but most of these minimum wage workplaces are along that spectrum. Dirty bathrooms, inflexible policies for everything, etc. are normal. It's just how workplaces like this are. If you want a shitty job that's more personal and flexible then you need to work for a small employer but that can have its downsides too.
Sure the content itself sucks particularly but that's why they pay bigger bucks than everyone else if you can cope then good for you. If you can't you leave. It's just like a call center but more extreme.
Don't mistake me as defending Facebook or Cognizant here, I'm not. I'm just saying this that all things considered this doesn't seem like a particularly bad deal as far a shitty unskilled jobs go. They all have their pros and cons and you gotta find something that works for your personal preferences. I'd take digging ditches over anything with rigid corporate policy about every facet of my job but that's just my preference.
Personally something I think would help a lot would be if they'd over-staff the place and schedule people for few enough hours that they can easily hold another job and make efforts to accommodate people's other commitments. Just doing content moderation every day is probably where a lot of the mental and physical health issues comes from. In my experience the kinds of jobs that really grind you down grind you down a lot less when they're your side gig to some other slightly less shit job.
well we've heard the line that the tech isn't there yet. or how you have learned from past attempts to help and are taking more targeted approaches.
ok, that works for the retarded masses and corrupt congress.
but I call bullshit. its actually more than that, its malicious intentional lying.
you know how you can help? heres a really easy way. take some of your money, give it to these slave/content-moderators.
100,000 to you is not 100,000 to them.
how much would you need to pay me to watch that shit? zuck man, you can't give me your salary to watch that. that shit damages you for life. and you think its worth whatever?
yall walk around (figuratively, on the internet) like you have something to be proud of. donald abraham is confident as shit.
but you know, yall are just like any other mega. sure each mega has its own demographic. but the entire reason for the existance of a mega ensures 100% you will engage in unethical behavior. and you will recreate the extremely oppressive society we live in everyday. Furthermore, you will recreate the belief in the opposite: things aren't so bad and we `good peoples`
Wow. Just Wow.
We know that FB has nudity detection systems, due to their past problems with banning breastfeeding discussion groups.
As I've spent the last few years pissing away absurd amounts of time on the platform, gotten in countless fruitless arguments, and seen the truly vile and toxic elements of my communities exposed and worn like a badge, this is an idea I've been thinking an awful lot about. After reading this article, I've never been more certain of that statement.
Shutting down FB will just create this problem in the next popular social media platform. This is a problem that needs to be solved, and shutting companies down won't do it.
When a social gathering (whether its a party at a club or a riot in the streets) exceeds capacity and becomes dangerous, destructive, and out of control the police shut it down for the good of public safety.
Absolutely unethical. Facebook should not be allowed to hide behind subcontractors.
Here are some other ideas:
Instead of having content moderators watch 200 of these videos per day, distribute the load over the entire Facebook workforce. Or, when a content moderator flags a video for removal, verify that by making a regular Facebook employee watch the video to confirm the removal. This should increase the accuracy rate above the 98% threshold Facebook has set. That should put a finer point on the problem.
I could barely read descriptions of these videos. There's no way I could watch one, let alone 200 a day. This work is like being in a war, and they should be paid like a soldier who is risking their life / mental health.
I have worked for too many startups with inadequate bathroom setups.
It is a big red-flag for me now.
I visited one private company where I talked to the owner - his office was plastered with pictures of his racing Porsche but the bathroom had rusted metal and peeling paint. Took me about 5 seconds to see where his priorities lay and I bugged out of there.
I am so glad i am not on facebook since 2008. If you think it's not messing with you, THINK AGAIN. Facebook is the worst thing that happened to us, except for Mark and it's employees
It seems that Facebook and Cognizant are not the only ones completely failing at protecting their people (employees, contractors, citizens).
The city of Austin recently passed a mandatory sick leave policy, only to be struck down at the state level after lobbying by employers.
It’s not that companies like Facebook simply fail to protect their people, they’re financially incentivized to undermine their rights. And since money == free speech in the US, it’s perfectly legal for them to do so.
In the end most failings can be traced back to financial interest (getting cheap, staying cheap, aiming for cheaper). But the whole article was about how Cognizant and Facebook fail those poor souls who have to go through a "peaceful war", with all the gruesome tragedies but none of the preparation.
The state doesn't seem to try harder either. This particular detail related to sick leave was surprising to me, I hadn't expected this.
They should have an investigative unit that tracks down the source of this material and partners with law enforcement so we can catch the bastards doing and posting this horrible stuff.
I can imagine a world without Facebook.
People can be awful. So can other organisms (independent of whether they're evil or just immoral). Not facing that fact, at all, isn't obviously best.
I wonder if this is true for them as well or if it's just N. America -- we've better employment protections and minimum wage in Ireland compared to the US.
https://www.irishtimes.com/culture/tv-radio-web/facebook-s-d...
Win-win?
A lot of the bad content is highly desired by criminals and can even be profitable to sell on the dark web (e.g. child abuse).
Absolutely unethical.
>>"Under the policy, the video was allowed to remain on Facebook. A manager told him that by leaving the video online, authorities would be able to catch the perpetrators."
Utterly disgusting and inhumane policy on facebook's part.
Continuing to display abject animal cruelty only lowers the bar for would-be imitators.
There have also been shown to be strong links between animal cruelty and human cruelty, including murder.
To be clear, the ONLY proper way to handle this is to immediately take it down and file a report with the relevant police agencies. Expecting the local police agencies to maintain the same staff of 30,000 people to monitor FB posts for crime is stupidly absurd.
This is at best depraved indifference on FB's part, and more likely a deliberate dishonest rationalization to keep posted something extreme that will get lots of 'views' and 'engagement'.
This is only one of millions of examples, including cooperating with the Russian / Cambridge Analytica / etc groups to corrupt elections in the US, England, and elsewhere.
Simply put, Facebook is deliberately poisoning society for profit, and far worse than any tobacco company ever did.
They need to be shut down. Now. There are far better ways to do everything FB claims to do.
(edit: add police report paragraph)
FB vigorously opposes this.
However, with 1) a staff of 30K++ screeners, 2) who knows how many moderators, etc, 3) many layers of management decisions and policy on what is and is not allowed on the platform and the reasons why/why not, 4) extensive human & algorithmic promotion and demotion of content, they are absolutely actively editing their site. This is arguably even more extensive editing than from any print or media organization.
The only reason FB wants to avoid the "editing" label is so they can edit as they please. I.e., they want to edit for 'engagement' and profit, not for social responsibility.
This cannot remain out of control without consequences, which we are already seeing in society.
Assume that you are 100% correct, the species is undesirable invasive and that is the proper procedure.
Repeatedly bashing their head while the creature is obviously screaming in pain is blatant cruelty. Not only do all the previously stated reasons to pull the video & report it to the police still stand, you provide yet another reason to pull it -- it shows a horribly bad example of how to do it, providing negative instruction and promoting bad & cruel technique.
Moreover, based on this and numerous other articles and events (e.g., leaving posted a doctored video defaming the person 3rd in line for the US Presicency), this sort of behavior is fully consistent with Facebook's modus operandi.
Simply, FB are happy to poison society for profit, and disingenuously justify it in the name of 'free discourse' and by claiming to have no editorial control.
We must at the very least hold them responsible for their content.
A classic sweatshop operations except it also destroys the slaves psyche on top of it.
Two parties weren’t really mentioned. The executives that set up the operation and the people posting the content. Both must be pretty twisted
Do users know that their personal information is being looked through by employees who don't work for Facebook?
I guess I don't understand the key change in the medium or the culture that requires Facebook's ponderous rule tome and 10000 anonymous content moderators in Manila or Phoenix. Is this just about the risk of some corporate ad being next to some undesired content for a few page views?
I’d love to hear an argument from someone defending this company that isn’t “everybody does it.”
As usual, the author manages to make the employer responsible for employee's life situation. But aside from an impulse of assigning blame on the closest most powerful actor related to situation, I don't see any reason to see Facebook in any kind of negative light for this. They need a service performed, they found people who're willing to do that service for the money offered, they gave them this money. They didn't create neither the job situation, neither the lack of education, neither the whole life of choices that lead these people toward this point: they aren't even in a any condition to control the labor market.
And speaking of labor market, I really don't think that they would do anyone a favor if they pay significantly higher than the market going rate. In my experience, such an experiment creates a very unhealthy office politics: people realize that they won't get this kind of compensation anywhere else, and their concern for not getting fired becomes more influential than personal ethics or professionalism.
Why not to embrace any content and just make police raids when someone uploads questionable content?
Is censorship solves anything?
police busting down your door for uploading a video kind of is censorship.
But yeah, I'm now starting to see why more and more people are just wanting to go to the censor and police raid system. A lot of this stuff people are uploading just doesn't belong in a civilized society. What started off as maybe just content on making non violent jokes about blacks or gays has morphed into showing children being disemboweled and videos of little old black or jewish ladies being gunned down in their place of worship. It's just gone too far.
Probably just have to file it under:
"This is why we can't have nice things"
> Under the policy, the [animal cruelty] video was allowed to remain on Facebook. A manager told him that by leaving the video online, authorities would be able to catch the perpetrators. But as the weeks went on, the video continued to reappear in his queue, and Speagle realized that police were unlikely to look into the case.
Of everything, that seems like an easy problem to solve.
The balance between protecting privacy and making abuses public is pretty nuanced and doesn't lend itself to one-bit thinking.
NDAs and non-competes have their uses. It's when they become part of the default boilerplate that everyone signs to get a job that the problems start.
I'm not sure exactly how it would work, and perhaps it wouldn't work in practice, but I imagine it might involve paying the (former) employee a certain sum every month for their continued cooperation, and the employer would reserve the right to unilaterally cancel the arrangement: it would be "discretionary" or whatever. So the employee has a motive to cooperate (unless they're terminally ill ...) but there's nothing to "enforce".
I think Non-competes should be limited to while youre actually working there
> engineers ($150k/year, benefits, upward career trajectory) and
> content moderators ($30k/year, no benefits, likely going to acquire mental illnesses).
If Facebook could get away with paying engineers $30K/year, no benefits, believe me, they would.
I find hard to believe humans are so sensitive that some visual stimulation is going to give them mental illnesses. We evolved in an environment where violence was part of real life and a real threat to our own well being, seeing it on a screen will elicit emotions we may not be used to in modern life, but causing mental illness is a stretch.
This just doesn't work in today's world. If it did, Equifax wouldn't be thriving the way it still does today.
"This, then, is our answer. We have no words to waste on you. When you reach out your vaunted strong hands for our palaces and purpled ease, we will show you what strength is. In roar of shell and shrapnel and in whine of machine-guns will our answer be couched. We will grind you revolutionists down under our heel, and we shall walk upon your faces. The world is ours, we are its lords, and ours it shall remain. As for the host of labor, it has been in the dirt since history began, and I read history aright. And in the dirt it shall remain so long as I and mine and those that come after us have the power. There is the word. It is the king of words--Power. Not God, not Mammon, but Power. Pour it over your tongue till it tingles with it. Power."
Their making a lot of money, can they not at least make working conditions decent? Too much to ask?
I used to casually browse a subreddit called watch people die. In 2016 I watched a video on that subreddit that gave me ptsd. At that point in time I had watched probably hundreds of intensely graphic videos, and the total pieces of intensely violent media I had consumed probably numbered in the thousands. I had been into it since 2008. I did it because of morbid curiosity.
At first I scrolled through the comments and noticed something very unusual: very emphatic comments warning people that videos can give you ptsd. Most videos have comments where people talk about how “I couldn’t even finish it” or whatever, so I brushed it off. After watching the video I immediately knew something was wrong. My body felt strange. My mind was in a state of hyper-tension or vigilance. It’s very difficult to describe. I also noticed that my libido would come and go in waves. I would go from not feeling any sexual feelings to being more horny than I’ve ever been in my life. I knew that something deep inside of me had been deeply affected. I went to sleep without much trouble. When I woke up I went into the bathroom to brush my teeth. I felt something coming over me. A sensation of panic. It surprised me because it was out of the blue and I’ve never felt something like that before. I then entered a full blown panic attack, which rocked me so hard that I fled the bathroom and threw myself on the couch. It passed, but I was drowning in anxiety and a sensation of doom. At this point I knew that I may have permanently fucked myself. I was scared. but I still had to go to work. I spent the next few weeks forcing myself through each workday while being suffocated by an overwhelming sensation of doom, anxiety and panic. It was the toughest thing I’ve ever done. But I got through it.
I noticed many things that I later learned are indicative of ptsd: having your eyes lock up, feeling ready to fly off the handle at the slightest provocation, an intense desire to subdue the anxiety with alcohol and drugs. I would walk down the street like an insane person, ready to rage on anyone who even looked at me wrong, and I had no history of anything like this.
I never had bad nightmares or trouble getting to or staying asleep, so I think I had some kind of light beer ptsd. But it was hell on earth. Everything I had heard about veterans losing their jobs and killing themselves all of a sudden made so much sense. Take it from me: ptsd is one of the worst things that can happen to you. And I didn’t even have full blown ptsd.
I got help from a few therapists, and they informed me that if my symptoms persisted more than a month or something, I would technically have ptsd. Other than that, the therapists were basically of no help whatsoever. The symptoms lasted well beyond three months.
As time went on, the symptoms got better. They seem to have stabilized now. If I’m distracted, I feel normal. But if my mind is idle then my thoughts always go back to it and with those thoughts comes the anxiety. Long drives can be uncomfortable. I’m at a state now where I’m in the clear: the symptoms are weak enough that they don’t threaten my ability to work and bathe and etc. and my ability to recognize and cope with the symptoms has increased a lot too. But it still bothers me sometimes and I am keeping my eye out for breakthrough treatments. Sgb and mdma look promising.
A thought can either be in your mind or not. When I’m feeling symptoms, it’s almost like the memories are somewhere in my mind, lurking. But other times they aren’t around. It’s like they have a life of their own. It’s something you don’t have control over.
The best coping mechanism I’ve found so far is meditation sort of. I think that part of ptsd is that your mind is fighting to block the memories and their emotional consequences. So when I feel symptoms I let my mind be open to any and all thoughts or memories. I totally relinquish control of my own thoughts and whatever comes into my mind, I allow it to come and then I watch it pass on. Opening the mind and simply observing the thoughts. This dramatically reduces the severity of my symptoms and often leads my mind to organically become preoccupied with something else.
It’s strange to think that a video can be so dangerous. But they can be. I was a grizzled veteran of gore videos and I thought surely that if they damaged the mind, I would have noticed a long time ago. Some videos, especially high definition ones, can for sure fuck you up. If you have children, don’t allow them access to the internet unfiltered. I saw this video on Reddit for Christ sake.
I worked tech support for Acer in the 90s and its a similar setup. If they took the time to have “relaxation” rooms, i doubt its as bad as the story makes it out to be.