What do other companies do with people who have to evaluate content like this?
edit: Sorry, it's an article about how other companies do content moderation, but not a huge focus on how they accommodate the workers who do it.
http://www.nytimes.com/2010/07/19/technology/19screen.html
"Flagged videos are then sent for manual review by YouTube-employed content moderators who, because of the nature of the work, are given only yearlong contracts and access to counseling services, according to Victoria Grand, a YouTube spokeswoman."
Also, I know this is going to be an unpopular opinion but, it's this kind of problem that puts forward the strongest case for increased online surveillance. I understand it's unfair that everyone will be punished for the actions of a few, but I'd rather be under surveillance than have more damage to the personal lives of these content moderators. Maybe we just aren't yet mature enough as a species for the fully open web we used to have.
I don't know what the solution to this will be. Probably a perpetual cat-and-mouse game of image processing algorithms trying to spot graphic/illegal content and black hats finding more and more novel and sophisticated ways to defeat those programs.
The long term solution has to revolve around education and re-examining societal values (the darker side of human behaviour has its roots in more socially accepted forms of human activity). The short to medium term solution is to target those producing and consuming the content, but this is fraught with its own difficulties. In any case, targetted surveillance and streamlining the related court cases is something we should probably be doing more of for now.
re-examining societal values (the darker side of human behaviour has its roots in more socially accepted forms of human activity)
No matter what societal values are, the minor person-to-person variance will always produce a distribution curve of personal standards clumped around society's "normal".
Which means there will always be a small number of people willing to view and send around content that is two standard deviations more "graphic" than what is acceptable in society at large.
It doesn't matter what the absolute value of 2σ is tomorrow. It will be just as offputting as 2σ content is today.
The intent to kill/maim/damage is something that has been condemned for centuries. Exceptions are made for war by those who seek power, but the solders who come out of the other end as psychologically scarred pay the price for such follies.
As for people with fringe interests, it all comes back to the golden rule. Leaving aside the murky waters of 'taking offense', any actions that impact the physical wellbeing of another should be open to public scrutiny. If I rape someone you know, that's not an act that you have to tolerate under the umbrella of free expression.
We executed criminals in public spectacles to be enjoyed by the whole family. Also, blood sports.
It's actually a marvel of modern society that for most people in the Western world exposure to the more gruesome parts of human nature is voluntary and confined to videos and pictures on the Internet.
Even if you could prevent all illegal sources of such material -- stop all violent crime, sexual abuse and murder -- how does that prevent legal ones? There are entire websites on the Internet filled with pictures from violent accidents or disturbing results of medical procedures that many might find emotionally scarring. Not to mention what we do to animals we use as food.
Also, ultimately what may or may not be traumatic to an individual can very greatly. And a lot of things that might be traumatic to many are necessary to display in some contexts (e.g. medical professions or studying historical events). In some cases witness accounts alone may be horrible enough to traumatise.
I don't like slippery slope arguments but I think this train of thought is flawed enough that there isn't even a slope, it's a straight drop all the way down from the onset.
The difference is in intent.
If I view a horrible car crash that was an accident, the images may be graphic and unsettling at first glance, but I can brush it off.
However, if I saw a video where bodies were mutilated to the same degree, but was caused by someone doing it for their own kicks, that's much more disturbing.
The mentally taxing nature of implementing private censorship is a reason to implement more private censorship?
I... um... don't quite follow.
I... um... don't quite follow."
I suspect you're misunderstanding on purpose.
The answer is not more private censorship, the answer is empowering law enforcement to intervene in the lives of those that have caused the issues in the first place.
If you're not prepared to go after those that are involved in the degrading behaviour, what do you propose? Stick your head in the sand and hope it all goes away?
It sounds to me like you are proposing that we solve the problem of individual companies having to hire some people to look at bad things by having the government hire a whole bunch of people to look at bad things.
No, you've misunderstood.
I'm suggesting that in order to cut down on the depraved content being produced, you have to go after the producers. Increased online surveillance makes it easier for law enforcement to track down the producers (and consumers) of this content, as well as providing stronger evidence for their conviction.
In other words, rather than dealing with the problem on the surface, which is what the content moderators working for online companies are having to do, go after the problems at the source. Cut down on the levels of dodgy content being produced.
Do you understand what I mean now?
While the best headline grabbing portion of such content moderation (child abuse) do fall into that intersection, it's a faulty assumption to construct your already-dubious reasoning that better surveillance will improve the live of the content moderators.
There's no need for overlap. You see video footage of rape, child abuse, animal abuse, beheadings, etc... that's enough to take someone to trial. One video is enough.
>"While the best headline grabbing portion of such content moderation (child abuse) do fall into that intersection, it's a faulty assumption to construct your already-dubious reasoning that better surveillance will improve the live of the content moderators."
It's not rocket science, it's quite simple. If you stop the ability of the producers of morally bankrupt content to produce the content, there will be less new content to remove. If you disagree, explain your logic.