Facebook flooded with 'sextortion' and revenge porn, files reveal | Hacker News Reader

Facebook flooded with 'sextortion' and revenge porn, files reveal

theguardian.com

87 pointsdaenney75 comments

Open articleView on HN

Previous discussion: https://news.ycombinator.com/item?id=14388102

I had a female friend last month whose ex-bf created a fake account and proceeded to set the profile and public photos and videos of them (having sex). He then went ahead and started sending friend requests to her entire family and friends. I felt so bad for her - it was awful enough to make her even have suicide thoughts.

What really surprised me was how horrible the FB system to report stuff like this is. It is basically non-existant. If this thing takes 1 month to be reviewed, that is 1 (eternal) month where the damage can be irreversible.

what is worse is that FB has posts about how they are "fighting" this without having any instructions or obvious process defined to report this. So you "report inappropriate content" and the report goes to a black hole while the perpetrator continues with their shenanigans. They dont even need to create a new acct, because they can keep going without the offending account getting blocked.

Something else is that the non-profit foundations that exist to fight this (the ones that will send legal take-down notices) are US-centric. This means that if you are from a latin american country and you face this (as was this particular case), you are screwed.

Yes obviously the lesson is not to take pics like that in the first place, but sadly a lot of the victims may not be that computer savvy or mature enough to realize what a mistake those things can be.

This isn't about FB solving the problem, it's about FB announcing the existence of their own internal bureaucracies. That is, they won't solve the problem, but they'll create an email address specifically for the problem.

This guy is in the videos as well, right? Could she somehow strike back harder and demolish his reputation? Tell his friends and family, and his employer, what he did to her? Who the hell would send a video of themselves having sex with someone to that person's family?

Unfortunately, in a lot of cultures and environments today (where culture is not only your global, national, or regional culture, but also the culture that you choose to surround yourself with), this kind of thing is much more devastating for women than for men.

I'm not saying that the existence of porn is supposed to hurt his reputation. I'm saying the fact that he would violate somebody in this way should hurt his reputation. The porn is just evidence of his doing this. I'm in no way trying to diminish this lady's situation. On the contrary, it makes me angry and I would want to see this guy see some serious consequences for it.

> This guy is in the videos as well, right?

I suppose that would depend on how well the guy knows about digital editing of video.

...for that matter, if you wanted to go the opposite way, and you had the right software (and a few other skills) - one could seriously put that guy in a very compromising situation by more digital editing...

You were downvoted... but you make a point that I (as a woman) would have no problems following.

I would call / message / text his mom. His sister. His boss. His dad. The new girlfriend.

Not sure what the norm is, but I reported a fake account using my wife's picture (just a single, regular portrait) and it was removed within 5 minutes.

A month would be ridiculous. I wonder if that happens often?

I report spam & fake accounts all the time and get responses back before the next day.

I regularly get friend requests from scantily clad teenagers in a city I no longer live in, whose profiles are full of links to porn sites. I report them, and a few weeks later get back a message saying 'we reviewed your report and found nothing objectionable'. There isn't even a report category that seems accurate - I usually pick 'fake account'.

There must be some real dullards working on the mod team.

> Yes obviously the lesson is not to take pics like that in the first place, but sadly a lot of the victims may not be that computer savvy or mature enough to realize what a mistake those things can be.

In a lot of cases pics like that are not even taken voluntarily but unknowingly.

could you elaborate on that? Honestly curious.

EDIT: about the issue, not the materials in question!

I presume the parent means where one of the participants sets up a hidden camera - for whatever reason (revenge, blackmail, self-gratification, monetary gain, etc).

Yes, something like that. Or when the victim is wasted or deliberately drugged.

> "54,000 potential cases... and 33 of the cases reviewed involved children."

That's 0.06% of reviewed cases, which is frankly less than I would have thought. I suspect that child abusers are a more savvy lot than the average chucklehead who does something like this.

I am sure there is a ton of stuff going on in the developing world were nobody is used to reporting anything.

Edit: Happy to see someone is leaking this stuff given how many flip flops we had to sit threw before someone admitted fake news was a thing.

I'd imagine that the low hanging fruits are picked off by law enforcement pretty quickly.

While others drag on for decades with establishment involvement - the U.K. being the current but not first example.

"savvy" = not using Facebook.

It's disappointing, but not super surprising.

Shit like this is why I'd never want to operate a site with a large user base that can freely upload content. Sure you could probably afford the servers for awhile with minimal monetization, but could you pay for even a half-dozen policy experts to moderate the content to a point where you don't send your largest potential audiences running screaming for the hills?

I am surprised their system for catching bad content is only user reports/flags.

While revenge porn and stuff is bad, there are a number of known child related incidents.

So if a private group of scumbags, are posting stuff to a private group, noone will know unless someone within reports.

Who needs the darkweb when u can have a private Facebook group?

Edit: as per replies below, they do use image recognition aswell for I guess known content lists

I suspect they sign up to the IWF lists for filtering.

International Weightlifting Federation?

https://www.iwf.org.uk/become-a-member/services-for-members/...

https://www.iwf.org.uk/become-a-member/services-for-members/...

They also use image recognition to detect nudity and previously classified child porn.

No, they use Magic Number among a bunch of various tricks. You can't really call it image recognition, more like 'data-hash comparison.' To train true image recognition would require a LARGE subset of stored illegal pornography.

Why? Just use two separate datasets, one of images of adults and children, to learn age, the other of porn and non-porn, to learn what's "nudity" or "sex".

Knowledge transfer training is really hard.

If you can do this (with different categories, for instance, determining what is nudity, then determining black people, to automatically classify porn with black actors automatically), it would be quite impressive. Results have nearly 5% error on CIFAR-10, which is full of clearly distinct categories - planes versus boats.

> it would be quite impressive

Not only impressive, but likely very profitable.

> Results have nearly 5% error on CIFAR-10

Any idea what the error rate is for human classification on CIFAR-10 images?

Can you explain (just a broad overview) why that wouldn't work? I mean, it's not like black porn actors are categorically different from black people in general... in particular, if you have an algorithm that can locate the face, you could just pass the face of the porn actor to the other algorithm to determine age/race/whatever...

What do you mean by "Magic Number" in this context?

It's in the file format

https://en.m.wikipedia.org/wiki/List_of_file_signatures

Also check out "libmagic"

I mean if companies like Google train ML models on millions/billions of user's private data without looking at it, then surely it would be possible to train this in an ethical manner for ethical purposes. And via some Gov collaboration on the legal side

You are confusing image recognition with classification. They use recognition, similar to reverse image search services. Not really 'data-hash comparison', but rather 'image-hash', since it may see through re-compression and other minor modifications.

Defining "bad content" is difficult. I don't think we're nearly advanced enough avoid human intervention. Using user reports works well.

Here's an example: if you go on YouTube you can find harmless videos of parents bathing their babies, and I doubt anyone would call that child pornography. Same goes for something like a birth video, even though it shows a vagina. How do you detect the differences?

I think it'd be an improvement if Facebook were more open with their policies and allowed nudity, even if it required flagging content as mature. To get around this people end up placing tiny black bars in any of the "offensive" parts. But how does that seriously change anything?

I'd say it's pretty reasonable to request users flag content. Detecting it programmatically is currently impossible, so what other choices do you have if you want to allow users to upload content? Should only multi-billion dollar companies be able to have user-uploaded content? How would you handle this issue if you were building a small media sharing site?

Some problems run deeper than their individual manifestations such as these for which one may propose techie solutions -- here, sex ed is the only permanent solution. It's time to find ways to explain to boys that sex is an ordinary way to have fun and/or connect romantically between adult people, rather than a source or arena of power or status as it is endlessly portrayed in media.

Sex and power/status are inextricably linked for much of society.

Society implies culture thus construct whereas this feature is observed in much (all?) of animal kingdom as well.

And that's the problem.

To boys? really?

Have you ever read the cover of Cosmo or seen clothing with blatant cleavage?

The power/control dynamic extends in all directions & I have no doubt that some of this revenge porn extends in the other direction..

>and the challenges for moderators, who say they are overwhelmed by the volume of work, which means they often have “just 10 seconds” to make a decision.

Maybe Facebook should just hire more moderators? I know they'd love to totally relegate this to some awesome NLP algorithm but we're not there yet. Facebook makes more than enough money to bring on some more employees (and clean up their guidelines).

EDIT: oops, had two articles from the guardian on FB up and referenced the wrong one.

"Maybe Facebook should just hire more moderators? I know they'd love to totally relegate this to some awesome NLP algorithm but we're not there yet. Facebook makes more than enough money to bring on some more employees (and clean up their guidelines)."

There are systems that can detect porn pretty reliably. Helping moderators spot any porn more quickly will also let them spot revenge porn more quickly. Any correct or false positives can be fed back into detection systems. It might still take more moderators, though. Facebook needs more anyway given the size of user base and money available for moderation.

> Maybe Facebook should just hire more moderators

They are doing exactly that this year

Facebook disabling 14,000 accounts in a single month because of that kind of terrible behavior does show this is a serious problem. But it's still a very rare problem. Given Facebook has 1.19 billion monthly users, that's on the order of 1 in 100,000 accounts.

That seems high to me. We should strive for this to be so rare that it's news-worthy when it happens at all.

Well, it's really just an extension of gossip, blame, scandal and shaming that has been around forever. There would need to be a major social change first. Otherwise, even the perfect tech solution would just force this bullshit to move somewhere else.

Somebody check my math:

Facebook has 1.13 billion dailies, and 1,800 potential daily revenge porn incidents: 1 in 627,777. Involving children, about 1 per day.. 1 in 1.13 billion?

Is flooded the right word to describe that? It actually seems pretty low to me.

So The Guardian puts this article up just as May's government is promising to tighten government restrictions on social media. That was convenient.

The Guardian isn't a right-wing paper, it's left of center, and I can't remember when it last endorsed the tories. You'd be far more likely to see that kind of thing from The Telegraph or The Daily Mail.

There's another explanation much closer to home which doesn't involve Theresa May: The Guardian, like many other newspapers, is under intense pressure from Facebook which is eroding their direct traffic in favor of traffic coming from FB. They literally compete against each other for clicks and ad traffic.

Sure, but the timing is suspicious. The editors at The Guardian aren't above giving a Tory proposal a little push if it's in their interest. Besides, Labour is proposing essentially the same thing.

They should have a verification process for accounts beyond verifying an email address. Verify identity, it's way too easy for people to be impersonated, there's a whole TV show dedicated to this...

You need a phone number to register a new Facebook account...

They're free with Google Voice

I don't think Google Voice will work for that nor will a Trello number. I saw someone complaining on HN that their landline didn't even work.

Burner phone?

Yea, someone tried doing this whole 'sextortion' thing to me on FB.

I laughed at them. I used to work in porn, you think you're showing people something they haven't seen (let alone probably want to see again since I 'retired' [AKA got married?])

Sad that shameless people such as myself seem to be immune to this kind of thing. Perhaps everyone should have a career in porn just to eliminate this issue.

Give me a break.

Reply on news.ycombinator.com