To Fight Revenge Porn, Facebook Is Asking to See Your Nudes | Hacker News Reader

To Fight Revenge Porn, Facebook Is Asking to See Your Nudes

motherboard.vice.com

52 pointskawera51 comments

Hi, this is a legitimate email from Facebook. Nude photo blackmailing is a growing problem. You may have heard that we, Facebook, are having users upload nude photos to fight revenge porn. Please reply to this email, attaching any nude photos that you want to block Facebook from ever publishing. Please also include a rough idea of the amount of ransom money you would be willing to pay to keep these photos out of public circulation.

This will also be very useful for censorship and trolling, tagging images as your nudes to prevent them from getting shared or having others get banned. That might be why the images are transmitted to Facebook first, to check whether it's actually a nude image of yourself.

One would imagine machine learning is being applied here to recognize some obvious things:

1) Does the face match the user who uploaded the image 2) What is the image of (food, architecture, human, etc) 3) Does the image seem to contain nudity

It's especially tough if they're being truthful that they don't store the image, only the hash. Very tough to dispute if the photo itself can't be reviewed in a dispute to see if it was a meme. The other comment about ML img categorization is probably the best bet.

I really hope this doesn't become reality. Otherwise, you can look forward to the phishing mail which will try to get your nudes. "DANGER! We found potential nude pictures of you! Upload your pictures to verify your identity, find who uploaded them without your consent, and block the pictures from being published. If you do not act the pictures will be published." ...

...So are they deleting the original image afterward? Are you now giving Facebook the right to advertise to you based on your nude images? What happens if someone breaks into your account? Are your nude photos of yourself visible to them? Also trusting a company expressly set up to data mine to help you prevent abuse is pretty much the fox guarding the henhouse. Facebook is meant to share. Let's hope they don't accidentally overshare.

Edit: I'm seeing the researcher saying they aren't storing an image but I'm not seeing where they say they aren't.

Edit 2: They are apparently storing a link to the image on the internet, but aren't storing the image itself, which again leads me to wonder how the hash is being generated. The image has to be uploaded to the servers at some point. Is it then deleted?

Absolute clickbait. Facebook is allowing you to flag pictures you have already shared in messages as something that you don't want to be shared any further. They are not asking you to upload pictures they don't already have, even if according to the article you can do this by starting a messenger conversation with yourself. No idea why this is being upvoted.

What is your source for your assertion that they're not asking you to upload any pictures? That conflicts both with the article linked in this thread's title and with their source: http://www.abc.net.au/news/2017-11-02/facebook-offers-reveng...

The fact, as reported, is that you if you want to block a picture (which a former partner may also have) and you don't have it on your facebook account already, they will tell you that you have to upload the picture to facebook (where they say they will retain only a hash) and then flag it. This is a test program in Australia, as reported.

Even the source on the original article you linked to isn't from Facebook. I can't track down a single citation for anyone from Facebook saying "send pictures to yourself and flag them".

Surely they could generate the hash on your own device..?

Or they can datamine and advertise to you based on tatoos, sexual orentation, visbile brand names, etc. Pop the mic on and get some audio at the same time to futher personalize advertisements.

And we know they pop on the mic. Too many times have I seen advertisements based on subjects only spoken, not searched for on any device.

This is happening to me with insane regularity, although I usually notice it in Google searches rather than on Facebook. Recently I met a gentleman for the first time, who told me that he ran an planning / design industry webinar series called the "Performance Network". I'd never heard of this so he suggested I Google for it. I got as far as typing "perf" on my phone before that's what the autocomplete suggested. Freaked the hell out of both of us.

This kind of thing is now happening a few times per week. Are there any non-eavesdropping explanations for how this is possible?

Maybe through gps tracking and seeing that you two were near to eachother?

There was a great Reply All episode about this just last week. It's about facebook, but a lot of the points would apply to google. https://gimletmedia.com/episode/109-facebook-spying/

Short answer, no, they probably aren't listening. But they know enough other information about you that they don't need to.

I'm not sure what would disturb me more. Knowing they sometimes use the mic to listen to me talking (this I could prevent but not having my phone around) or that they already know so much about me that my conversations add little information to what they know about me.

Well, I can say that my experience and my g/f too, have been from conversations we've had at the house we live in and it's usually something random.

If you could show it to others you'd become rich and famous.

Since sha256 or whatever isn't going to cut it, I'd assume they were at least using some sort of perceptual hash concept (https://www.phash.org/) and it's not implemented in Javascript.

The system would be pretty useless otherwise if you had to report the same image multiple times for each time someone recompressed it / took a screenshot of it / a single bit was changed.

The other likelihood is that they aren't just generating a hash for the same reason FaceID isn't just a hash but actually doing more interesting model-building that they only implemented server-side.

Either way, it's not very great UX. But I can imagine that if you're at a place where you need to preemptively block your nudes from imminently transmitting through Facebook, you probably don't care.

It states that they create a hash server side, why couldn't that happen client-side?

> it's not implemented in Javascript.

They can implement it. Hell half the readers on this blog could implement it..

I’m sure they are using PhotoDNA if true, not a normal generic hashing algo. They already use it for matching CP images. It will still match an image even if it’s been edited (cropping/overlay/basic edits/etc).

Is it possible that they're breaking the law here, by soliciting inappropriate images from any users who are minors?

Isn't that what Anthony Weiner is going to jail for?

that is a criminal offense and you can skip straight to police

Is soliciting inappropriate images from minors still illegal, if it's made impossible for employees to see them and the images are not stored?

Wtf? They could easily hash the image on your device without transmitting the image. And of course it doesn't apply to pictures taken by someone other than you. For those it would need to use machine learning to detect your face + nudity.

Although hashing on the device might make it easier for someone to work out how to doctor the image so it doesn't create a match

You only need to make a one pixel change (or just use a lossy compression format) before a hash such as SHA256 or MD5 returns a false negative. You need to create a perception hash of the image. I still think that could be done on-device, but who knows what kind of machine learning/object recognition stuff they're doing on the server side.

as many have said there perceptual hashing also exist (opposed to cryptographic hashings) like phash of photoDNA

At the risk of asking a technically dumb question (I understand the basics of hashing), would there be any technical impediment to creating an app that hashes the file itself on the handset, then uploads to FB?

A perceptual hash set for scale/rotation/crop/colour invariant search? no. But that's presumably not all Facebook is doing and that is worrying. There isn't really any reason for them to have direct access to the unblinded data - all they need to be able to do is ask "does this image description sound familiar? here app on phone, here's a more detailed extract of the questionable image and here's the reference... are they the same?". There is no reason for Facebook to request these images for unhashed upload unless they have an ulterior motive or are lazy.

If they're just hashing the image then blocking images with identical hashes, sounds like a good way for "activist users" to block arbitrary images (perhaps even ads—wouldn't that be fun!)

> Marines United exemplified a part of Facebook that is difficult for it to moderate: secret groups.

Why are secret groups any more difficult than any other kind of group for FB to moderate? FB has access to their own data.

Because there will be no outsiders who flag inappropriate content

Because Facebook doesn't want to pay people to do the work of directly policing them, and instead relies on community reports.

I wonder in what form they store what the article calls a hash. Certainly no simple file hash - I guess it's some kind of feature extraction that lets them match the image against others?

Facebook probably is even just reusing the same systems that flag child porn or other illegal images. One of their better and more user-friendly ideas for sure (if you are locked into their ecosystem anyway, you might as well use their surveillance expertise to control the content you spread there.)

Probably PhotoDNA or similar. https://en.m.wikipedia.org/wiki/PhotoDNA

That is, a hash that survives resizes, cropping, etc.

come on , this isn't complicated, don't over think it-- a woman (or man) meets someone and sends a nude; regrets the next morning; fb im the image to themselves and flags it their partner uploads it, a fb employee sees a nude that matched a hash, the only reason they see it is the hash matched, you're trusting fb not to do anything but hash the image and if your partner alters it maybe the hash doesn't match, but typically the upload is the same and when it's clear it's revenge porn the account is banned-- it'll probably catch a boatload of bags of dicks if people trust fb, but maybe they shouldn't

This doesn't make any sense. You shouldn't need to send Facebook explicit photos of yourself. All they should need is personally identifiable information - your face, tattoos, moles, etc. Women should be able to use bikini shots and men should be able to use swim trunk shots. From that they can scan photos and find your likeness in nude shots.

Nothing good can come from this approach Facebook is taking.

It's supposed to use a hash of the image, so they technically need the exact same image. Doesn't seem like they're using machine learning.

it shouldnt make sense, every BOFH would love nothing better than to sit around looking at your nudes

I assume machine learning doesn't work well enough when the photo gets shared to friends of friends and facebook has to compare the bikini shots against 10,000 other users

Sadly, "Not the Onion"

And how does a simple modification such as a watermark not break the hash?

It's a lossy hash function. https://www.phash.org or photodna are both starting points.

    [message deleted]

Probably not. I assume they pay thousands of Southeast Asian "independent contractors" a few dollars a day to identify dick pics.

It sounds stupid... Storing the hashes? How many permutations of that photo will be protected? Surely the attacker can tweak the brightness or crop the edges.

I would assume they're using the same sort of techniques that image search (e.g. TinEye) use, rather than literally just taking the SHA1 of the image file.

Microsoft has somewhat similar technology called PhotoDNA.

How does PhotoDNA work?

PhotoDNA technology converts images into a common black-and-white format and uniform size, then divides the image into squares and assigns a numerical value that represents the unique shading found within each square. Together, those numerical values represent the "PhotoDNA signature" or "hash" of an image, which can then be compared against signatures of other images. While the technology cannot be used to identify a person or object in an image, nor can it be used to recreate an image, it can be used to find copies of a given image with incredible accuracy and at scale across the 1.8 billion images shared online every day, even when the images themselves have been altered.

https://www.microsoft.com/en-us/PhotoDNA/FAQ?3158ae95-fce7-4...

Facebook are using (or was using PhotoDna.) dunno if they still do. https://m.huffpost.com/us/entry/864695

They do, for policing known CP images.

Reply on news.ycombinator.com