Hopefully this can help you.
(Disclosure: I work at Microsoft but not on PhotoDNA.)
I'm happy that Microsoft is providing this as a free service. It's going to be a lot less painful for me to use it than to figure out how to run my own (or in this case, figure out how to even get it).
Somebody please just give me access to the PhotoDNA code, the hashes, and a little funding. I'll make an API anybody can use for this. It's ridiculous how hard it is to do this. It's still easier for people to get spam IP lists than to see if CP is being uploaded to their servers. You can't just have it available for Facebook and Google or it doesn't work, you need to make it available to everybody in an easy, simple way.
Seriously, if you are connected with this at all or want to fund this work please email me, I am more than happy to work on improving this: kyle@neocities.org.
edit: I should clarify not child porn, but personal photos such as instagram and facebook which are private/semi-private and then are posted to public forums.
Really though it's not too hard to whip something up yourself. I did it for a bunch of those 'booru' sites (roughly 3 million images) like this:
- Find image hashing library (I used https://github.com/JohannesBuchner/imagehash but there's a nice series of articles here http://www.hackerfactor.com/blog/?/archives/432-Looks-Like-I... if you want to implement your own)
- Build a database of image hashes using said library
- Use an algorithm that allows you to lookup hashes by distance. In the case of hamming distance (used by many image hashes) you can just throw them in MySQL. You could also use any of the nearest neighbours search algorithms like k Nearest Neighbours or locality sensitive hashing (you'd want one of these for larger datasets)
What are they afraid of? That "pedophile hackers" will be able to reverse the hashes and get the images?
Authenticating access to the service is desirable for many obvious reasons.
Help me out with the obviousness please? Are those reasons more important than deleting child pornography from the web?
There are a lot more legitimate uses for a public repository of CP hashes along with free software for verifying them locally. Not only entrepreneurs and online community operators who don't want the stuff on display, but also users of poorly moderated online communities who don't want the stuff in their browser caches.
I am curious how one goes about developing a service like this without having to see child porn itself. Is there a database somewhere with known hashes? I'm assuming there would have to be along with a way to generate hashes yourself so you could test as I couldn't imagine running automated unit testing with real child porn.