Reverse-engineering facial recognition | Hacker News Reader

Reverse-engineering facial recognition

sterlingcrispin.com

Since the site seems to be down, I'll mention that there was an article on Medium covering this artists' work a few days ago: https://medium.com/matter/this-is-what-your-face-looks-like-...

Seems like what he did was take Umass' Labeled Faces in the Wild (LFW) dataset, combine all the faces together and generate a composite rendering. Hardly a representation of "What your face looks like to Facebook".

Articles like this do little to advance the discussion of the real concerns around facial recognition other than scare folks into an Orwellian vision of the future a la Hollywood and Minority Report.

In addition to the work that the NTIA is doing around setting privacy standards for facial recognition applications (http://www.ntia.doc.gov/other-publication/2014/privacy-multi...) there are a lot of companies using the technology for good, but it is very difficult to get as much press and attention for those types of positive stories.

For example, my own company, (http://www.kairos.com) is working with some very brave and passionate guys at HelpingFaceless.com in India who are using facial recognition to help combat the growing problem of child trafficking and enslavement in India.

See more about their story here: http://social.yourstory.com/2014/07/helping-faceless/

That's not what I did.

I used a genetic algorithm with facial detection/recognition algorithms as fitness functions

Recognition algorithms like eigenfaces, fischerfaces will reproduce one individual.

More general detection algorithms like YEF realtime object detection as a fitness function will result in more general faces which represent some of the learned features from the LFW database. I'm not just combining images, there's lots of machine learning involved.

Recheck my site in two weeks or so I'll be posting my MS thesis about the subject with more technical details.

(Wrote this on my phone excuse the brevity)

You might want to summarize that at the top of your post, then, if you're going to submit it to HN. When it contains such nuggets as:

These masks are shadows of human beings as seen by the minds-eye of the machine-organism. These DATA-MASKS are animistic deities brought out of the algorithmic-spirit-world of the machine and into our material world, ready to tell us their secrets, or warn us of what’s to come.

It's not surprising that HN viewers are confused about what you actually did.

In any case, seems like a cool project, now that I know what it is :-).

Thanks, but btw someone else submitted this to HN I hadn't planned on fully publishing the project until a few weeks from now when my thesis is due but it was posted to medium.com so I quickly put some documentation together

There are more technical details at the bottom of that page in the form of diagrams

For some reason, I thought that green usernames (like yours) meant you were the poster. Sorry! My mistake; you can't control when someone else posts your work.

Green usernames indicate new, or "green," accounts.

"Think of the children!" Whether it's for backdoors to your data in the cloud or facial recognition, protecting the children is ALWAYS what governments claim they need the tech for. Their track record is terrible, almost always using it for control and surveillance.

I want to see cameras on every police officer in every city, before even considering more giving them more surveillance technology.

I don't think you read the article very far. Yes, he started with an averaged face (which then matched nothing, by the way). Then he added (I'll simplify for you) random dots to it until it matches something. Thus it is a reverse lookup from the way facial recognition is usually used. Usually you give it a face and see if it has a match. In this case you basically feed it random face-like data until you trigger it, then you have some insight into what it is looking for.

The fact that the results look very little like human faces do provide insight most people don't have into how the algorithms work. The algorithms basically work by measuring distances between dots in the end anyway, distance between eyes, etc., and that's very unlike how humans recognize faces which has more depth to it.

Bit of self promotion, but related due to the dataset:

http://deeplearning4j.org/facial-reconstruction-tutorial.htm...

It's pretty neat seeing what a "general" face looks like.

As a researcher at a face recognition company, this is interesting, although the fact that algorithms change pretty frequently makes me think the work is quite ephemeral. Also I'm wondering if he's heard of liveness detection.

Reminds me of this:

http://www.zachblas.info/projects/facial-weaponization-suite...

Which is also pretty interesting.

His project is pure poetics/politics and has no direct relationship to computer vision

Its as much anti facial recognition as a paper bag

That video is incredibly grating and slow to the point. And the mask just seems to be random faces mixed together, but they did it in such a poor way it just produced random garbage.

This guy seems to be optimizing his mask to fool facial recognition.

This is really cool. This work, along with the previous work of reverse engineering handwritting recognizers is important for us to better understand what our tools are actually doing. A lot of what happens inside these ML algorithms is not terribly well understood, even by the creators, this work gives us a really good insight into what the mind of a machine is actually thinking. When I was a teacher, I used to try to make sure students understood things going both ways. This is very important work in this respect.

For people struggling to understand, it's a little like having a tool that uses automatically generated regular expressions to find things in text, except you don't know what regex the computer is actually using. For my name it could be as simply as "bane" or something crazy that a human would never think up, say b[a-z]+e. It matches my name, but it's interesting to explore what else it matches: "barserphursnatche" is also perfectly acceptable. Remember, you don't know what the computer is using to recognize things, so you have to simply probe: randomly generate strings, use some kind of genetic algorithm etc.

In the end, you probably won't find b[a-z]+e, but you'll find some expression with a sizable intersection like b[a-y][b-z]*e. Once you have that you can use the expression as a generator and build a list of all the possible names it will match with something like Microsoft's Rex [1]

What we find is that lots of the stuff the computer likely matches are not my name at all, or even look like a name (e.g. "baghuadsoadshasdguoasdughasdgaodsguafdghaufogafuhgafduoe") and it gives us insight into what it's doing.

I think why I like this work is that it makes me think of a sci-fi short story:

- A boy encounters an alien civilization

- The boy befriends a robot from the civilization

- They go off on some adventures together

- During a scene of doubt in their relationship, during a stressful final adventure, the boy asks the robot what he thinks of him. The robot 3d-prints off one of these heads and gives it to the boy.

1 - http://research.microsoft.com/en-us/downloads/7f1d87be-f6d9-...

> Computer systems built to represent human identities have contained with them many ontological assumptions about what it is to be an individual and what personal identity is. These systems define the human as a “what” ie: that which can be measured, not as a “who” ie: our inner self.

> If the state of the art in computer science can produce a unique feature that describes an individual as such, what good does that do the individual if this knowledge is only leveraged against them?

> If private citizens personal information, social graphs, and communications are being analyzed then the results should be made available to said persons to empower rather than enslave them. This attitude has become popular in personal fitness but not in communications, biometric identity, or social networks.

> “We see the world, not as it is, but as we are” - Talmud

I love this. Really fantastic work.

Thank you please recheck my site in a few weeks for my full thesis

This does pose the question whether these mask work in real life. I bet their software uses the fact that they can control the lightning and picture quality. Creating a direct copy of a face seems easier for real life purposes anyway.

Other than demonstrating the flaws of current facial recognition algorithms there does not seem to be any use for this.

Anyway, pretty cool use of evolutionary algorithms and nice pictures.

I was hoping that an article about "reverse engineering facial recognition, facial detection, and image correlation techniques in order to reveal how they represent human identity" would have at least one mention of eigenfaces:

http://en.wikipedia.org/wiki/Eigenface

I know its not in plain sight but if you RTFA there's a diagram at the bottom that mentions them.

http://sterlingcrispin.com/data-masks/map_of_attempts.png

Cool tangential topic: http://www.onformative.com/lab/googlefaces/

...Reminds of being up at night as a kid, and seeing faces in shadows, patterns on the wall, vvhatever

Cache: http://webcache.googleusercontent.com/search?q=cache:http://...

Edit: Original seems up again

Reminds me of this: https://www.youtube.com/watch?v=qS5HWBNvf9U

Sounds interesting. But the link received "HN hug of death".

Reply on news.ycombinator.com