Facebook Adds Face Detection To Photos

I've been working on automatically detecting visually describable attributes in face images. These include everything from coarse attributes such as gender, age, and ethnicity; to more detailed ones such as nose size, eye shape, facial hair; and including some imaging conditions, such as blurriness, lighting, and facial expression. (I think this is what you meant by 'characteristic', right?)

We demonstrated how to train such classifiers and use them for building a face image search engine in this project:

http://www.cs.columbia.edu/CAVE/projects/facesearch/

We demonstrated how to use these attributes to perform face verification ("are these two images of the same person?") in this project:

http://www.cs.columbia.edu/CAVE/projects/faceverification/

Those project pages have descriptions of how everything works, and also links to the actual publications themselves. We've also released two databases that might be useful for training your own classifiers:

http://www.cs.columbia.edu/CAVE/databases/facetracer/

http://www.cs.columbia.edu/CAVE/databases/pubfig/

Now to actually answer your question: we found that the key to training different attribute classifiers is to use different features for each one. The "secret sauce" of our work is a feature selection algorithm that looks at a large pool of possible features (divided into regions of the face to extract features from, what type of features to extract, how to normalize the feature vector, and how to aggregate the normalized values) and picks the most appropriate ones for a given attribute.

Most existing face recognition algorithms typically choose a few low-level features (often things related to image gradients, since this gets rid of lighting variations) and use these to compare face images. However, works such as ours are trying to change this by looking at higher-level attributes in addition to just the low-level features. The big reason for this is that low-level features require very precise alignment between pairs of faces to work well -- a precision not reachable on real world images. Luckily, the level of alignment possible with today's methods is good enough to accurately train our high-level attribute classifiers, and so they end up being more useful for recognition in real world datasets.

Finally, one of the best benchmarks for performance on real-world face recognition is "Labeled Faces in the Wild" (LFW):

http://vis-www.cs.umass.edu/lfw/

That has results showing the performance of the best current approaches as well as links to papers. It's also a nice dataset of images to test on.