How We Use Deep Learning to Classify Business Photos at Yelp

I had to do a task like this recently, but I cheated by using the Clarifai.com API. Given a picture, it will return keywords it thinks describe that picture. That might be objects (building, people, apple) or the setting (outdoors, night).

So instead of dealing with images, I was dealing with image tags, which allowed me to build a naive Bayesian classifier to determine which of two categories an image fit most closely. Sadly, while I got very few false negatives for one category, I got about 50% false positives on the other, and haven't been able to improve on this yet.