yes, this is essentially the Clarifai product: use a CNN (without the softmax) as a feature extractor for images and then do ANN search on them in the embedding space.