Inspired by the Pinterest paper at KDD on implementing Visual Search, I have created this barebones but functional implementation of a Visual Search server using ~450,000 Female Fashion images crawled from an image aggregation website. The code uses Pool 3 layer features extracted from the Google's latest Inception model using Tensor Flow. The extracted vectors are then indexed using an Approximate Nearest Neighbor implementation from Nearpy. The AMI provided contains both images and pre-computed vectors.

In future I plan to add more images ~2 Million in the same domain. Test various combination of nearest neighbor indexes and multiple vectors per images using some form of a multibox style detector. I will also add a script to launch spot GPU instances via Cloudformation to economically index images using S3 and SQS. I am building a companion iOS swift app however, since Tensor Flow hasn't been ported to iOS yet, its still in development.

https://engineering.pinterest.com/blog/building-scalable-mac...