Data-Efficient Image Recognition with Contrastive Predictive Coding

“When used as input for non-linear classification with deep neural networks, this representation allows us to use 2–5× less labels than classifiers trained directly on image pixels.”

Interesting paper on an unsupervised pre-training approach using overlapping image fields to massively reduce the amount of labeled data required to train image classification models.

Reminds me a lot of the way word2vec is trained with overlapping word vectors and negative sampling...