Also, see things like the DeVISE paper that replace the 1-hot encoding output on the 1000 imagenet paper with word vectors so the model can generalize beyond 1000 classes:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.466...