How would you characterize the abilities of transfer learning? Not disagreeing, simply curious.
As the article states, the majority of the hidden layers, particularly the early ones, are very general ones that usually work out to identify edges/colours/etc... that generalise very well.
Because you pop-off the last fully-connected layer, transfer learning a network that has been trained on something like imagenet still generalises quite well to something completely different like medical imaging of tissue samples, although distance between the two sets will still be your main limitation.
Before I get ahead of myself, I'll refer you to this paper that tests the degree to which transfer learning is generalisable [0]. They conclude that difference between the two datasets is the main limitation, but that those models still work out better than random features, which I think proves that it's sufficiently generalisable for most tasks.
[0] http://papers.nips.cc/paper/5347-how-transferable-are-featur...