I think the important part is that it "learned" to distinguish this concept of cat face without that being the direct intent provided through labeling. Given the set of images, it learned that "concept" which we can then retroactively label to make it useful in the typical human sense by associating it with cats. If we had to compose computer "knowledge" by training it in everything by manually specifying what it was training for, that process would surely be insurmountable in a general sense. Yet this seems to provide the possibility of automating this process.

A separate network of connections between "knowledge" bits could maybe be used to associate related knowledge like you talk about in terms of their relation to the world. This could also probably be formed in a similar manner giving the algorithm the ability to distinguish "important" concepts contained in the data. The thing I find most odd and interesting about this is that the network tends to identify different important concepts than humans do.