This makes me feel like neural networks are nothing but glorified hash functions. Well, if you think about it, here we are just optimizing for hash collisions of similar things.
But of course NNs are definitely just functions, in the strict mathematical sense. You could replace one with a large lookup table. The interesting part is the training: how to come up with the function in the first place.