NNs can approximate any computable function. Any function can also be approximated by a polynomial. This is all proven math and has been known for a long time. I think of these as different facets of the same thing and each has different tools for computing, analyzing and understanding.