Have you looked at the APIs for any of these libraries? I think there is a perception that these are all for NN only because NN is the hot topic that everyone is jumping on noe; but tf for example is a general matrix manipulation library with a bunch of amazing, extra NN stuff shipped alongside it.