Another way to put it would be that "all physics is either perturbative or non-perturbative", although this one doesn't sound as catchy since it is transparently a tautology ;-)