Not using a set of axioms written in the original signature, of equations using the three operations and the constant 1. It's certainly possible to come up with a finite description of the true sentences in the theory, but only by extending the signature or using some method of description other than a set of axioms.
Usually when people say "not finitely axiomatizable" they just mean first-order logic. For example, Peano arithmetic is finitely axiomatizable using second-order logic, which allows quantification of predicate variables, which is required for the axiom of induction.