To an extent, but I think there is still a counter example that gets to why this is such a subtle distinction (computable vs "reducible to a mathematical procedure").
Consider an "analog" computer that is in principle noise-free (i.e. elementary operations like addition on this hardware are performed by adding up two analog quantities, e.g. voltages or currents, and these quantities are infinitely smooth, not discreticized/digital, and not disturbed by hardware imperfections and noise). I can create a model of such a device in theory and if I use only the axioms of classical physics (which we now know are wrong), then such a device will be capable of computing and representing numbers that can not be expressed by a Turing machine (i.e. it would be not just faster than a Turing machine or a quantum computer, it will be capable of things that a Turing machine can not ever do).
This is a device that one can imagine existing, and to quote you it is "reducible to mathematical procedure with definite outcomes". It just happens to be a mathematical procedure we find to be rather unreasonable for our universe (because of unavoidable classical thermal noise and non-classical quantum effects that cause discretization).
Some physicists and computer scientists (me included) like to take this as a starting point, as a reason for why certain theories of physics are probably wrong. I personally find this to be a very elegant, powerful, and subtle approach, up there with Noether's theorem or the second law of thermodynamics. "A theory of physics can not be true if it permits the construction of a computing device that can solve the halting problem" seems pretty much on par with "A theory of physics can not be true if it disobeys conservation laws / causality / locality / entropy considerations". All of this with the caveat that one should not be dogmatic about these vague statements, rather just take them as a general guideline.