I guess the naive answer is completeness. Every Cauchy sequence converges to a member of the space. For example, quantum mechanics relies on the formalism of Hilbert space, defined as a complete inner product space. This gives us nice things like the spectral theorem for unbounded operators, without which we wouldn't be able to define probability (the Born rule) or time evolution (the operator exponential e^-iHt).
Can you formalize quantum mechanics using computable numbers? I don't actually know, but let's say yes. I assume it's more work with more edge cases, so I would ask the same question: what do you get for the trouble of building a formalism around computable numbers?