You seem to forget that maths and physics are meant to
describe reality, not
be reality. Any abstract concept (like numbers, forces, shapes, temperatures, energy levels, ...) we use to describe a real thing is different from that thing itself. If you think about it, even 2 isn't "more real" that Pi, because there are no 2 identical things in reality.
When we say "a human hand has 5 fingers", we use the abstract concept of "finger" to make that description. It is abstract because each "finger" is actually unique. It's just a handy approximation we use (pun intended) for descriptive goals.
Basic maths is not reality. Therefore no, irrational numbers do not "fall out of it".
Edit: format