Statistics is even worse. A mix of old tricks developed to avoid computations when these were expensive. See [2].
[1] A Radical Approach to Real Analysis https://www.davidbressoud.org/aratra/
[2] The Introductory Statistics Course: A Ptolemaic Curriculum? https://escholarship.org/uc/item/6hb3k0nz
However, if you want to get really serious about that, you'll need that zero plus an infinitessimal be equal to zero. This is impossible in classical logic due to the excluded middle (which forces each number to be either equal to zero or non-zero).
For a more algebraic treatment, and its important applications to automatic differentiation, I'd suggest starting with the relevant wikipedia articles: