Rigorously division by zero is "undefined" in mathematics.
"There are mathematical structures in which [dividing by zero] is defined. [...] However, such structures do not satisfy every ordinary rule of arithmetic (the field axioms)." https://wikipedia.org/wiki/Division_by_zero
You're familiar with the ordinary rules of arithmetic (ORA!) applied to the real numbers. However, there are many number systems that follow different rules.
The rules of arithmetic for floats, wraparound ints, elliptic curves, and asymptotic analysis are all different from ORA! (and each other) but they are useful in their own ways.