You don't even need to use "infinity", it starts out as just a variable representing some unknown quantity, then you "round to zero" on output.
I actually collected a bunch of old Infinitesimal calculus math books.
Indeed they are more intuitive, people like Newton and Leibniz invented/discovered calculus by thinking in terms of infinitesimals, but it took time to be made rigorous, in the XX century. By then network effects got we stuck with epsilons and deltas, given that was the approach made rigorous earlier, and broadly adopted, despite being more cumbersome.
There are a few similar ones on IA, e.g.
https://archive.org/details/in.ernet.dli.2015.148501/page/n8...
On that page the 'h' term is the infitesimal, as in
d(x^2) / dx = 2x + h