There is very little that physicists have lingering unease about when it comes to (ab)using maths to get their way with the physical world.
Examples abound, and have actually led to many a new avenue in math.
Distributions (as in the Dirac distribution) are a very good example of this (IIRC convolution didn't have an identity and they needed one, and they just made up "functions" that were allowed to have infinite values as long as they had finite sums or something along those lines).