If you're talking about renormalization, let me say that as a working physicist, this critique is misguided. In a Wilsonian picture there are no infinities. Lattice field theory, for example, has none.

After my first two QFT classes I found renormalization extremely suspicious and mystical. But then I started doing lattice QCD. My PhD advisor said:

""" Look, you do a simulation on a lattice with some lattice spacing. You adjust the parameters of the simulation to reproduce some physical result.

If you change the lattice spacing but want to get the same physical result, you've got to change the parameters you put into your simulation.

That's renormalization. """

and I was enlightened.

Some QFTs don't have a UV fixed point and this doesn't work. But that's a signal that the theory isn't good all the way into the UV. QED, for example: weak physics comes in before any badness from pure QED has the chance to bite you.

What Physics needs is an infinite, unbounded rug... like a carpet, but it covers the walls and ceiling to loop back onto itself. This avoids discontinuities when the edge of the rug interacts with the vacuum.