My point was that we have accepted GR even though we know it implies a divergence at the center of a black hole, which we know exists. We assume the math is probably mostly right, we do have a huge curvature at that point, but it can't be literally infinite. We assume that if we had a slightly better theory (probably quantum gravity) that would fix the divergence without changing too much else.
By contrast, we couldn't say "the theory that predicts the ultraviolet catastrophe is mostly right, we're just missing a tiny piece to stop the divergence", because we had good experimental evidence that atoms don't collapse almost very quickly at all. So, while the infinity alone was a motivation to look for a better theory, it was the extreme contradiction with experiment that motivated a fundamentally different theory instead of an adjustment.
> But the invariance arguments and procession of mercury would be grounds for taking it seriously. Einstein's argument for the theory depends only weakly on MM.
Looking this up some more, I had a mistaken impression. The independence of the speed of light from the velocity of the source was already relatively well established by 1905 - in fact, this was part of the motivation for Maxwell's equations from 1865. Maxwell's equations had been thoroughly tested and were well accepted, especially because of experiments by Hertz in 1900. However, the dominant concept was still that light is a wave in the luminiferous ether - but several attempts to measure the interaction with this ether had produced null results. Even what we now call the Lorentz factor had been experimentally measured through experiments on the speed of light in a moving medium (then thought to be a drag coefficient between the medium and the ether).
Einstein then came up with the idea that constancy of the speed of light should apply in any reference frame, not just some special frame defined by the ether. That means that not only is the speed of light independent of the speed of the source (which isn't that strange for a wave), but it's also independent of the speed of the observer. This was only experimentally proven much later, in 1932, by the Kennedy-Thorndike experiment.
Still, SR followed from theories that were well proven empirically, and had obvious experiments that could falsify it.
> This view suggests that a physicist in 1915 (or 1925, etc.) should not try to work out properties of gravitational waves, for example, which seems obviously ridiculous to me.
Well, I think here there is a difference between trying to work out consequences of an otherwise well-established theory (such as working out exactly how gravitational waves would look like in GR, or arguing based on Maxwell's equations) and trying to come up with a new theory that modifies existing ones (such as grand unification or some solutions to BHILP).