Speaking as a pure mathematician: It's really not.
When I say this stuff is pointless, that it doesn't affect ordinary mathematics, I'm not just talking applied math; I'm talking, well, all of ordinary mathematics, pretty much. Number theory, combinatorics... every now and then a question comes up whose answer depends on the continuum hypothesis admittedly but it's seriously rare (and on those few occasions when it has happened, I think often someone comes along and says that this means that you were really asking the wrong question in the first palce).
I mean, OK, there are people such as Harvey Friedman who work out arithmetic consequences of such assumptions. But the statements they come up with are seriously contrived, not the sort of thing ordinary mathematicians care about, even if the statements no longer involve infinities.
Hell I work with infinities sometimes and I don't care about this stuff! Of course, I'm not a proper logician or set theorist. But doing computations with ordinals and well partial orders is fun... and the answers you get never end up depending on this sort of stuff. :P