Rescue and abort options are also much harder. The moon is close enough to easily resupply or rescue people on the surface, mars is much harder.
But if somehow both of these bodies were orbiting around Earth, Mars would be just orders of magnitude more straight forward than Mars, and I think it's relatively likely we'd already have permanent outposts, if not colonies, there. So the mental model of it being viewed as a stepping stone is somewhat misleading. The Moon is hard!
And also I don't think the distance will be that bad. We've already had 374 day ISS stays which is far longer than any possible transit to Mars (though nowhere near as long as a late-stage mission abort would entail) and the overall effects of such a stay were not markedly different than significantly shorter stays on the ISS. So it seems very unlikely that even a late stage emergency abort would be fatal.