> I mean, Antarctica, or the bottom of the sea, or the peak of Mount Everest all wipe the floor with Mars on every metric you can think of
The pressure at the bottom of the sea would probably make buildings safer on Mars than at the bottom of the sea, so there is one metric where Mars wins. On Mars, you've only got to deal with a pressure difference between outside and inside of -1 bar, or smaller if you don't use normal Earth sea level pressure.
At the bottom of the sea, its 1 bar for every 10 meters depth.
A major hull failure on Mars, or even in a vacuum, leading to explosive decompression is not in and of itself fatal. You quickly lose consciousness, but you have up to maybe 30ish seconds of usable consciousness to try to save yourself by getting an oxygen mask on or getting through a working airtight door to a non-breached area. It won't be comfortable, but the skin makes a decent spacesuit. You'll have problems like capillaries bursting in your eyes, but short term these things are annoyances rather than threats to your life.
A major hull failure deep underwater is a lot lot worse. Deep enough, and you'll be killed quickly by the water pressure itself, and if not, by the collapse of the building as it is crushed by the pressure.
> but really, if we get wiped out it's going to be self-inflicted [...]
Are you dismissing things like large asteroid strikes or natural pandemics as too unlikely to bother with, or would you count those as self-inflicted because we didn't put enough resources into defending against them?
> [...] and compared with the difficulty of setting up a viable colony on Mars, the difficulty of sending a few nukes over there is negligible
Or perhaps you are just considering things that we cause? However, even if we just consider things we cause, Mars would be safe from many of them, even many deliberately inflicted. Yes, if we do ourselves in with nuclear war someone might toss a few to Mars, too, especially if there are Mars colonies that are allied with one of the sides in the conflict on Earth.
But what if we do ourselves in by something like a terrorist or doomsday cult release of an engineered virus? The perpetrators probably won't have the resources to load up a rocket with it and send it to Mars.