That is true, if it succeeded for a while and then failed it could produce a casualty count on the order of Gulf War 1. Not doing it could result in civilisation extinction via nuclear war or asteroid strike. I still favour doing it, but I see how reasonable people could disagree.
I believe Elon said it would take 40-100 years for the population to reach that point, and he is saying that there will be a return mechanism, and a probably an extreme labour shortage which favours a strong bargaining position for both the entrees and the Mars local population. Ideally I'd like to see the return ticket + life support until the next return flight provided as part of the entrance fee to minimise risk, but there is still the problem of enforcing the contract.
[EDIT: I don't consider this a "fair" argument so I left it out, but as an aside, if minimising pain is your goal then mass sterilisation is the optimal strategy IMO -- there is no reason to continue the human race which will certainly produce millions of people in pain -- and if you have another goal which requires human existence to fulfil, then decreasing our extinction probability should be a huge priority]