I think that's the wrong argument to make. Generally speaking, I think it's wrong for others to force you to be charitable (in the very least, a tax break would be nice if it's about charity) but immigration shouldn't be and isn't about charity.

In fact, most studies point to the positive economic impacts of immigration over time (though there are a number of studies that also point to the short term costs).