Right, and if those unskilled immigrants don't all find jobs as nannies and nurses (since I won't be the only one keeping them from the most vulnerable), then I'll be taxed to pay for them anyway. That's not "nothing".
If the cultures are very different, and they come in huge numbers that refuse to assimilate, then I suffer dilution of mine. That's not "nothing" either.