Amongst other things, this is the 'lump of labor' fallacy - that there are only a certain amount of jobs to go around, and that none of those people from a poor country, when given a proper place to live, learn and work, will actually turn out to be good entrepreneurs, inventors, scientists, engineers or whatever.

Edit and of course you're ignoring that having a failed state next door is probably a fairly negative thing, in that they're great places for drug gangs, terrorists, etc... and end up costing lots of money in any case.