I doubt that statement, too.
That said, do you really think that "unrestricted" immigration (excluding border cases of known criminals, etc...) in the US (or Canada, Australia, etc) would not have a substantial negative impact on quality of life? It seems obvious to me that it would, but if you have any good arguments, studies, links to the contrary, feel free to post - I will read them.
Yes, I do believe that unrestricted immigration would not have a substantial negative impact on quality of life for the resident population. And would have an enormous positive impact on the migrant population.
People from the poorer parts of the US are already allowed to migrate to the richer parts. Why stop there? Also as poorer nations joined the Schengen-Area (related to the EU) there have not been qualitify-of-life-decreasing mass-migrations.
You may argue that there can be quality-of-life degradations when the immigrants are not properly integrated. But the US seems to do a pretty good job in this respect. (My home country, Germany, much less so.)
Sorry, I do not have enough time to write down a real argument, but if you are really interested, I can perhaps prepare a blog post or so.