>and a lot of countries (Australia, Japan for instance) have in fact a successful policy of allowing only a bunch of them

I don't know about Japan, but Australia allows in large numbers of immigrants.