What's wrong with losing access to a giant talent pool of hundreds of millions of people who can move immediately, visa free?

I think it's pretty clear.