Timezones, language factors, cultural factors, retention and exclusivity challenges plus basic logistics stuff like payroll and taxes all matter a lot. None of that's insurmountable but it takes a level of effort that can be very high.
And I say this as someone who quite likes working remotely.
I just don't see it disrupting the world in the extreme way it is sometimes portrayed.