It's not just globally more efficient, usually each country benefits (although I wouldn't want to over commit by saying each country always benefits). But the gains from trade are often not distributed evenly within a country, for example manufacturing employees might lose their jobs while the owners have higher profit margins and/or consumers benefit from lower prices. Furthermore, diffuse gains such as lower prices are felt less than acute costs like losing a job.
Time and transition costs are also very important. In some cases transition costs may be very high, e.g. a whole generation of miners might not be able to transition to other types of work such as software development. In the long run the economy as a whole should benefit, but those miners are unlikely to be convinced.