Or, to put it another way, if two people wish to trade, it must be in both of their interests. If I, as a westerner, go to a third world country and buy a bunch of stuff that from my perspective is cheap but from their perspective is expensive, we both end up happy. What's so terrible about that?
(Of course where it falls down is where labor is cheaper because of poor worker protections and exploitative practices, in which case it constitutes an end-run around the labor laws of the richer party. But they are the ones that should be upset, not the poorer party, whose labor laws would be terrible regardless of exploitation)