All of which to say, is while you raise an excellent point all the evidence i've seen suggests the two are entirely unrelated projects. If anything increasing globalisation in the long term increased prosperity for everyone involved (just not necessarily by equal amounts) and vastly improved conditions.
If anyone has a counterpoint, by which i mean historical complaints or serious academic analysis, i'm happy to hear. None of this is a moral judgement on the relative evils and merits of empires and Victorian England, which is not the topic, just my opinion of why from a practical standpoint one has very little to do with the other.