I'm not keen to rebut it at much length because my previous comment already leaves the wrong impression: my point is less about the morality of the past few hundred years than the stupidity of the narrative we built around it.
If I did expand on things, I'd list off the usual examples (Conquistadors and gold, Chinese opium trade, slaughter of native Americans, ad nauseum).
I don't know how to make those points without sounding like I'm appealing to emotion. The actual point is that - though the Steven Pinkers and Hans Roslings might disagree - people subject to those conditions clearly are worse-off than before (and than today, compared to the nations who plundered them).
To the original point of the thread, the colonial powers attained such obscene wealth (ie: by stealing gold, oil, farming cotton with slave labor, etc) that even the poorest among us, until the past decade, was rich compared to the rest of the world.
What changed my view about the world is thinking about the economics.
Take the US, for example. In the popular imagination, the success of America is all about entrepreneurial spirit, etc etc. But the other aspect is... four million square miles of (in a manner of speaking) undeveloped land! Entrepreneurial spirit is great, but literally having more natural resources than you know what to do with (didn't get around to developing California till the 1900s, for example) seems at least as important.
Given the past several centuries were a free-for-all for nations with empires, it doesn't surprise me that the colonized societies, whatever their inherent strengths and weaknesses, wound up in bad shape.