Alternate theory, both are true.

Western societies took advantage of multiple other societies to plunder their treasures. Those same societies didn't have the infrastructure and/or care to preserve these things themselves.

Sometimes two things can be true.