But archaeological remains aren't really necessarily an artifact of the culture that lives there now. Sometimes there is a continuous cultural tradition that claims them, and sometimes not. The Mississippian mounds of the midwestern US aren't an artifact of the U.S. of A., for example—they were in danger of being looted by Americans more than anything, in the 19th century certainly.
And much as an international militia for the defense of archaeological remains is a bad idea, I think the OP was talking more about ISIS blowing up the Temple of Baalshamin in Palmyra or the Taliban blowing up Buddhist statues in Afghanistan. The locals are likely descended from the people who built those structures, but we do all lose something. These religious structures were not shibboleths, but public structures into which people poured their creativity, their personality, their values, their traditions to communicate them within and beyond their community. I am not a Christian, but I am moved by the Sistine Chapel—this was Michaelangelo's goal. I am moved by the Hagia Sophia, by the Ishtar Gate.
If the Romans decide to blow up the Vatican, or the Canadians invade Missouri and carry off the artifacts of Cahokia, it's a loss to more than the residents of Rome and St. Louis. Even if (as with Palmyra &c.) it's locals who object most strongly and are the most injured.