Nothing you said refutes either of those claims. The Ottoman Empire controlled Greece at the time the Elgin Marbles were taken and it's unclear that they granted permission to remove them. They probably didn't so they were likely stolen from the Ottomans at the time. Further, the Ottomans were decidedly not a Western nation. Greece's history does not imply they are part of the West either. In a literal sense they are not, being part of the Eastern Roman Empire.
Meanwhile, repatriation of artefacts taken from Asia/Africa is a more modern movement that has coincided with anti-colonialism. So although the debate about repatriating the Elgin Marbles has been going on for ~200 years, in my opinion, it is being lumped together with the anti-colonial movements and even fits into that pattern. If you don't believe there is a movement to repatriate artefacts taken during colonial times from non-western nations see the link above or do a search.
There is no movement to return loot or spoils of war to western beneficiaries. So it seems pretty clear to me that anti-colonalism motivates the current claims which is a political ideology. If it was strictly about the ethics of looting, you might see people advocating for a return of items to churches and western countries but no one does that.