I don't think that secular society posits some kind of natural inalienable objective moral law. I'm sure you know this but many, many people do not believe in objective morality
A "right to be forgotten" seems to mean a conflict with the classic rights of others to remember what they know, and to tell each other what they know.
remember that the holocaust, the great leap forward, gulag, the imprisonment of galileo, daily human sacrifice to the aztec sun god, and witch-burnings in the spanish inquisition were not just legal but legally required
therefore, anyone who condemns one of them is appealing to a moral standard they consider superior to, at least, the legal systems of the countries in question at the time