If you are willing to recognize that society has had an incorrect view of rights in the past, you must recognize that there is a standard beyond society to appeal to.
Unless you want to invoke divine will as that "standard beyond society," humanity and its arbitrary social constructs are all you have available to deal with, and some problems can't be reduced to objective and absolute terms as a result.
You could invoke "nature" but nature doesn't recognize rights. No creature has a natural right to food, safety or property. Nature engages in warfare, cruelty, violence, rape and slavery with abandon, and without even the pretext of morality. Even anarchists wouldn't want to live in a world governed entirely by natural law.
A slave will appeal to society because society has the power. But if the society does not listen to the appeals of the slave, we can still judge that the slave is correct and the society is wrong. And that judgement is not arbitrary.
The same grounds that we use to argue against slavery today. Just because the society around you doesn't believe certain rights are real doesn't mean you can't personally buy into them. "Rights are a societal construct" doesn't mean that nobody has a right to believe different things from society. It just means that our conception of rights derive from societal consensus, not a higher power or objective source, and conversely that there is no higher power or objective source we can invoke to override society's mores.
> If you are willing to recognize that society has had an incorrect view of rights in the past, you must recognize that there is a standard beyond society to appeal to.
No, you can just criticize them using the today's moral standards. You can be sufficiently committed to your moral standards to be willing to impose them on other people without simultaneously believing that those moral standards are a objective property of the natural universe.
Hume's is-ought guillotine is a thing. A normative statement does not magically become an positive fact just because that normative statement is an extremely strongly held (e.g. slavery is bad). The notion of inalienable rights is a rhetorical trick that lazily conflates the normative and the positive.
Unless society doesn't recognize your right to believe differently, no?
The is-ought problem does not rule out the existence of normative facts. It simply states that they can't be proven by positive facts. You seem to be trying to rule out the existence of normative facts by appealing to positive facts.
Having an opinion doesn't make that opinion an objective fact!