What test are you using to determine which rights are valid and which aren't?
You point out some differences, but who's to say those mean one is a right and one is not?
You could say something similar about abortion, for instance. Throughout most of history abortions were not available and certainly not in a safe form. But now they are recognized as a right.