I think that's where constitutional "scholars" go wrong. The "rights" do not exist in the absence of a secular social contract.
What they mean is that there's no reasonable, fair social contract they can imagine that lacks those rights. It's perfectly possible for a social contract to lack certain "inalienable" rights, though. All you need is a pure, unconstrained democracy to see that.
Unfortunately, interpreting the Constitution as a living document and reading new rights (like the "right to privacy") into the Bill of Rights and/or the 14th Amendment -- "because of course the founders would ideologically agree with us that individuals have new rights A B and C" -- leaves the door open not only to good interpretations of rights, but bad interpretations of rights dictated by the majority against minorities, or dictated by powerful minorities against a disinterested majority.