Aren't the primary elections the business of the parties conducting them and not really part of the democratic process at all, but rather a way for parties to find their favorite candidate to put up for the real election? In that case, it doesn't matter for voting rights if a party excludes someone from their own election, does it?