Alright, what about laws that only levy income taxes on people making $300k or more? That's eliminates millions of Americans without targeting any protected groups. There's no bulletproof method to allow only some people to vote that wouldn't be abused to disenfranchise others.
Also, surely you don't think a four year old should weigh in on who gets to represent them in Congress? We disallow voting on a minimum age, because children are still developing in all sorts of ways, there is no maximum age that takes away your right to vote when you reach it. You can't equate a minimum age to disenfranchisement.