Nonvoters would win the Electoral College if that was a thing.
It should be mandatory voting like in Australia instead. You don't cast a ballot, you get fined, and voting day is a mandatory national holiday. If you really don't want to cast a ballot, you cast a blank or invalid one.