Step one: Public list of every single polling location.
Step two: Communication to the public that observers are freely admitted. Post a notice that some extra women will be needed in some areas since some Muslim female voters only identify themselves to women.
Step three: Public voters list that show who in the county is eligible to vote.
Step four: Communicate the polling day well in advance. Hire people from the local community to handle the actual taking of ballots.
Step five: Polling day. Starts off with every observer verifying that the serial numbered ballot box is empty. This happens for every box. Person walks in. Every observer increments the number of total people coming in. Person identifies themselves at the registration table with government ID and as a backup, utility bills or statements of note from public members for the homeless (A priest could verify that a homeless person lives in the area, for example. In some places the homeless do not even need to be verified), every observer crosses that person off their own copy of the list. If the person is a female Muslim, only women observers get to verify the identity of the voter. The voter then gets a piece of paper with the options clearly printed in large print and goes to a private place that has a poster that shows how to fill out the ballot. The voter puts an X by the person they wish to vote for. Folds the ballot and brings it to the ballot box. The observers see that the voter put the vote in the ballot box. Once the box is full, it is sealed with a tape that matches the serial number of the box and is left in open view to observers until it is time to open them and count them.
It is now known that how many votes is in each box, how many people voted, and the total number of votes for each candidate all with open access to Observers from the public (but more usually, the political parties).
How fucking complicated is that? Paper, pens, ballots, observers meanwhile we satisfy the only possible complaint (Covered Muslim women) while never ever having allegations of ballot stuffing. The only allegation that comes every now and then is the homeless vote, but it really doesn't matter in the grand picture.
Do we have other problems? Sure, of course, yeah. But things like this: http://en.wikipedia.org/wiki/Robocall_scandal happen everywhere and if the public were more informed to our process then they wouldn't happen at all. You can always call elections Canada to verify a polling station relocation.
Keep computers out of voting. I want every vote counted by humans. Not just paper ballot receipts held to check in case of allegations of voter fraud. No. Observers and humans do the counting. I don't see the point of public encrypted votes.