Electronic voting is a problem looking for a problem. You need voter verifiable ballots with a high degree of tamper resistance. As far as trustworthiness goes it's hard to complete with ballot boxes and adversarial observers.
I think the solution is to crowd-source verifying the vote. If every ballot were given a unique number or code, and when the ballot was counted each ballot with who it was counted for as available for download on the web, third parties could verify that the numbers were counted correctly. And each voter now has an anonymous number that he can use to look at how the vote was counted and verify that it was counted correctly. If it wasn't, he complains. The political parties would be happy to urge their loyal constituents to check that their vote was correctly counted and to sue should they have a sufficient number of members claim theirs was wrong.
It wouldn't just be that their vote would be revealed, though, it would be that then the bad actor could contest the vote, claiming that they were the voter and their vote was counted wrong.
But if people were in the habit of destroying their unique code immediately after the results are posted, then they at least have plausible deniability. "No, I burned that piece of paper yesterday. Can't remember what the numbers were."
"Vote for this guy or you're fired" type stuff
But I think I would rather have a system where you need to intimidate and coerce people en masse in order to change the vote than a system where you just need to coerce the handful of people who are responsible for counting the vote.
* The candidates themselves
* Journalists
* TV cameras
* Party representatives
* Electoral officials
* Anybody else who has a good reason to be there
The volunteers doing the counting are not going to risk a long stretch in prison with such a high chance of being caught.
Hidden vote tampering of the pencil-under-the-fingernail variety is a solved problem here in Australia. Each vote count is done by an employee of the electoral commission. They're watched by volunteer scrutineers (plural), provided by the major parties, with each scrutineer looking to maximise their own party's response and distrusting all other parties. You effectively have two opposing meatspace inspectors willing to challenge anything that looks shady.
Paper voting isn't perfect, but it's far, far better than any electronic voting system that the general public can use.
EDIT: to clarify, with any electronic medium, the only way to confirm that your vote was counted as cast is to cryptographically sign it in some way with a key that's under your control. Even us techies have problems doing that, and the general public could never be expected to do it, especially the socially disadvantaged members. Not even security conference attendees keep up to speed with their cryptographic signing, so how can we expect mere mortals to do it?
With paper votes, you confirm that your vote goes into the box as you've marked it. You can then opt to be a scrutineer and watch that box travel to the counting area, and confirm the votes are counted correctly. And has the benefit of having the record independently verified if required.
Any form of voting which has a "just trust us on this" stage is not good enough.
Although some people really want to get the electronics back so it's an ongoing battle of educating people (even most hackers and others in the security industry that did not think a lot about voting in particular).
I've heard that pencils are sometimes used to prevent an attacker from substituting pens at the polling booth, with modified pens containing disappearing ink, but that seems like a problem better controlled by using new, easily identifiable pens each election.
As for serialising the ballot-sheets, that enables a third party to check that a person they coerced into voting a certain way actually did as instructed, which undermines the value of a secret ballot.
I don't see much downside to this sort of system, there is a paper ballot for any checks that are needed or desired, and the vote tally is readily available when the election ends.
Plus, it seems to me the process is not particularly tamper resistant during the ballot tallying phase, when compared with what is possible using cryptographic techniques.
(If you're so inclined, you can also just get your name marked off and walk out if you want to donkey vote, which is the equivalent to a protest/blank vote. Or draw penises on the voting paper).
If you don't vote, you get a nice letter in a few months from the Australian Electoral Commission with a fine of about $100 or so for not voting. Voting is also on a Saturday so most have the day off, and there are early polling stations open for a while prior to the day so if you can't make it on the day you can do it then.
I think our voting system is good but it has flaws. I recall reading an article a few years ago about somebody who showed up to vote (i.e. Got his name marked off) at multiple polling stations on election day just because he could. He received a fine for doing it but all of his votes were counted.
Edit - googled it apparently you can be charged with fraud for multiple voting or for voting under a false name (up to six months prison - so slightly more than a slap on the wrist and a fine).
I should mention that a donkey vote is not a equivalent to a blank vote. A donkey vote is when you number the candidates from 1 to x from top to bottom. You actually give your vote to whoever happens to be at the top.
The loss of the senate ballots and subsequent post-election re-run of the entire state - which had a different outcome - was a massive failure of Australian electoral process.