The thing is that bank security is at least theoretically possible. Banks and/or governments act as a central sources of trust in such systems and trust for can be derived from this source. I.e. a bank could build the hardware tokens from scratch themselves. With an election the trust model is inverted. I'm not sure there is any practical way to allow the trust sources (every voter) to audit the hardware tokens.