So, you'd be forced to reveal your bitvote address to me under threat of violence. Once you vote, I just check that you voted for me. And if you didn't...
A real cryptographic voting system goes to great lengths to prevent this scenario. In such a system, a voter can prove to himself that his vote counted toward the proper party, but crucially the voter cannot prove such to anyone else. This prevents coercion.
Of course, real cryptographic voting systems have a bunch of other nice properties, but this is a really major one that most people overlook.
Example of a cryptographic voting system: Helios (https://vote.heliosvoting.org/). Also see the tech docs (http://documentation.heliosvoting.org/verification-specs/hel...). Helios is an online system, so it's not ready for prime-time voting --- it would need to be adapted into an in-person scheme, which cryptographers have already created --- but it does demonstrate the concepts involved.
This is an area of active research, BTW.