It's not about using the system as an "user". It's about the fact that it bases democracy on principles which are way beyond the mathematical capabilities of 99%+ of the electorate!

I mean, if my government decides to use some crypto/blockhain magic, there is no way I can figure out if using hash algorithm H in a method X is valid choice or if it is a backdoor/negligence.

This is an issue which simply does not happen with paper ballots (backdooring these is of course possible, but really expensive especially at scale). The attack surface is easy to reason out, unlike with cryptography.

Maybe it does not matter though (and maybe I am afraid of mathematicians secretly taking over the world, ha!)