Beyond simple statistics and random sampling, everyone voting is important because let's say 1000 people are required to determine the result beyond reasonable uncertainty. Then those 1000 consequently hold too much power and thus can be easily bribed and the result affected far more easily than if everyone voted.

Of course, psychologically, everyone needs to vote to have a say. But beyond even that psychological thing, everyone voting is really a security measure against tampering.