Election commitee (or whatever it is called in different countries) has unlimited and uncontrolled physical access to those machines and their keys. What stops them from uploading a firmware that would count every N-th vote for candidate A no matter what was voter's choice? How do you detect that?

Unlike electronic voting, paper ballots are much more difficult to manipulate. If a voter has marked a box for candidate X, you cannot change it or ignore it if there are observers. An observer can verify that voting goes according to the rules and votes are counted properly. The most popular way to "hack" paper voting is to organise groups of people and ride them on a bus from one polling station to other so that they can vote multiple times, but it is more difficult to do, and easier to spot than simply replace the firmware in a voting machine.

In the case with an electronic machine, you cannot see what's happening inside.