So instead of trusting a few thousands (several thousands, in reality) and a system where you can literally walk in the place where they are counting votes and observe them, you'd rather trust a few tens of people to write the software and audit it? How does that follow?
And this doesn't even take into account the fact that a voting system, as it was mentioned, should be understood by the people voting. Good luck explaining to people what the software and the hardware are doing.