Except bribes undermine social fairness significantly. Notice that as the clerk presumably knows how much money he should give you for your tax returns there is a strong incentive for him to ask for a bribe that covers a significant share of your tax returns, effectively making the entire tax return process useless for the citizen.

Also, if you think of a slightly different scenario, imagine the office is only allowed to issue X tax returns a month (maybe the money comes in packets), and the bribe is required to cut the line. Now it's really unfair and wrong, and a lot more so for the briber than for the bribee.

You should have in mind that it's not actually that hard to bribe people indirectly. Any place with a bribing culture tends to have people you can hire to unofficially bribe some government employee to do something for you. In Brazil these are called "despachantes" (dispatchers), and even though in the last decade or so they have been getting progressively more useless as the country moves away from a bribing culture, you should keep in mind that a lot of despachantes are legally incorporated and even pay taxes. One can't be blamed for hiring that sort of person.