I always thought it should be the opposite---paying bribes should be illegal, but accepting them legal. It would be like collecting a reward for turning in the briber.
The crime is this: to (1) participate in bribery and (2) to not be the first party to report it to the authorities.
Immediately that the bribe has occurred, a race initiates between the two parties to be the first to register it with the authorities. The bribe money becomes prize money to the winner, with the loser facing consequences.