Remember: the whole point of bribery is not just for the briber to gain an unfair market advantage. It also lets the bribee unaccountably extract phantom taxes from the wider economy. Furthermore, because participation in this system is illegal, the bribee can further blackmail the briber into paying additional bribes to avoid prosecution.
Naive prosecutions of corruption can, ironically, institutionalize it.