There's a limited supply of living former presidents. Far fewer than the number of institutions looking to up their prestige by getting a former president to give a speech. How much do you think it should cost to get a former president to give a talk?
Here's another argument. Non-political celebrities also have very high speaking fees (or the equivalent). Yet obviously these celebrities aren't being tacitly bribed. What is your explanation for that?