People like e.g. Bjarne Stroustroup or Linus Torvalds are also paid 6-figure speaking fees depending on the venue, and they're far less-known names than Yellen or Clinton. And not because tech conferences are trying to bribe or corrupt their speakers (i hope), but because having a big name really drives attendance.
It probably sometimes happens that speaking fees are paid as a bribe, but that's illegal and also not specific to speaking fees (similar popular schemes are to hire the person as consultant, or to buy something from him for a much higher price than the real value). It also raises the same red flags, when it turns out that the speaking fees were far above market value.
Unless that was the case here (I don't know how much other speakers are paid at Citadel), it seems premature to assume nefarious behavior.