I am in no way in favor of staff-inmate relationships, but in most relationships the power dynamic is uneven. For example, in most marriages with children the breadwinner has significantly more power, but that does not make sex an assault.
Taking this charitably, I would agree with you that in general, sex might still be consensual even if there is some degree of subjective power imbalance between two people. rockstar-fan. politician-staffmember.
However, this does not mean that all power-imbalanced sex is consensual. teacher-underagestudent. undertaker-corpse.
(Strange things can happen when you mix up "there exists" and "for all"...)
* The amount of power imbalance. A husband or wife can always get divorced, a staffer can quit (though we shouldn't trivialize this and it's exactly why sexual harassment laws exist), and a rockstar can go home. It's not, relatively, nearly as big a power delta as the one between a prisoner and those who imprison him. Hard to think of a bigger power difference, actually.
* The kind of power. The power being wielded by prison staff is violent coercive power. They can throw the prisoner into solitary or just beat him (or, if the staffer isn't actually a guard, they have access to people who are). In fact, that coercive power is what keeps the prisoner in prison in the first place.
Between those two things, it's really hard to see how a staff/inmate relationship can be consensual. "You want to do this, right? You know that being in a position to hurt you is literally my job, right? Oh good."
edit: Bring a response instead of a down-vote. This community's weak point is when discussion is put to the wayside in favor of a quick and easy up or down vote. More to the point: Eff off anonymous down-voters, a prison guard is not a bread winner in a sexual assault relationship. This is just a really bad analogy that shouldn't be given support.