The orthogonal issues are
1) Non-consensual unwanted advances
2) Curbing avenues for sexual attraction
I call them orthogonal because,
- A person with enough avenues for sexual attraction at workplace or otherwise and choose not to do anything non-consentual and to recognize the people he/she feels attracted towards as equal humans even if they are his/her superiors or juniors.
- A person with few avenues for sexual attraction can choose to be non-consentual and delegitimize the humanity of his/her victims despite the lack of abundance of opportunity.
Mixing up these issues ends up confusing infidelity and sexual harassment. A person who seeks and engages in consensual sex outside his/her marriage is guilty of infidelity but not guilty of sexual harassment. A person who makes unwanted advances towards a married partner is guilty of sexual harassment irrespective of the fact that the victim is a married partner.
Note that I would categorize anything that makes the other person uncomfortable as an unwanted sexual advance. If a person talks about his/her pornography fetishes, but the other isn't interested, it is unwanted. If a person grabs the other other by the waist, while the other does not like it and tries to wriggle free, it is unwanted. If a person asks a subordinate for drinks and the subordinate is unwilling but pressured into accompanying, it is unwanted. Being unwanted is a clear signal for a person to not engage in a certain activity in the future and the victim should not be expected to create a ruckus in order to reject unwanted advances.