It's not necessarily about power imbalances - subordinates might express interest in a superior, or a colleague might interest in another colleague, and both situations can be just as problematic as a superior expressing interest in a subordinate.

The problem is simply one person deciding to single out another person for any deviation of expected treatment especially when it has no relation to the social context or work performance/duties.

Hitting on a coworker is potentially as problematic as inviting a coworker to church. Being extra friendly to a coworker or customer of a different gender is just as problematic as being nasty and disrespectful to a coworker or client of a different ethnicity. Every scenario mentioned is grounds for making the person or possibly even those next to the person uncomfortable, leading to harassment claims. Furthermore, the idea of only behaving a certain way if the other party consents is a lack of principle. It potentially jeopardizes the protections a workplace is supposed to provide to all workers and customers. Just because a door is left open does not mean one should enter.