I think there is a huge societal weight that is hard to escape because most sexual liaisons are also social relationships.
We date. We get married. The people we are romantically attached to in a public way get judged by others and that judgement impacts us.
The only way to separate your actual sexual preferences from your desire for social approval is to get involved secretly, out of the public eye. And that's inherently problematic.
How do you arrange such secrecy? Will it come back to bite you later? Can you successfully transition from secret love affair to public association with this person?
The plot for the movie Legally Blonde is rooted in family interference in Elle's boyfriend's love life. She's gorgeous and they are in love but she's not what his family wants for him.
There are countless other such examples in both fiction and the real world. Hollywood even coined a term for marriages intended to cover up socially disapproved preferences for LGBTQ stars during a certain era: the lavender marriage.
The world often interferes significantly with whom we can marry, which means it interferes significantly with whom we can openly be sexually involved with.
It's a quite difficult thing to escape, even for people who simply cannot make those expectations work for them (like gay men being forced to marry women and pretend to like sex with women, a still fairly common occurrence in the world). If your sexual orientation and preferences make those social expectations not overly onerous, the path of least resistance is to just go along to get along, at least publicly and superficially to some degree.
But I suspect it still does enormous harm in ways that go largely unrecognized.