There are some pretty good articles and videos about the subject; some random points:
Puritans, who believed that sex was sinful and suffering was noble.
The likelihood of something happening in real life. Most people will live their whole life without getting into a violent situation, which keeps it a fictional one, but almost all will be confronted with a sexual one. Violence in media will not beget violence in real life, generally speaking (if it did, given how much violence there is in video games and movies, we'd have a Problem), but sex in media will normalize it. Or that's the theory anyway.
Another one: Sex is always happy, a happy relationship, people looking pleased with themselves after the saxophone-music-pan-out smoking a cigarette (to invoke the 80's trope). Violence is almost always bad, the hero in agony, the bad guy coming to a gruesome end (falling off Nakatomi Plaza, into helicopter blades or a jet turbine, getting roundhouse kicked into defeat, etc), etc. It's a punishment, whereas sex in a film is depicted as a reward. Generally speaking.