I don't have any problem with an isolated casual sexual encounter. As far as I'm concerned, that's none of my business. However, I absolutely hate living in a culture that glorifies it, to the detriment of loving relationships. I also hate living in a world where the vast majority of available women have such awful attitudes as to be unmarriageable.
Those "puritannical attitudes" exist for a reason. Some people take them way too far, and their application is very unjust (e.g. gendered double standards); but they shouldn't be cast aside as relics of "the dark ages". There are many great things about loving, long-term, monogamous relationships and family formation.