> I'm not against polygamy or polyandry
Pedantic, perhaps, but while its generally acceptable to use "polygamy" to mean "polygyny" when you aren't discussing "polyandry" (because virtually all real-world polygamy is polygyny), if you are explicitly discussing both forms, its should be "polygyny and polyandry", "polygamy" includes both.
> On the other hand, we've long had a tradition of traditional married couples where the man keeps a mistress on the side, usually in secret (though secrets don't tend to last very long).
Its really weird to call monogamous marriages "traditional" when comparing them to the polygynous marriages, since while the latter have fallen out of favor progressively in the last several centuries, they have quite a long tradition pretty much everywhere.
That said, its actually in many places a long tradition for kept mistresses, particularly of powerful men, to be fairly open.