Allowing polygyny is not the opposite of criminalizing adultery; its quite orthogonal.
Its the opposite of criminalizing polygyny, which is also common in the US, but that's a different issue.
This is certainly what tumblr ideologues who haven't bothered checking scientific facts (behavioral biology) like to believe.
Saudi Arabia criminalizes adultery which is "having an affair". Polygyny means you're legally married which is allowed in Saudi Arabia but having an adulterous relationship is criminal.