There are plenty of layers at which it's misogynistic, but the most obvious one perhaps is that it wholly centers around the man in the relationship. Why must the wife be the source of the husband's unhappiness ("nagging wife" trope)? Why does he get to be the great philosopher if they're both unhappy?

You can say "it's just a saying, it would be equally true with the roles reversed" -- but then, why aren't they?

Sure, on the misogyny scale, it's pretty mild, but sayings like this that implicitly reinforce the male-centered world we live in are in some ways the most insidious.