In general, the two biggest financial benefits have to do with reduced health (and other) insurance rates, and not paying taxes on all or some of gifts, inheritance, or other wealth transfers between spouses (similar to those between parents and children).

The largest financial penalty in many countries is that spouses will pay more income-related taxes in dual-income families vs. if they were single, due to how progressive taxation bands are set up.

The benefits of government recognition for most people who marry aren’t financial - they’re to do with parental, health, estate, citizenship/immigration-related, and other rights that largely can’t fully be reliably created by contracts.

A lot of them don't apply to single people, like the laws about inheritance or social security or medical decision making. The laws specifically refer to a spouse who just doesn't exist for single people (but which could apply equally well to a male or female spouse).

In some cases, a single person could specify somebody in particular, but it's more flexible. This is a common default case that applies widely, and those who choose not to get married can apply the same overrides that others can.

In the case of social security survivor benefits, there's a definite asymmetry where married people get a benefit that single people don't. It covers a common case for two people who have their finances so tightly linked that the salary really is joint between the two of them, such that the social security payment really is intended to cover both.

There should perhaps be a similar rule for people who have a similar arrangement without the benefit of legal marriage, but it wouldn't be easy to recreate the entire legal apparatus that joins the two, which is what makes that survivorship benefit reasonable. Effectively you'd need to create a state of civil union, and now you're just playing semantics with the difference between civil union and marriage.