This is not true. The enormous rise in wealth of the world in the 20th/21st centuries has coincided with large increases in personal freedoms as well.

At any rate, I'm not sure a graph from the US starting at 1950 is sufficient for establishing historical trends.

The trend is consistent with other datasets:

https://ourworldindata.org/working-hours