Once we start talking about "a lot less" the options to not work become scarce.
For instance, my estimate is that with the minimum number of millions ($2M) one can support a family while unemployed in the USA and pretty much the rest of the world. With 4% SWR such family will have $80K per year, which is more than the median household income. They will pay little taxes and will qualify for a subsidized ACA health plan.
If you are talking about less than a million, you won't be able to support a family in US. It is possible in a 3rd world countries, but it is not always an option for a lot of people.