Society is the only thing enforcing the private property laws that allow you to amass stuff in the first place. If society wasn't involved there would be nothing to stop anyone simply taking your stuff. Society enforces such rules because they are conducive to a good society for everyone. It has every right to stop enforcing them at the point at which that ceases to be the case.

> That or we stop pretending we live in a free society.

This is exactly what we should do. There is no such thing as complete freedom. All freedoms come at the price of restricting some other freedom.