> because "society" decides they are no longer productive.
You have to consider the benchmark. Do people deserve to live in a castle if they aren't productive enough?
Living in a single family home in the middle of NYC requires a whole lot of productivity because you are literally displacing dozens of other people. You have to be as productive as all those people combined to be worthy of replacing them.