He does benefit from the increase in value, because land increases in value due to increased services and amenities. If you put a train station nearby someone's house, their property hugely increases in value, and they benefit from the nearby train (or school, or fire station, or port, or whatever else).

Now, we could certainly defer the payment of the tax until sale of the property for, say, pensioners, as no one wants to be kicking out retirees from their homes.

However, the issue of people losing their homes due to increases in value already exist for a massive section of the population: those who rent. As rents increase, many renters are forced to move. They don't even get to benefit from an increase in land value, from the community they helped build and improve.

A fantastic way to solve this issue would be a land value tax that taxes the economic rent of the land, in conjunction with removing all income and sales tax. This means that land sale prices are low, renters have more income to spend on things besides rent, and people sitting on land don't benefit enormously without returning that value to society.

The tax can be deferred until sale of the property for pensioners, for their primary residence.