There are plenty of ways of handling this; in New Zealand, local council rates (like a property tax) are used to pay for infrastructure, and after the city budget is set, each property owner pays in proportion to the value of their property (currently land + capital improvement). This can be separate from the ideally national land value tax, and take into account different infrastructure requirements.

The other reality is that the single family homes in the area which just got a massive boost in infrastructure spending are benefiting from those improvements disproportionately; their land could be providing access for 100 families to those services but instead are only allowing access for 1 family. Their land value will have skyrocketed hugely because of the improvements around them, so they should pay the tax to return to society the land value they gained.

If it doesn't make fiscal sense for them to pay that tax, then they can sell their now hugely expensive piece of land for a tidy profit and live somewhere it makes more sense for them to be, allowing their land to be developed into a 100-family apartment.