Your investment in maintenance and improvement of the structure is, in any in-demand area, a tiny contribution to the appreciation of the value of the property.

By "protect it against drastic change", I'm assuming you're talking about political action. That part of your contribution to the value is literal rent seeking.