This is a big part of why land-value tax sounds like a better idea to me. It wouldn't be immune to complication either, but the scope of required nosiness seems at least less incredibly intrusive.
But most of the time, in the course of getting passed, proposed simplifications become additions instead of replacements. It's depressing.