A big problem is that local governments place a lot of weight on "community input," but the people giving input don't accurately reflect the community. People who show up to meetings tend to be strongly opposed to change and fit into demographic groups that have a lot of free time, i.e. older, richer, and whiter. This causes a lot of positive changes to be either outright vetoed or delayed indefinitely at the whim of NIMBYs.
I don't disagree with you that this happens, but I'm not sure that removing the local democratic infrastructure is the right way to solve this problem.
I'm not a big fan of the practices of a lot of these low-level elected groups (seriously, fuck you Palo Alto representatives), but the principle behind the existence of all the local town and county governance systems (and particularly the heavy reliance on elected people) are things that seem like really good ideas to me, and that I think should exist in more places (maybe they do, I'm sure the commenters here will alert me to such places ;) ).