I'm suspicious of seductively clever writers like Chesterton. You'll find this quote on his wikipedia page:

> "The whole modern world has divided itself into Conservatives and Progressives. The business of Progressives is to go on making mistakes. The business of the Conservatives is to prevent the mistakes from being corrected."

Having read some of his Christian apologetic essays, I suspected the fence argument was fundamentally motivated by religion, and again from the wiki page, it originated in from his book "The Thing: Why I Am a Catholic".

EDIT: GP said "Progressives tend to be very dismissive of the Chesterton's Fence analogy, because they think change is an unalloyed good." to which parent complained "Why did you need to make this political?". I found Chesterton's own words both relevant and sufficient warrant to broaden the discussion along the lines of the GP.