I've heard that story a few times (ironically enough) but can't say I've seen a good example. When was over-architecture motivated by an attempt to reduce duplication? Why was it effective in that goal, let alone necessary?
The key is to avoid the temptation to DRY when things are only slightly different and find a balance between reuse and "one function/class should only do one thing."
One of my favorite things as a software engineer is when you see the third example of a thing, it shows you the problem from a different angle, and you can finally see the perfect abstraction that was hiding there the whole time.
My view is over-engineering comes from the innate desire of engineers to understand and master complexity. But all software is a liability, every decision a tradeoff that prunes future possibilities. So really you want to make things as simple as possible to solve the problem at hand as that will give you more optionality on how to evolve later.
The spectrum is [YAGNI ---- DRY]
A little less abstract: designing a UX comes to mind. It's one thing to make something workable for you, but to make it for others is way harder.
A callback to do the processing?
> a flag
Oh.
> Now... several arguments... probably just a few lines that get run for all the types
Yeah, that does tend to be where it leads when new parameters are thought of in terms of requesting special treatment, rather than providing more tools.
Yes, yes, "the complexity has to go somewhere". But it doesn't all have to get heaped into the same pile, or mashed together with the common bits.
Yes the initial HTML looked similar in these few places, and the resultant usage of the abstraction did not look similar.
But it took a very long time reading each place a table existed and quite a bit longer working out how to get it to generate the small amount of HTML you wanted to generate for a new case.
Definitely would have opted for repetition in this particular scenario.