Over engineering is the result when you focus too much on “future” needs.
It also doesn't always have to be a dedicated architect in a proverbial ivory tower. It can be an informal meeting of developers around a whiteboard. Don't get scared by the word "architecture".
IHMO the experience, the pragmatism and the soft skills needed for a successful/productive informal architectural meeting are too rare for this solution to work consistently.
Personally I've abandoned all hope and interest in software architecture, while on paper it makes a lot of sense, in practice (at least in what I do) it just enables way too many people to throw imaginary problems in the mix, distracting everyone from what matters with something that may eventually be a concern once/if a system hits a top-1% level of criticality/scale.
Yes, this happens too easily. It's the crux of Ward Cunningham's original observation on tech debt discussed recently [1]. He basically said: all of you thinking you can use waterfall to figure it all out up front are deluded. By getting started right away, you make mistakes and you avoid working on non-problems. I can fix the mistakes with refactoring but you can't ever get your time back.
Most teams live in his world now. Few do too much up-front design, most suffer from piled up tech debt.
I hope you give architecture another chance. Focus on the abstractions themselves [2] and divorce that from the process and team roles [3].
[1] https://news.ycombinator.com/item?id=40616966#40624446
[2] Software Architecture is a Set of Abstractions, George Fairbanks, IEEE Software July 2023. https://www.computer.org/csdl/magazine/so/2023/04/10176187/1...
[3] JESA section 1.5, https://www.georgefairbanks.com/assets/jesa/Just_Enough_Soft... "Job titles, development processes, and engineering artifacts are separable, so it is important to avoid conflating the job title “architect,” the process of architecting a system, and the engineering artifact that is the software architecture."
Even if you clone us, our gut feel doesn't transfer to our clones. So, we can recognize under- and over-engineering in our guts, but how can we help someone else? Is there a better way than the risk-driven model?
1. Identify and prioritize risks
2. Select and apply a set of techniques
3. Evaluate risk reduction