I found an excellent way to avoid premature abstraction and optimization and to write better software in general was to explicitly consider v1.x a throw-away.

Build something expedient that works well enough to deploy in the field, get actual user feedback and system metrics (e.g., where are the actual bottlenecks). Do a few iterations on user feedback and system metrics. NOW, you are much further down the road to a true final spec, and you can use that real information to design the real system to scale up on.

One Test Is Worth A Thousand Opinions.

This plan first tests your ideas against the real world of users, hardware, and data flows, and keeps a lot of technical debt out of the scaling system.

I discovered it a bit by accident, having previously been really big on early abstraction and planning, but sort of having to do this in one startup, and it was a real eye-opener how well it worked.

The code in front of you works today, but will become unfit for purpose and un-salvageable, and we want to ensure that when that inevitable end happens, there is a sane and safe way to systematically chop it out and replace it with something else, something you are not capable of predicting.

There's substantial overlap with general principles like loose-coupling and modularity, but the framing changes how people apply them: Instead of trying to create durable Amazing-Thing which will be used for many years by people amazed at your foresight making it "flexible" and "modular" and "customizable", you focus on creating Inoffensive-Thing which can be easily killed off or dismantled for useful parts.