It turns out rule 5 (Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident) is both true but also hard.
Eric Evans' Domain Driven Design is a good book on the topic.
Eric Evans' Domain Driven Design is a good book on the topic.
A key point, though, is that you learn the right domain models/abstractions over time. Refactoring is critical as you gain more insight into the domain. If you’re constantly questioning your modelling of the domain, and refactoring towards a better one, you’ll end up with a great model and thus a clean, understandable, easy to extend/modify system. If you stick with whatever abstractions you chose at the start, when you knew way less about the domain/business problems, you’ll likely end up with poor abstractions, and a code based that’s slow, tedious and error prone to modify.
Convincing the business that it’s worth setting aside time to constantly refactor towards better domain models is often the hardest part, but crucial.
The problem is not the self-evident algorithm, but the delicate implementation (or god forbid, at scale).
Take in 1000 web requests per second. The data is all strictly validated and has about 60 fields a record/req plus dealing with errors.
How does that go from webserver to (rolls dice) kafka to a (rolls dice) cassandra that can be queried accurately and timely? How much does that cost?
Oh, that's not a programmer problem. Except it is. Creating a fantasy niche of describing problems as data vs algorithm is the canonical ivory tower disconnect.
I'm making an argument that the stark reality of what's hard in software development is not the simplistic "Rules of programming", which have limited utility.
The reason the "rules" aren't self evident (or followed), is because we live in the reality of disparate functionality paired with an ever-changing technical landscape. You can't just make a DB KISS abstraction and expect it to hold with all the different repository types like (rolls dice) Athena after using (rolls dice) CockroachDB. There are concerns that are not purely algorithmic vs data structure that are far more influential and important to understand. Even knowing these details and cases, becomes less useful as time goes on and new technologies emerge.
> I am having a hard time understanding what you have written
If you're not interacting with new environments, tooling, and problems, regularly (every year or 2) you don't encounter the real pain which is far more important to your career and your ability to produce functional software. Reading almost every postmortem, the number of lines attributed to "we changed the data structure to O" is dwarfed by "we learned that technology X does Y, so we had to do Z".
This is only incidentally related to distributed systems, which is indicative of a disconnect with the problem described. Of course when you sit around in the same environment for a long time, you can observe and optimize on structure and algorithm, but that's not getting you to market (you're already there) and that's the nature of maintenance...not just being a fire extinguisher who is on call.
It is the responsibility of tech leaders to minimize this (accurate) stereotype. Choose boring technology, and only build your own or choose something exotic when it gives you competitive advantage - because the reality I also see is that 99% of devs aren't working on anything new or unseen in the field. Even at the FAMANG companies, most people I know are working on boring problems.
So when your CTO or architect or whomever buys into the hype for X technology, make a good argument against it by proposing a better solution.