Senior engineers should not be con-men.
What you decide and do now, affects what you can and must decide next.
Except unlike in chess we don't really have the exact rules of the game anywhere. Still we must make choices which affect what moves can be chosen later.
It is also like building a house, once you decide how to build the ground-floor and what it stands on, that greatly affects how the floors above that can be built (not too heavy, not too light etc.).
It is often difficult to understand all the consequences of a design decision, what can be done and chosen after that, until you actually implement that design.
It happens all the time of course. But I wonder if it is a bit of a trivial concept. Of course what was done before will affect what is the optimal choice now.
But I agree it is good to keep it in mind -- when you are making the current decisions. If I decide this way now, what is the cost of my available options in the future? Pretty much the way chess-players evaluate their possible moves.
While I am sympathetic to the concept of reasoning under uncertainty, the way you've framed it is hardly engineering anymore.
Design projects come with a spectrum of data requirements, and each has its own cost function for improved accuracy. For example, knowing exactly how long the bridge you need to build matters greatly, and the cost of surveying should be pretty low. In contrast, measuring the soil at the site you build on top of has a variety[1] of options with varying costs. You can't ever be 100 percent certain that your sampling was good enough, but the samples more you have the more confident you can be. Depending on the project and design, you may not need the most expensive and most accurate option, but you probably don't want to just rely on priors built through "experience."
Translating this to the world of software, we have a variety of data collection strategies. A/B tests, canaries, software instrumentation, tracing, offsite monitoring, f-scores, beta tests, market surveys, and more. The best thing you can do as an engineer making an uninformed choice is to design the system to collect the data you are missing and to make it easy to change your design later. Amazon calls these "two way doors" versus "one way doors"; if your decision was wrong you can go back and make a different choice. As an example, if you are designing a caching system, spending a bit of time to generalize the API will allow you to swap between policies as you collect data about cache hit rates. If you don't, the cost of changing your mind goes up as more things depend on the nuances of your API.
Rather valorize making uninformed decisions, I'd prefer to valorize flexibility and continuous data collection. Seen this way, the old adage "There's never enough time to do it right, but always plenty of time to do it over" is a coherent philosophy.
[1]: https://en.wikipedia.org/wiki/Geotechnical_investigation#Soi...
... sooo there is no distinction between a senior engineer and a non-senior engineer? you've really said nothing in this comment.
senior engineers have seen enough things to know what the right solution is in many cases, without having to take a bunch of time to collect evidence. that's what makes them senior, not just 'good'.
When dealing with uncertainty (nearly always) it's generally easier to identify and rule out what 'wrong' solutions are. There are often multiple obviously 'wrong' options (no, we should not keep a user's password in plaintext, even if the goal is to make it easy to recover a lost password), but picking a 'right' one... from a whittled down list, it may often come down to familiarity or convention rather than an arbitrary "this is the only 100% correct solution".
1. Identify every project dimension with risks. The big one: project -> use -> user -> reward -> costumer fit? (Where user and customer may, or may not be the same.)
Price cost economics? Technical complexity? Technical experience shortfalls? Project resource availability? Solution resource efficiency? Safety? Third party dependencies?
2. Identify each significant risk in each risk dimension.
3. Identify the simplest question (or two) whose answer will reduce or eliminate each risk.
4. If a risk can’t be mitigated by answering a single question (or two), maybe it is really a combination of risks, or risk dimensions. If so break them out, repeat.
4. Craft the simplest research task to resolve each risk question.
—
This sequence is a special case of focusing energy on upstream tasks, before investing in downstream tasks.
Or as I like to say: move slow to move fast.
—
A meta risk is loops in risk/work dependencies.
Classic example: Stakeholder wants to see something working before you resolve risks of getting something working.
Identify the simplest, sparsest form of a loop is one way forward.
Alternatively, break the loop. Identify alternate ways to satisfy the stakeholder. Find a way to remove dependency on the stake holder (constructively).
This is what the previous commenter referred to as "making a decision without all the evidence". If the senior engineer turns out to be right, and the hypothesis proves true, the experiment becomes the system. If the hypothesis turns out false, the system might still be workable, otherwise it's refactored into some new experiment.
The senior engineer is better at making that initial guess as the correctness of a hypothesis. Just like a senior scientist.
So you need to instead have a series of features that are architecturally solid, have KPIs to measure success, and be willing to throw out working production code that doesn't move metrics enough to warrant their complexity.
Engineering is all about what you do when you don't have perfect information.
This is why real engineers have things called "safety margins".