A Senior Engineer's Check-List (2019)

What you decide and do now, affects what you can and must decide next.

Except unlike in chess we don't really have the exact rules of the game anywhere. Still we must make choices which affect what moves can be chosen later.

It is also like building a house, once you decide how to build the ground-floor and what it stands on, that greatly affects how the floors above that can be built (not too heavy, not too light etc.).

It is often difficult to understand all the consequences of a design decision, what can be done and chosen after that, until you actually implement that design.

While I am sympathetic to the concept of reasoning under uncertainty, the way you've framed it is hardly engineering anymore.

Design projects come with a spectrum of data requirements, and each has its own cost function for improved accuracy. For example, knowing exactly how long the bridge you need to build matters greatly, and the cost of surveying should be pretty low. In contrast, measuring the soil at the site you build on top of has a variety[1] of options with varying costs. You can't ever be 100 percent certain that your sampling was good enough, but the samples more you have the more confident you can be. Depending on the project and design, you may not need the most expensive and most accurate option, but you probably don't want to just rely on priors built through "experience."

Translating this to the world of software, we have a variety of data collection strategies. A/B tests, canaries, software instrumentation, tracing, offsite monitoring, f-scores, beta tests, market surveys, and more. The best thing you can do as an engineer making an uninformed choice is to design the system to collect the data you are missing and to make it easy to change your design later. Amazon calls these "two way doors" versus "one way doors"; if your decision was wrong you can go back and make a different choice. As an example, if you are designing a caching system, spending a bit of time to generalize the API will allow you to swap between policies as you collect data about cache hit rates. If you don't, the cost of changing your mind goes up as more things depend on the nuances of your API.

Rather valorize making uninformed decisions, I'd prefer to valorize flexibility and continuous data collection. Seen this way, the old adage "There's never enough time to do it right, but always plenty of time to do it over" is a coherent philosophy.

[1]: https://en.wikipedia.org/wiki/Geotechnical_investigation#Soi...

When dealing with uncertainty (nearly always) it's generally easier to identify and rule out what 'wrong' solutions are. There are often multiple obviously 'wrong' options (no, we should not keep a user's password in plaintext, even if the goal is to make it easy to recover a lost password), but picking a 'right' one... from a whittled down list, it may often come down to familiarity or convention rather than an arbitrary "this is the only 100% correct solution".

Overall, it just sounds like they plan on making problems and then bringing them to me to resolve. Which, sure, would require 'managing'.

The domain expert (our internal customer) has a badly specified procedure for picking the correct configuration for some sub-assembly. It depends on a number of different parameters some of which are continuous variables and some are discrete. Which continuous variables are relevant depends on which combination of the discrete values are chosen; for instance choosing nominally equivalent parts from different suppliers can mean having to express the specification in different terms. So the customer asks for a user interface that will make it easier for the end user (not the domain expert, a less qualified person) to explore the database and a few simple tools to do some calculations so that they can decide which parts to order.

While writing up my proposed solution I realize that a lot of what the domain expert has expressed as matters of fine professional judgement look suspiciously like they are in fact rule bound from the point of view of the actual end user. When I quiz the domain expert about this they tell me that they though that the task was too complicated to automate. When I finally extract from them the actual rules I see that the actual end user has no real freedom of choice, that the rules change about once a decade, and that all that is requires is a small form to gather the two pieces of information that are not already in the system and a fairly small set of database queries. All the rules can be encoded easily in a couple of small database tables that the domain expert can update when the conditions change.

I point out that this is actually less work, can be delivered sooner, and is most likely more reliable. The domain expert agrees and we are both happy.

Over the decades that I spent writing applications I frequently found that customers failed to ask for what they really wanted because of mistaken beliefs about the relative difficulty of various kinds of computing tasks. Instead of asking for what they needed they asked for what they thought we could provide. This frequently caused more delay than if they had simply asked for the best solution from their point of view.

Often the customer asks for what they want rather than what they need. Very often what they want is rooted in solutions or partial solutions that they have seen or implemented themselves. They think that they simply need a better version of their own home grown tool when in fact their tool is the way it is simply because of the limitations of the tools they had to hand, usually Excel programmed by someone doing it instead of the engineering that they are actually competent in. Using the appropriate solution, often an SQL database and a very simple front end, simplifies the task dramatically and makes the tool both easier to use and much simpler to maintain.