The Biggest Mistake I See Engineers Make

I am not a software engineer so I am obviously not the target audience here, but I do like reading these types of articles because sometimes they have nuggets of wisdom that can be applied to other fields.

Having said that, I would make the following points to writers who wish to engage outside of the narrow circle of their profession:

* It's nearly 250 words into this piece before we even get a hint that we are talking about software engineering here, and even then it's only because the writer mentions a PR. I know enough to know what a PR is, but a general audience would not. It's not until some 620 words into the article that software is actually mentioned and the general reader can say without a doubt that software engineering is what we are talking about here.

* Along with PR, IC is something that is never defined and this one was new to me (and a little tricky to search, but I now understand that it is an Individual Contributor). The PRD was easier to track down, but again not obvious, even from the context.

Again, I get it - I'm not the audience and this is a bit of industry in-talk. But I think it's something that worth thinking about - you don't have to dumb down your ideas for them to be understood by a much larger audience than your immediate colleagues.

I’ve been in this situation (worked too long on something without syncing with others). But I’ve also been on the opposite side. Some of my biggest accomplishments in my career, where I really added value, were when I went deep on my own and figured something out.

Think of software design as a space with dimensions. You’re trying to optimize for something over that space. Development is ideally a series of incremental steps toward the optimum. The daily standup is where you check in and make sure your last incremental step was in the right direction. Generic developer of level N can make those steps. But what if you are stuck in a local minimum? How are you and your teammates going to find what is over that next mountain unless someone goes out there and brings enough provisions to spend the night? There’s gold in them there hills.

While I generally agree with the article I think a bit of nuance it could use is that sometimes not looping in other people, even if it causes problems now, can be the right move for your overall career growth. A key component of growth is being allowed to make your own mistakes. If you always let stronger engineers come in and handle the tough bits you might never obtain the sort of knowledge and experience that let you advance and frankly, your company doesn't want you learning those lessons on their dime if they can avoid it. Going off on your own can be one way to gain that experience even if it irks the people you work with.

As a manager you need to help set up a culture where showing unfinished work is ok. On my current team I will often make a quick prototype and ask our TL "Can you take a look to see if this is a completely off base approach? This code is not done yet but before I put more time in it I want to validate it's direction." My TL says that this behavior wastes his time and I should only show him fully finished work. He'll often say there's no tests or linting errors instead of looking at the approach. Same thing happens with design docs. If I'm even editing for spelling he'll refuse to look at it. "Ping me when you are finished editing".

It's a complete waste of my time to completely finalize something, have him take a look at it for 2 seconds, and then say it's the wrong direction.

And yes, we have tried to specify tickets more but he wants more general stories to allow the engineer to identify and approach. He just wants us all to identify his approach without him telling us.

Misaligned incentives. Engineers want to do cool work; laying the foundation is often the only way to distinguish themselves. Interrupting it by involving others at each step is often counterproductive (three women one child in 3 months etc.) Managers on the other hand need replaceable cogs doing the work at somewhat predictable pace.

No.

The biggest mistake engineers make, and they make it often for year after year, is not moving to another company when it is the right time to leave where they are.

To zeroth order, all engineers are overdue for a move.

> They train in school environments where they do classroom projects on their own, or work on long-term intern projects in a silo. They are used to having to figure it all out up front, submit it for review, and then get some sort of result, like a grade or return offer.

This is something I struggle with everyday. PhD students usually come from finishing their master and totally have this mindset. They feel like doing things only by themselves will be more valuable than if they seek collaboration. But we deal with very hard problems and previous experience is one of our best assets, collaboration with other team members is not only very well valued, but essential to get our work done.

A very importnat point, in my opinion, is that the students should not have to figure out all this by themselves. It is our job, as supervisors, to help them to change this mindset. Easier said than done...

Some of my best work ever was produced exactly when I went deep and explored the problem space in exhaustive detail and produced an ideal solution after a week -- or five.

You can't plan and iterate on a plan that's forming as you go along -- it's a creative process and that's mostly chaotic by nature. And that's a good thing when doing exploratory work.

I get it, companies want assembly workers but a good chunk of software engineering doesn't fit that mold.

The whole article smells of control mania and micromanagement to me.

--

I agree with some points. Devs are prone to using overly complex solutions and it does pay off huge dividends to bounce ideas with somebody 2-3 times a week just to make sure you're not going down a rabbit hole that won't help the problem you're looking to solve. Sure.

What the author fails to account for is that your average manager and dev team are usually not at all helpful. They want you to solve a big problem but still want it done quickly even if they do absolutely nothing to help you clear up the requirements or familiarize you with previous work. And then get grumpy because predictably you can't do the whole thing quickly.

Well, AT ONE POINT SOMEBODY has to invest the time and effort to untangle the big spaghetti and put it in orderly boxes. Either remove roadblocks for me or get out of the way and let me fight this herd of ducks alone. Grumbling about how not everything fits into your nice Scrum charts is not helping anything.

--

I sympathize with the overall sentiment that we should do our best to iterate and not have work done in one giant step. But that's not possible for some tasks and this has to be respected, not mercilessly micromanaged.

There's this general advice of "keeping people in the loop", and I like the advice proposed at the bottom of the article, but I do think there's potential for more guidance on this topic, as teams work differently depending on the overall workstyle or the specific circumstances of a project.

If your team is all a bit allergic to endless design doc discussion, dropping one of those in Slack might not work too great. But maybe pair programming with one or two others on the feature to help get feedback could!

Others do like to plan things out, so having the running Google Doc, or "one-page design docs" with recurring check-ins with a core group of people can do wonders.

There's the Amazon-style "write the marketing pitch", or internal-facing API documentation. And for some people iterative can mean shipping your API in layers, whereas for others it means shipping features bit-by-bit.

There are a lot of options, and I would love to see a huge compendium about it!

In a perfect world, this advice is absolutely true (and I've been guilty of it more than once). In reality, here's my experience: - poor specifications from the start, with moving targets in the middle - nobody available or willing to give definitive answers - management stuck in endless meeting loops rather than having time to look at what engineers are actually doing - engineering team (often severely) understaffed and dealing with multiple projects each - not time, patience or mental capacity to dive into what everyone else is doing -> result: crazy amounts of wasted time for everyone

This, as so often, is a result of lack of time, patience and _planning_. It feels like "Agile" has become a convenient excuse to just start and figure out what it is we're building as we go along...

There was a discussion about this a few days ago for those interested (588 points, 385 comments):

https://news.ycombinator.com/item?id=30074949

It is management failure, not engineering failure!

at highlevel, you must know what you're supposed to deliver, which can be extremely difficult when not everyone does (incl yourself) so people start to second guess things to 'get things moving' which can be a huge risk and potentially completely wrong.

Understand the actual problem , write it down and repeat back to get confirmation.

write down the (non)functional requirements from the stakeholders, the exact scope of work before doing anything, again repeat back to get confirmation,

In the murky world of greyness where people do not really know information or the full set of data, agree (again before starting work) on little subtasks/ plans to get the information (PoC)

Write down the Risks/ Assumptions/ Issues/ Dependences of your actions and again get them signed off

all this takes time and can be extremely boring relative to getting stuff done, so a lot of people just do stuff they think is right

My experience with this issue has correlated highly with poorly written specs and/or stories (or worse, none at all). This in turn usually means that someone in the team is being lazy about the work to be undertaken.

I get it - writing good specs and stories is hard work. But as a way of organizing and "guard-railing" work, they are priceless. If you can't explain in words the work that needs to be done and share and socialize that, you're not ready to start building the product.

I can count on one hand, literally, the number of engineers I've worked with who could magically craft an amazing product out of thin air and even then, there were parts of those codebases that were horrifyingly difficult to modify down the road.

It really depends, you can turn this into an advantage instead of a mistake. I think the element of it that is a mistake is the bit where its happening in a way end customers don't see, or the reason its done quickly is its for an MVP/initial launch and to speed that up as much as possible.

There is a certain threshold in a team where incremental communication yields increasingly decreasing returns very quickly, and slowly starts to become a cost. If you can find the sweet spot, and leverage your engineering knowledge to peek in, this 'mistake' can be an advantage.

Speccing something that is under-specced before coding and splitting big fuzzy task into a list of less vague tasks is a critical skill.

There's some similarity to big tech interviews: you often get an intentionally fuzzy task and your first job is to ask clarifying questions and tell your assumptions out loud. If you go straight to coding it's a big red flag.

In this vein, we can say that kind of interview prepares candidates to the real job where others are busy and don't have infinite time to write perfect specs and just hand them out to you to implement.

Sometimes discussion with other people takes more time than actually doing something. Trying to loop in other people make junior developers paralyzed of doing anything outside theirs tickets. You will create a generation of developers that are useless if requirements are not fully defined and tickets are not broken down into tiny increments by senior dev. I am tired of handholding people.

I like these kinds of reflections, but I always miss the context.

These abstract counsels don't translate well to ability to identify when we actually need to apply them. Providing context, detailing one or more situations when the writer learned these things may contribute to transfer a bit of tacit knowledge.

The biggest mistake I see managers make is not letting their engineers work on anything for longer than four hours without demanding a status check abd injecting FUD into the process. Utterly demoralizing.

This is very subjective advice. You could also endup with very patchy inorganically grown software when you dont do any upfront design. It depends on the scale of your project. If you are adding a minor feature to the product vs if you are creating the datamodel of a new product. I would only show the datamodel once I have run enough experiments to make sure that it can support the next couple of years of application software

I think the author slightly misses the point in the "why do they do it" section. People do it because being a software engineer, at least in US tech companies, is one big pissing contest. People, even if they're being nice, are just constantly trying to demonstrate technical prowess -- technical superiority over others. That's how you get respect and that's how you get promotion.

Contra to this, putting together design docs/PRDs/RFCs at an early stage is useless busywork, and engineers should go grab a coffee together and have an open-ended informal discussion about their work before they do anything else. Attempting to stop your team from working in isolation by throwing paperwork at them is less effective than just getting them to talk to each other.

Like it or not, that is how software gets built.

In the construction industry, there are construction managers and individual skilled workers but behind it are the architects. There’s hardly a mention of the solutions architect role in tech work discussions. They are neither manager nor IC in the definition of what they do.

The biggest mistake I see engineers make is designing a to z without getting a to m rock solid. Or planning how they’ll structure their future companies under an umbrella without launching one yet. Worrying a lot about much later stages without even getting started.

i don’t have the link handy, but this was discussed here a couple days ago

TL;DR YAGNI + do an actual "build vs buy (including FOSS)" in good spirit on absolutely everything.

The biggest mistake I see other engineers making is opting to build vs buy when it's commodity stuff that we could easily and cheaply buy (or even use FOSS for), e.g. log shipping software, monitoring agents, etc. I see folks building "frameworks" for agents they'd like to run, and dreaming up entire ecosystems, of course with complex message busses and cert strategies...

Please don't re-invent Fluentd / Logstash / Prometheus etc unless you have a very compelling reason to do so. Also, your UI will suck, and if you make support staff use it, they'll hate you and you'll lose trust. There are so many great free dashboard tools out there, just pick one and learn it / use it. For stuff like reporting, there are a dozen ways to do that with stuff like Jupyter notebooks running on schedulers (and please please please don't try to write your own enterprise scheduler, again there are many out there that have been hardened over years of battle).

Don't run your own mail server. Just do not. If you've never dealt with IP + domain + ASN reputation and deliverability issues before, please just trust folks when they tell you that you don't want problems like that. Just use SES / Mailgun / SendGrid / whatever.

Don't re-invent JIRA. JIRA can do what you want if you take the time to learn it, and maybe add a few plugins or integrations.

Don't re-invent Jenkins. Jenkins can do almost anything. That doesn't mean it's the right tool for the job, but in many cases it's probably good enough to get you by, and easily adds visibility + access control + logging + who + what + when + kinda why + where and SCM integration to any ad-hoc task-execution or [see above] task scheduling needs. Jenkins is probably one of the most heavily battle-tested pieces of software in the FOSS enterprise software world, along with Apache and MySQL. I run Jenkins at home.

Don't use Java when you could do it with Bash, or even systemd / built-in OS functionality. Learn how linux works and what facilities are available on your given OS. You will likely find that many common problems have already been well-solved in depth.

If you can just pay AWS to run the service, it's probably worth it. Their stuff is also quite well battle-tested (at least non-early-access products), and stuff like RDS, CloudWatch, SQS, etc work pretty darn well these days. If you're already in AWS, don't just run everything yourself and treat it like a virtual datacenter -- to do so is to miss the whole point of "the cloud". You'll save OpEx in the end until you're operating at huge huge scale, and even then, when your finance team negotiates with your AWS TAM, they'll work out a decent-enough deal.

Edit: and please please please do not try to write your own etcd / zookeeper / Consul / Vault / Cassandra / Kafka. There's a good reason almost everyone uses this stuff: they works well, the ways they fail are well-understood, and they get updated regularly. You won't do a better job. Even if you work at a 10k person company and can dedicate a team to building it, you won't do better. Your implementation will be buggy and suck for years. Just buy it (for free). When I interview folks with stuff like this on their resume that they've built, I usually see that as poor judgement (unless they actually worked on one that ended up actually getting used outside of where it was invented).

Don't even try to touch building your own SSO. Buy Okta, Ping Identity, or Google SSO (Google Apps).

I'm not saying don't invent anything ever, just that I see so much damn waste when it comes to over-eager engineers wanting to build stuff that already exists as their own career builder vs solving the actual business problems in the most efficient way possible, which I've found to more likely than not be just figuring out the process / workflows and gluing a few pieces of existing software together. Not every problem needs to be solved with 18 dozen microservices and GRPC.

Programming is a one-person task. Looping people in gives the rest of us something to schedule meetings around. So it really depends on what you want, getting the job done, or creating more jobs.