Avoiding Worry Driven Development
seangoedecke.com
seangoedecke.com
>Untangling messy parts of the codebase, identifying and removing dead code, cleaning up build, deploy and monitoring systems - these tasks often feel overwhelming because they can overwhelm you, sucking up days for no real gain.
Those aren't the risks I'm thinking about when I'm worried about problems like this. Difficulty/time is not the concern. Unknown-unknowns are the concern.
Upgrading our NodeJS version takes about 10 seconds with Elastic Beanstalk on AWS, but we're not on the latest version, even though it seems to run without issue and our tests pass. Why? Because maybe it'll expose some race case that would only appear in production. And worse even, maybe that race case will affect a lot of people sometimes in an extremely hard to reproduce way. Maybe it'll break something that isn't apparent for weeks.
We had a bug where some SMS notifications weren't being sent in production, and it was hurting our retention metrics for a couple months before one of our engineers stumbled across it. It was caused by a maintenance upgrade of one of our very few dependencies. In a small company or on a new product, you don't have the kind of bulletproof reporting to know when something like that is broken - your metrics are moving around quite a bit by default.
Code that works and has been working for ages without issue is code that I am not interested in changing unless I absolutely have to. Leaving/avoiding code that hasn't been changed in forever is sometimes the best option to ensure stability.
This all being said, I agree that this worry can be carried too far. Like I said - I've gone back and forth on this. Fundamentally it's a tricky line to walk.
If your monitoring and alerting and quotas are setup right you'll know if the version update is ok.
To me that is very scary work though. Very difficult and risky and this article re-applies (at least to me).
It's also a great way to empirically find dead code.
Reminds me of something I read, like an illegal life tip; Don't fix an unknown problem because if it goes wrong people often won't understand or care about your good intentions and you will end up with blame (blame culture).
Whereas if a problem where to occur on production, and you were to fix it, well then you would be "hero".
Not trying to advocate for any particular way but just trying to reframe the question in more of a political/human way as opposed to an engineering one.
Another benefit of code-reviews is that they share the risk. So making a bigger change will feel safer to the individual.
I would simply congratulate the engineer for seeing something that I didn't.
Every once in a while something like this happens to me, and I found that being humble is a good way to relax at work.
Perhaps we should measure the humbleness of "10x engineers" to find out.
Interesting. When I'm humble, this means I don't think of myself as a rockstar ninja developer. I'm fallible just like the next guy. Right?
But my own aspirations to do great work are still there! So because I'm not that great, I work hard to get better.
Reflecting on the above, it seems right now I'm not humble enough, I'm too satisfied with myself. Thank you for giving me the chance to reflect and improve.
I tend to think that "rockstar ninja developers" are just as fallible as the next guy too. everybody makes silly mistakes sometimes.
I think the main problem with your argument is that the reason we do work is not always so clear. Sometimes our work is just pointless in the bigger scheme of things, but we still have to do it. It can be hard to find motivation from within yourself in this case, and it's easier to play the rockstar developer role.
So the answer to the ancestor's question is "no".
Update: initially I misread you as "affect your performance" reviews, in which case I would probably look for another job.
If you really meant as being humble affecting my performance, I fail to see how that is possible
Well, being humble could affect your desire to prove yourself, and from there your performance.
1) Transactions per second 2) Total time to process HTTP request and return response 3) Timing of specific sections of code 4) Exception counts 5) Specific function counts (get as detailed as you need here) 6) Response time as viewed from an external requester (with percentile breakdown) 7) Version numbers 8) Anything you feel is important to know about your software
I always deploy canaries and compare the stats of the canary box to the ones running version canary-1. Any difference in stats must be investigated and explained as valid before a new version can be deployed beyond canary.
The greatest tool I have ever created for regression testing, I call "A/B test". I record incoming network traffic, right off the wire using "ngrep". I have a tool that plays back this traffic to multiple different destinations and compares the response from each. Any difference in response between old version and new version must be explained and not caused by bugs. This tool also reads log files and other forms of output and does the same type of comparison. Obviously some things are always different like random numbers in responses, unique cookies, timestamps can differ by 1 second sometimes etc. Those things are removed before comparing.
This tool almost 100% guarantees no unintended change of behavior slips through a new release. The only way something could slip through here is if some odd request that wasn't tested causes it, or something in the data that we don't compare (due to it always being different like random number) causes something. I run 10 million requests through this A/B test and a "no unexplained difference" result is required before fully deploying.
not most engineers I know; they need to make money and will just do whatever to make that money.
There's plenty of "have this baby in a month" tasks that I don't want to become entangled in.
There's plenty of tickets I put off doing because I know it will be hard to feel like I'm making progress, or frustrating to navigate the social component of.
Sure, maybe if I approached my work totally dead inside and with no expectation of enjoying it or caring about anyone else I worked with, I could probably just mechanically go down the list of tasks, but I suspect that would also cause me a lot of other mental problems on the way.
There is a misconception particularly about startups (and I'll be the first to say I was living under the same rock) where they're seen as a magical box where you put a junior developer in on one end and they automatically turn into a seasoned vet on the other end. Yes, this may happen and you definitely will ramp up in terms of skills (both technical and people skills besides learning how to operate as a creative professional, etc) but it requires effort on your employers/managers part too—training, one on ones, feature planning, talking about how you would build before you actually do, what you're building and why, and how/why architectural design decisions were made, etc.
If you find your self out of your depth and in positions where you have to make senior dev decisions with no context or idea how to make them and why you choose the outcome, your management has failed you and perhaps themselves as the outcome will likely harm the team/company more than it will harm you.
The author said they’ve not seen this much, but it’s what drives most bad decisions: ego.