Algorithms We Develop Software By
grantslatton.com
grantslatton.com
This is true, and I've discovered it myself by losing a branch.
However, who the hell has time to write everything twice? There's 1,000 things waiting to be written once.
I appreciate this doesn't cover anything more than the basics, so something like the normal behaviour of comparing two password fields for the same content doesn't work, but I find these controls are useful for getting something simple up and running.
Also, it doesn't do much.
Nobody. The article even says, "N.B. Obviously, don't write literally everything twice. It's a heuristic. Apply intelligently."
If you raise the quality of your codebase, you can implement those 1,000 things faster, and you reduce the odds of they'll have to be reworked.
By that time, you will have probably left the company and someone with less care about code quality will come and undo your work.
Also, like security, it is hard to show to a manager data and graphs showing how much we are saving
And also if you're not in a bad starting position: you could have a system that is very hard and slow to work with, meaning that you can't feasibly introduce much easy to use functionality, especially if the domain logic is tightly coupled.
Such reasoning is based on a flawed assumption: that the value of time is constant in time.
While smooth is indeed fast in the long run, it is slower in the short run, and time before the release date is much, much, much more worthy than time after the deadline.
Shipping functionalities now and bugfixes later is what pays your salary. Waiting for the perfect code makes your customer seek comfort at your competition.
External deadlines are often meaningless, but customers are not the only users of the application. Once you release your part and keep developing/debugging/polishing your code, your colleagues can move on with their job and so on and so on.
As unfortunate and unnatural as this may seem to us programmers, shipping _is_ a feature in professional software development; and, in the quality/time continuum, "something now" beats "all of it tomorrow" in every scenario.
PS: I do get that the decision on releasing a product to the public has different constraints in the medical or aeronautical industry than a photo sharing website, still enabling the rest of the organization to move on with their tasks is too often underrated.
But you still have to put your stuff out there to test it.
All code is a cost. Features are what users pay for. There is scarcely objectively good code. One person’s smooth is another person’s rough.
Deliver stuff. Act on user feedback.
If you are not developing commercial software, the above is invalid advice.
That said I think it's worth noting the advice is to junior engineers. These days I feel most of my code is good enough most the time to not warrant a rewrite, as I can usually catch my self before doing something too stupid.
About me, I dunno. I can usually catch myself doing something stupid. But fixing it on the act often takes time from finding the stupidity hidden in the more complex corners (usually on how the software interacts with other things).
Nowadays, I tend to stop to fix visible issues if other people are going to interact with the software. But if not, I find it much more valuable to get a bad thing there fast, so all the problems become known.
If writing everything twice results in more maintainable code (eg higher cohesion, lower coupling, less verbose, more self-explanatory) then it's possible to get massive returns over the life of the module.
The developer who's going to receive page duty alerts about the feature, will have to fix its bugs, track user analytics over the feature, suggest and implement improvements to improve these metrics, write documentation, educate and support other team members on it.
Write it twice, then you can use it a dozen ways, and now you’ve got a hundred things to write instead of a thousand.
If you write code and don't think it needs to be rewritten, you are either an expert in your domain, or believe you have written code that fits your problem perfectly. Again, if you are not an expert in your domain, then what you have written is at best a solution that works without a second thought, but more likely could use another rewrite. Most software does not need a rewrite, but if we're talking about ways to reuse in a thousand ways, rather than a hundred ways, you need to have the luxury to rewrite code that is used in so many places that it's almost required.
“So you get maybe 2x higher quality code for 1.25x the time — this trade is usually a good one to make on projects you'll have to maintain for a long time.
N.B. Obviously, don't write literally everything twice. It's a heuristic. Apply intelligently.”
A few basic sanity checks (some unit tests, a little discipline avoiding the abstraction high, whatever your flavour may be) is fine in many scenarios. Not all features require tons of monitoring and documentation. Everything in our line of work is a trade-off!
Every rule has exceptions, this isn't worth mentioning.
I suspect that algorithms as a framework demonstrates the structural aspects (e.g., how some searches are more extensive), but might hide the driving factors. Indeed, the article examples were almost all hacking personality, not technical or process solutions.
E.g., most over-engineered solutions are driven by fear, often itself driven by critical or competitive environments. Conversely, much of the power of senior/staff engineers comes from the license to cut corners afforded their experience. Or people use the tools they know.
You can't get to great by holding onto good. It's easy to go from bad to good, but takes some courage to toss good to start over for great. The lesson there is that we stand (hide?) behind our work, and we need to let go of it to improve it.
A meta-lesson is that managers need to deeply understand personal space of each developer and the social dynamics of the team before they can do their job effectively, and a key part of that is likely checking in with developers in a way that enhances their courage and self-observation instead of making them fearful and paranoid.
Absolutely. Jira is equally slow for everyone except for the ones who are allowed to skip it.
At the places where i managed to become star developer it was by hitting hard early and achieving great results, and after that you’d get a lot of slack cut for you which allows you to continue deliver at the star level with relatively light effort, and definitely much easier than say the mediocre grind i produce at the current place where “Jira” is truely slow with us.
I suspect this is the common tale of poor compensation: despite being a star developer, employers won't pay for properly for this, just a mediocre annual raise, so you move on after a couple of years to another place, which offers you a huge raise (new starting salary compared to the previous job's salary). And the new place, while giving you a poor environment that hampers your productivity, still pays much better than the previous places where your productivity was much higher. And because of this common workplace dynamic, it's usually not worth it to put in much effort to be a star employee, unless you find the rare employer that actually rewards you for it. Is my guess correct?
My self observation is my performance is relative to how rushed I am to complete something. So I tend to perform much better at places that develop features on a 1-3 month cadence rather than somewhere that expects development progress to occur on a day/week basis. Even when the overall amount of time spent on development is the same.
I think having to show my unfinished work to people takes its toll on my confidence. I know it's crap because it's not finished, and I spend so much time talking about what's missing/poorly designed that I come away from demos thinking, "I just showed everyone how terrible I am at this."
My goal is always to be a star employee. Since I benefit from doing a great job just as much as my employers do.
>not worth it to put in much effort to be a star employee
it isn't even possible at the places like i'm currently in, at least for me. I mean, i'm still rated as high performing employee, just a regular one at that. You do your components in a very large platform product, make sure that they aren't source of pain, and just coast under the radar.
The difference between high performance in a trusted environment and high performance in an environment that lacks trust is significant. The problem isn't Jira - it's peer reviews and QA and gates to show both that the process is followed and that the quality is sufficient. The "high performer" is granted the trusted environment -- or at least the organization structures itself to minimize the pain to the "high performer."
So, someone can do the same amount of work but not achieve the same results because of the system.
So, then the logical answer is to throw the processes out! Get rid of Jira and PR and don't let quality block release! That works in a small environment, but the moment your company needs to generate SOC audits you will fail and you won't be able to sell. Or you will never be able to go public because your SOX auditor won't pass you. (And we won't even talk about PCI compliance!)
So, then, you end up in a push/pull. One of the part time jobs of management is monitoring the system and balancing the constraints. Here are some ways people have tried to balance the constraints:
* Pair programming. You reduce the theoretical output by 50%, but you fix that by putting QA and PR into the process. (You also have a feedback loop from two people reducing blocks and can stop interruptions by only interrupting half of the pair.)
* Mob programming. You reduct the theoretical output by 25% (assuming team of 4) but you drop QA and PR and you always have someone available to unblock. If you have a team of specializing generalists, you can always have the experience to solve the problem.
* Surgical team. You have a series of junior developers who support a staff engineer. In this model, the staff engineer figures out the hard parts and feeds the rest to a series of junior engineers. Junior engineers are also available to write tools and tests and qa. It seems like you have fewer people working on the key problems, but one person having it all in their head and unblocked continuously can compensate for the lack of theoretical output. (In this model, it is hard to hire seniors because one bad senior can tank the entire team.)
* Team of seniors. You still have the reduced output of the process, but since they work in a high trust environment, PRs can be glossed over and QAs are just there to cover the release management. (One new developer can slow everyone down until they're up to speed, and a bad performer can tank the whole team as it devolves into mistrust as processes have to be applied to all.)
* Hierarchical teams: A combination of seniors and juniors where the seniors take harder problems and juniors support seniors and take less complicated problems. mid-level are senior-light. These teams look like they're working closer to theoretical maximum but end up being slowed down by the processes of mistrust.
* Scrum Master/Project manager. For complex projects requiring a lot of interaction and a large number of people, take the money allotted to developers and give it to non-developers who take care of those parts. People complain about this here because it's often misapplied.
* Star Developer autonomy. For people who have proven themselves, the organization warps around them. It's like the surgical team, except these people are often tied to multiple teams. Instead, the other teams work being unblocked by these star developers or clean up the messes/maintain the work of the star developers after the fact.
All of these can be valuable. Some of them are not "agile." But the bigger the organization, the more difficult the problems get because they're problems of interaction and trust.
And none of them are about the amount of work, but rather the amount of perceived productivity by a given person.
One of the major projects I worked on was a virus genome analysis pipeline. Our initial funding was for single-segment virus analysis, but about a year into the project, our grant collaborators needed to demonstrate multi-segment virus support within two weeks. My PI and I agreed on a quick and dirty method that would produce the analyses needed and could be done in the allotted time. That piece of code was fundamental to how the whole pipeline worked, however, and so as the pipeline grew, it took on the shape of the "gun to the head" decision we had made to the point where other, more essential features of the pipeline had to be delayed so I could come up with more workarounds.
There were clearly other issues at play here (scope creep and lack of separation of concerns were huge). My time at the lab came to a close, but if I were to have a chance to continue that project, I would start over from scratch rather than deal with the baggage that that one "gun to the head" moment created.
I understand that it's a heuristic and not meant to be taken as 100% truth for every situation. I also understand that it's trying to avoid that "paralysis by analysis" that's so easy to fall into. I just question how useful it truly is as a heuristic, especially since it seems to go against the "write everything twice" algorithms presented in the rest of the piece.
My best projects were like that. I'd figure out something quick—some combination of reducing scope and doing "things that don't scale"—then spend time refining the conceptual design and interfaces, and finally rewrite the initial piece based on that new design. This can absolutely work better than just trying to write something "good" the first time around, but it looks wasteful to somebody superficially tracking individual "tasks" you're working on.
I think the author is presenting them as analytical tools that might or might not be useful depending on the situation.
Very often when faced with a difficult problem it's hard to know where to start attacking. Any idea, even if simplistic and wrong, can be useful to start gaining insight on what is going to work and why; even just refuting the original idea with a clear counterargument might suggest alternative avenues.
OT: this is IMO part of the reason why people like LLMs so much. Maybe the answer is trash, but articulating why it's trash gets you unstuck.
The author then recounts advice he gives to juniors, which is to stash the work and rewrite it, claiming that the next day the work will be rewritten in 25% of the time and 2x quality. This is unsubstantiated though. For juniors this suggests it will help them develop their capabilities to reason about implementations of problems without needing to face a a large amount of them.
The author then gives another advice which is to ask for a solution to a problem then after the initial proposal, ask for a 24h solution. This is meant to generate "the real solution". He likens it the a path algorithm heuristic to reach your goal quicker.
Overall the methods are not well discussed in terms of pros and cons, nor substantiated with experiments.
Opinion: I think they may help some juniors who need to build up experience and may become stuck in development patterns. But they would rarely be useful to develop someone to be a senior, if all they do is chase fast implementations. In a way the post gives conflicting advice: write twice and write better, and think twice and think about the fastest way to achieve the goal, instead of engineering a problem.
The author hasn't really convinced me of these approaches, and especially the last one smells of eXtreme Go Horse.
Care to explain?
Perhaps if you're at 0 experiments you can't really have an opinion? Which is the situation of the person I replied to.
A related old idea about writing/reworking software three times: "Do it. Do it right. Do it fast."
Startups provide ample opportunities to get experience with what the article calls "gun to your head heuristic". You have to decide what and how to compromise.
And, if you want your startup to be successful (not just hit your metrics/appearances, and then job hop), you can't just do it like school homework (where the only goal is to slip something past a grader, and forget about it), but you have to creatively come up with a holistically good compromise solution, given all the factors. If you do this well, it's creative magic that can't be taught, but can be learned, through experience and will.
The version I've heard is "Make it correct. Make it readable. Make it performant."
How experienced is this “CEO and engineer”?
If something cannot be coded within that 24 hours, something else is odd, not the feature. Transitioning from SWE to DevOps and then Leadership roles, most of my day actually is spent with all the reasons/excuses why "it cannot be done", and try to eliminate them. Probably my developers hate me for it, but I always push hard for an immediate first solution instead of doing days of soul-searching first, but over time we encounter and solve enough roadblocks (technical, social, educational, ...) that it actually happens more often than not to have surprisingly fast (and good enough) solutions. That speed is a quality in itself, since it frees up time to come back to things and clean up messes without the shipping pressure mounting up over days/weeks - something working is already there on day two.
The trick is of course to _not_ sell this 24 hour solution to upper management ever, or else it will become a hell of a mess fast once this becomes the outsider expectation.
Rather than trying paths in the dark, first look at a map, then try a few paths.
The remaining 10% is rather straightforward.
The alternative is "write it once and be stuck with it". You want tech debt, cause that's how you get tech debt.
By definition, the first time you do something, you will learn a ton. So you've gained two things: 1) hard-won technical knowledge and 2) a bunch of sub-optimal code written before you obtained that knowledge. The way I see it, keeping that proof-of-concept code around forever is a terrible tradeoff - giving up #1 to save #2 at any cost. Code isn't that special, especially on the first pass.
I think a lot of software development is "way finding" - Experimentation to figure out an architecture, an implication, a performance improvement, etc.
We often don't a) call them experiments, and b) we don't constrain them well. I.e., we use the scope of the feature to bound the experiment, instead of taking a step back to figure out the right approach, we dive into implementation.
I'm curious if there's a more formal way to think about this all?
Algorithms we develop software by (18.08.2024)
Never underestimate the power of hiring a new employee and training them how to do what the software would do, thereby writing zero lines of code.
That $250k in initial feature development costs + maintenance might only be $75k in personnel costs a year, which is a break-even of ~4 years. Depending on the problem, that might be the best option.
Sounds clever :)
As a manager, this "thought exercise" is dangerous. You think it is a fun and harmless exercise to your reports to really focus, your reports at best think you don't trust them, and at worst think you are threatening them with violence (hypothetical or otherwise) unless they tell you what they think you want them to say. Psychology safety will be at rock bottom pretty quickly.
Nice job you have there, would be a reaaaaalll shame if anything hypothetical happened to it huh? Now estimate again, and this time don't make me angry <imitates pulling a trigger of an invisible gun at someone's head>.
Absolutely terrible behaviour.
> The purpose of the thought experiment isn't to generate the real solution. It's meant to put a lower bound on the solution. Then you think of a real solution with that lower bound in eyesight, and you'll find it's often better than your original solution.
He does not say to actually implement the quickest hack you can imagine, nor to skip the mundane steps that avoid tech debt. He says spend 10 minutes imagining a quick hack after laying out your initial solution, and incorporate anything you learned into your actual work. To me, it sounds very similar to a startup chipping away at their original idea to get a MVP.
> The purpose here is to break their frame and their anchoring bias. If you've just said something will take a month, doing it in a day must require a radically different solution.