Only solve one new problem at a time
bennadel.com
bennadel.com
- It works. You added 2 things. It doesn't work. Which thing broke it?
- It doesn't work. You changed 2 things. It works. Which thing fixed it?
- It works. Two people changed it. It doesn't work. Who broke it?
- One donut left. 2 programmers went into break room. No donuts left. Who ate it?
Now change "2" to "1" in all of the above examples. See how much easier?- A fan
I got so excited by your response, I figured what the heck and turned my original response into a code-generated, html-driven, svg-drawn comic strip and put in on the front page of my portfolio here: edweissman.com. (Hit the 2 big arrows to surf the other 18 comics. For me, this is way more fun than anything else I ever did on that page.)
Enjoy and thanks again!
Exactly. I rolled my own CMS just for this. No images, just html and superfast svg built from database parameters. I'll be launching a full webcomic only site by the end of the year. I hope to include source at some point. I'll keep you posted.
You talked about having a positive mental attitude being really helpful in overcoming shortcomings. This implies you didn't start that way and you learned the behavior.
Can you explain the thought process you adopted? I sincerely struggle with having a positive mental attitude (see I did it again) but I have recognized it's a huge indicator of a better life (social, work, family etc)
I always had a positive mental attitude. Not really sure why. Difficult childhood? Last kid picked in kickball? Couldn't get a date? Knew nobody was ever going to give me a thing so I would have to do it myself? Who knows? Way too much to think about for an answer now, but that's OK because...
My problem wasn't that I didn't have a positive mental attitude.
My problem was that I didn't know I had a positive mental attitude.
I was a typical nerd who found solace in math and programming but somehow worked my way up into a dev / project management / digital rainmaker position. My customer wanted me to do something he knew I could do that no one else had been able to do (something boring sounding like "implementing a factory shop floor control system").
He became so exasperated at my pushback (I can't. It can't be done. It's too hard. etc) that eventually yelled at me something like, "You know the business! You know the tech! You're good with people! You have PMA! So what's the f*in problem?) I asked, "What is PMA?" He said "Positive Mental Attitude!"
That conversation changed my career (and my life). And he became my major mentor. (That's why I thought of him in that "Advice that changed your life" thread last week.
This may sound bizarre and counter-intuitive but I think the short answer to your question may be, "You have PMA when you decide you have PMA."
Over the years, I've witnessed countless counter-examples to this: people don't have PMA because they don't know they have PMA so they never "breakthrough" like they could.
I'm so glad I met that mentor. Thanks for reminding me.
Hope this helps.
I think it depends on the situation, what the best approach is. Is it a system, which is running in production and you are operating an "open heart"? Well, better only change one thing and check. Is is a newly developed thing, which you develop on your own dev machine? You can easily fix many things and many more will still remain for further iterations.
“In retrospect it is clear that without all-up testing the first manned lunar landing could not have taken place as early as 1969,” von Braun wrote. “It sounded reckless, but George Mueller’s reasoning was impeccable.”
e.g. code changes are easy but merging and build procedures take a log of time, encouraging larger batches of concurrent work.
Step 0: Shorten the test cycle.
- It works. You need to add 16 things. You add 1 thing at a time. 16 operations later you are good but the deadline has passed. Fired.
- It works. Add 16 things. It doesn't work. Remove 8 things. It works. You've eliminated 8 culprits. 4 operations later done. Delivered on time. Get a promotion.
- One donut left. 2 programmers went into the break room. No donuts left. Who ate it? Not any of the 30 programmers outside of the break room. Ask the 2 programmers who did it. Prisoner's dilemma. Fun had by all.If you’re doing all 16 at the same time and they’re not one liner trivial things, how do you even know where exactly the 8 from your divide-and-conquer strategy are? And can you actually back them out without further breaking things?
The longer you delay fixing bugs, the more concurrent issues are present in the codebase. So then when you add a feature, instead of either having 0 errors or diagnosing a bug in 1 new thing, you now have to diagnose n+1 bugs, where 'n' is the number of unresolved bugs.
The more this is allowed to fester, the worse this becomes, to the point of hopelessness. I've seen a codebase with ~1K open crash bugs, and hundreds of memory corruption bugs. Developers were making things worse and convincing themselves that they were fixing things because randomly the memory corruption was less bad on that test run.
*) To make things simple, just ignore false bug reports for now. Pretend all bugs are real, such as "makes the program crash".
Exception proving the rule. Yes, sometimes for something as deep as adding currency you do just need to fully refactor BUT I think you're still doing it in the spirit of "One Problem At a Time" You were adding currency, not adding currency AND updating the date library AND touching up the unit testing mocking AND adjusting that CSS for the checkout button (or whatever equivalent).
I've come to realise that a lot of problems in software engineering are a result of either having one thing achieve two or more goals, or having two or more things to achieve one goal. The thing may be a function or a person.
If the thing is too big for one person, don't add a second person. Break the thing down until you have enough smaller things to assign to people exclusively.
To change my phrasing -- If someone manages a large team, let's assume that they are responsible for all that the team does and can be viewed as a single entity, synonymous with a single engineer working at high bandwidth. That entity is absolutely not solving singular problems at once. Individual engineers within that team are not working in isolation from one another and they all have interconnect impact unless their delivery is absolutely rote. In order to move at a speed necessary for on-time results on complex problems, the advice proposed in the article must be violated.
Also a manager is not synonymous with a high bandwidth engineer.
Also, a high bandwidth engineer building all that is way more valuable than a manager building a team to do it, in terms of bottom line.
Sure managers are important and good ones produce results. But that’s not equivalent to a 10x engineer. That’s just the normal expectation for a manager.
For example, there's a bug in the login system, again. Ok, we could probably fix it, but this is an old system built by outside contractors (aka "built cheaply, but expensive to maintain"). It's not the first bug, and won't be the last. There's some technical debt in the way it stores unsalted sha1 passwords, password resets work in a way that allows an attacker to effectively lock out accounts and do a DoS. Looking forward we have customers asking about MFA and SSO support that this system just can't do.
"Solve only one problem" philosophy says to just fix the bug. This is a case where replacing the system (whether buy or build) might make more sense. You have to be pragmatic about this of course, because if you can't live with the bug for however long it takes to replace the system, it isn't going to fly.
I'd still advocate some constraint, though: iteration 1 should fix the bug(s) and underlying tech debt, but shouldn't be adding anything new.
Some of the most productive work I do is this style because it's often only a bit more effort. I used "login system" only as an easy to understand example, it's often more subtle, along the lines of: refactor a class so it has unit tests and fixes 3 other related but low priority bugs at the same time, instead of just patching. Instead of a day, it takes a week, but more value was delivered (other bugs fixed), there's significantly lower chances of bugs in the future (as future dev work happens), and we can reduce or eliminate manual QA on that component. The extra 4 days have saved dozens we'd spend over the next year if we dogmatically stick to "one thing at a time".
When you're starting those "replace X system" projects, what you're not anticipating is that half of those projects fail: they're either end up taking way longer than planned, or they don't get finished at all because they miss some critical requirements. The bigger the project, the more risky it is.
Your login system has two problems:
1. It has a specific bug
2. It has tech debt
The issue is that you're using current bugs as a justification to remove tech debt. Because, let me guess, you want to refactor, but you can't sell refactoring to the management.
Here's how management might look at your arguments:
> password resets work in a way that allows an attacker to effectively lock out accounts and do a DoS
Did it actually happen? I worked at a SaaS startup that had over 50K users, it had the same potential problem. It never happened. Side note: some security standards actually require this DoS vector to exist.
> Looking forward we have customers asking
And how far in the future are we looking? It might be years before it becomes a real problem.
You might be absolutely right in your desire replace the login system. Or you might be a perfectionist that's eager to solve problems that don't matter. The great thing about incremental process is that for a relatively small cost those risks go away.
Totally agree! However, there are ways to do them successfully. I've done it many times.
There's no need to ever get into a position where you miss a critical requirement, let alone having it prevent completion. My favorite method is building replacement systems in parallel, leaving the existing stuff in place. Often there's a good opportunity to do a totally new feature this way first (maybe allowing you to increase your total addressable market), before you start adding existing features from the old system.
> Because, let me guess, you want to refactor, but you can't sell refactoring to the management.
Ugh. This statement grates at my soul.
First, let's deal with "want to refactor". Up front, as a technologist, I'll admit I do sometimes want to refactor bad code/systems/whatever purely because they're bad. However, as a professional, I know I need to justify that work.
This brings to the second part, "sell refactoring to the management". This is just not a conversation that I ever have.
If I am (or my team is) being slow due to constantly refactoring code, management is going to question what I'm doing and hold me accountable.
If we're slow and causing customer complaints because every time we touch code we add more bugs and start a break-fix cycle... management is going to question what I'm doing and hold me accountable.
What I do is look for spots where there's a provable return on investment (justification), and simply make that part of the next bug or feature we're working on. If you can take a system/component/whatever that is constantly getting bug fixes, spend slightly longer on the next fix (because you're refactoring) and then have fewer bugs after and/or be able to implement features faster, that's a win. Do it a couple times, and you'll never be questioned about this type of work.
The discussion around doing a large-scale replacement is entirely different, and the ways to justify it are using real numbers such as: potential to hit new market; time spent on support; time spent fixing (repeat) bugs; time it takes to implement changes; time spent fixing new bugs anytime someone modifies the system. Just like above, if you can't prove this, you shouldn't be doing it.
>> Looking forward we have customers asking
> And how far in the future are we looking? It might be years before it becomes a real problem.
Let me clarify my sentence: "Looking forward in the backlog, we currently have open customer requests.."
I agree with the sentiment I think you're getting at, which is you shouldn't build to unknown future requirements. You'll pretty much always get it wrong. I teach this to juniors with an analogy like: If you build a foundation for a skyscraper before you actually need a skyscraper, what you'll often find is it needs one more underground parking level than you thought and to be rotated by 15° -- so everything you built is not just wrong but is actively blocking you from building what you need.
> You might be absolutely right in your desire replace the login system. Or you might be a perfectionist that's eager to solve problems that don't matter.
Let me just be clear, because I wasn't originally, that my "login system" was a fictional example. The real examples I have require too much time to explain adequately.
> The great thing about incremental process is that for a relatively small cost those risks go away.
The point is sometimes a rewrite makes more sense because it solves several problems at once. This is where I take issue with the original article.
You hit on the reasons a full out rewrite fails: Not understanding all requirements. Doing a switch without backwards-compatibility. Going months/years without delivering value.
At least I think this is where we align again: The key to a successful rewrite is building incrementally, delivering value as quickly as possible.
And that one fix is a big win if it addresses multiple problems. We should look for commonalities among problems.
Whether you explicitly break up your "problem" into smaller problems by organizing your workflow that way, or implicitly break it up by simply biting off bits as you work seems to be the difference. The former being more planned, with the latter requiring greater mental tracking and ad lib task management.
The greatest disadvantage regardless of how you work is task switching. If you're doing push-ups and sit-ups, it's easier to do 50 each x2 than 1 set of 2 each x 50. So if you include switching tasks as a task, you're doing 3 exercises, not 2, and your work becomes 150 instead of 100.
Benefits: A) Easier for your colleagues to review and understand your change B) You can undo your change easily without affect other changes
Now, if you do 2 changes per PR, A and B become a lot more complicated and just take more time.
Example: a couple of years back, I needed to do a new web app. I could have relied on my tried-and-true jQuery and HTML 5 skils. But with all I had read about Vue and the examples I saw led me to the conclusion I might do better timeline-wise (thus, reduced risk) by using the web app as a vector to learning Vue. I wasn't wrong, things went much faster once I get the hang of how to Vue. My client was so impressed with the work, he asked if I was just middle-manning a white-labeled tech solution from a third party. (Thanks bootstrap-vue!)
How would that have turned out if I did jQuery / HTML 5 then a rewrite into Vue to keep it "one problem at a time"? It might have taken 2.5-4x as long.
Anyway I look at the "solve one problem at a time" as a useful tool in the tool belt. But it is not an overriding principle. Risk Containment is that principle. A few years back, this was a peripheral tactic, even an afterthought. I've been working with some Risk Containment Ninjas for a couple of years, and the key lesson for me, the iterations are shorter in duration, the cost is less, and the idea evolves to a better end result. It forces you to think about every tiny piece of a project as driving home a MVP.
You'll be much better off if you learn the fundamentals then use a real project to polish your skills than trying to learn from scratch with a deadline looming.
This goes back to that "innovation tokens" thing though, a new language is one innovation and you don't want too many others at the same time.
I'd even say days, not weeks if the language is similar to one(s) you already know.
As for "learning the fundamentals", it's incredibly difficult to know what you don't know or understand and what you need to know/understand until you're trying to solve real problems. It's so inefficient and unmotivating. I knew the absolute basics of Spring Boot when I started working as a Java developer, but it wasn't until I started working a job with Spring Boot that the real learning started and I felt the motivation and drive to learn everything I could.
Frankly, businesses need to adjust and learn to be okay with paying for learning curves, instead of expecting people to know and have experience with everything under the sun. There are long-term benefits that aren't easily quantifiable.
When I was consulting, I would occasionally make a time box deal with the client. One project sticks out to me: I thought their project was perfectly suited to Rails at the time, but the majority of my projects had been done in Django. “I’m going to try building this in Rails. When I hit a 40h timebox, I’m going to stop, look at progress so far with a critical eye, and assess whether we’re getting value from this or not. If I don’t think so, I’ll abandon the Rails project and start over for free with Django.”
As it turned out, the whole thing was done in about 30h with Rails and we didn’t need to have the rest of that conversation.
I don't believe there's a 1:1 straight translation from (in this case) Ruby to Go without a lot of extra work in the form of re-evaluating features, flows, performance, etc; the problem they solved wasn't the business feature, and a programming language should never be a problem.
My approach, ironically was the incremental approach mentioned in the article. So the first step was for me to view it as one problem - build app using X. Then the steps might have looked like
1. Learn ABC
2. Apply ABC
3. Refine ABC
I always know enough to build something. Eventually, I know enough to refine it.
Now the author said this is slow and I agree but at the end of what would be a learn-first-build-later approach is an actual, built system. You can now solve the next problem, enhancements.
To any new developers, I should mention that a good foundation of fundamentals is key to this kind of approach and I admit, there are much better ways to learn and build.
Maybe I'm just jaded. I cant count the times I've been handed projects where worked 'incrementally' meaning the original team mistook that for use all the time and budget yet not deliver the product. Or the times I've been handed projects where everything from the language to the os had changed without changing the timeline.
Absolutely only one one change at a time if you can. The real value comes in being able to take a situation with multiple changes and still make it work.
Concurrency also means solve one problem at a time, but in multiples timestamp instead.
Pausing between iterations on a thing can be an opportunity to sit back and collect the observations you need to know what to do in the next iteration.
But pausing between iterations of a thing simply because there is some other thing you want to start on and you just can’t wait to get started, even if it means pausing work in something that isn’t actually done yet, is how vaporware is made.
I felt that personally working on eesel [1]. We kept a tight focus on the problem (making it easier to refer back to your work more easily) and it's become far more than we could've expected. Ironically, the classic "is this a company or a feature" question might actually be a validation of keeping scope.
The only caveat I'd add to the post is that you should try to solve only one problem, but you should spend time exploring before hand to discover the broader set of problems and prioritise carefully.
I personally have a tendency to cherry-pick easy tasks and procrastinate on the gnarlier ones. Single-threading projects would force me to confront those difficult tasks and "eat the frog".
Do one thing and do it well.
It's a different side of the same coin.
As a Sysadmin this methodology is CRUCIAL when it comes to troubleshooting.
Make one change.
Test.
Test again.
Ask Bob from accounting to test it. (Bob will break it. Dammit Bob, thank-you.)
Rinse, lather, repeat.
- I have ended up on his site numerous times after searching unrelated topics (amazing SEO?) and the content is generally very useful
- He has all these happy photos on his site of meeting people at conferences
Content has been useless every time. His blog hasn't helped me once, despite stumbling on it 20 times a year sometimes.
Offshore(next day): I see.
Local(next day): Process ABC threw an error.
Offshore(next day): We will look at A.
Serialize as much as possible. Take advantage of time zone opposition and asynchronous communication.
If you throw everything you have at a problem all at once you might solve it without figuring out which of the things you tried worked.