A systematic approach to debugging
ntietz.com
ntietz.com
Usually if you can do that, the bug's as good as solved. It's the ones you can't reproduce that take all the time. He does explain how difficult that can be.
Also, one of mine: "The hardest bug is where nothing happens." If something goes wrong, at least you can start from that. When nothing happens, you don't even know where it was supposed to.
Frameworks are notoriously bad at doing nothing and saying nothing if you don't have your inputs just so. And then you're just staring at config files for an hour hoping you will achieve satori instead of rage.
Learning a UI is especially bad that way. You double click, select a menu, etc., and nothing at all happens.
When the UX designer decided that greyed out buttons are terrible UX and bans them, so instead of disabling them (greying them out) you have to remove them if they aren't clickable.
Now, instead of being puzzled by why he can't click the button the user doesn't even know there is a button and has to start on by solving another puzzle first before solving the puzzle about how to enable the button.
PS: back in the day this could be solved by a tool tip or a paragraph in the help file. Today it seems such helpers are forbidden too.
I’ve used Cruisecontrol, which being first is uniformly awful by modern standards. But not lying about features awful. And spec files have partially vindicated some of their decisions. But only partially.
Unfortunately, it had no idea what you already knew, and would continue popping up bubbles until you turned it off. Of course, there was a fair chance that within seconds of turning it off, you would want info on something again. So, you'd mouse over to small global icon for it, pull down a menu, choose enable, and then it would gleefully show balloons for much of what your pointer passed over on your way to the one thing yow needed help with…and again on the way back to the menu toggle to shut it off.
But eventually, some kind person invented a system hack that made it turn on for just the duration that you held a modifier key, and it was glorious. For users, at least. For programmers, it was still annoying, because you were supposed to create all those descriptions: one for each enable control or interesting element and, to do it Properly™ one for each possible reason that the given widget was disabled.
Most users never got to experience the modifier-key method. Most developers, if they implemented it, only provided the item descriptions, without bothering with the disabled state.
Microsoft decided to partially imitate Balloon Help by providing, and asking third parties to provide, a tiny question-mark button in each dialog box. Clicking it let you then click any one item to get a description of it. Which was annoying when you wanted to know about multiple items. And more annoying when many or all of the items you clicked didn't have descriptions. If you weren't in a dialog window, yow were supposed to use the program's Help option, which was completely different.
Both Balloon Help and question-click-help eventually were replaced by that unfortunate idea: a smallish floating window that is simultaneously too small to show the content (often having scrollbars in both directions) and too large to get out of your way. With seemingly nothing satisfying available, help systems continue to diverge, offering varying degrees of utility like local HTML that opens in a web browser (often slow to launch), actual online web pages (useless if not connected), sidebars that resize your workspace and open sluggishly, and animated talking characters that won't accept "Go away!" for an answer (::shudder::)…
On the whole, I'd take something like "Balloon Help with modifier key", except for when a tutorial would be more appropriate.
…
But back on topic of grey buttons, other terrible-but-trending designs are "everything is light/medium grey" (makes everything look disabled), "one or more enabled elements are fancy/coloured while one or more equally enabled elements are grey" (often used when the greyed options are "Obviously what any sane user would want, but that negatively affects some target metric").
Which is why you should try hard to do it :) A lot of people... do other things instead.
Maybe the worst is trying to think up a solution.
My book about debugging would be called "Stop Thinking and Start Looking!", if I wrote it.
That’s not to say that you can’t guess the first, or second place to look, but that’s where it should end.
The second thing I see often from smart devs is that they find some bug, assume it’s the one they came looking for and declare victory.
The third is the worse by far, which is to blame the cloud provider/vendor for all issues, especially you not reading their documentation.
Preemptively give up and say, "We can't reproduce that."
I've never seen or heard anyone else express that idea before!
I guess I don't have to write that book then :)
In the terrible movie The Internship they have this "contest" for the interns, where they're all told of a server that has suddenly stopped serving up digital photos.
So they all start writing on a glass wall. No intern says, "hey, let's go and look at the server logs!" Not cinematic, I guess.
(or for a summary of just the rules) https://courses.cs.washington.edu/courses/cse474/18wi/pdfs/l...
As the number of potential culprits goes up the odds that you will identify the correct one first are very small. Most of what makes these processes slow is getting emotionally invested in solving the problem in as few 'moves' as possible. Occasionally that will work for you but it's not a safe long-term strategy.
What you want to do instead is to formulate several hypotheses, and then figure out the relative likelihood and difficulty in testing each one. Determine if partial tests of one hypothesis also satisfy or invalidates parts of others, and then begin the process of winnowing down possibilities in a calm and collected process of elimination, balancing cost of validating an assumption versus how much of the problem space it eliminates.
Once you find the bug you still have to fix it, and getting worked up makes for cures that are as bad as the disease.
Think of it like Wordle - with each guess you try to:
1. Use as many new letters that you can (so you can eliminate them),
2. Use all the almost-correct (yellow) letters in new positions,
3. Keep all the correct (green) letters in the same position.
So each guess is a test, where you try to invalidate as many letters as possible and satisfy the existing proven letters.
Feature-focused management often fails to take comprehensibility into account. Initial development hours are rationed carefully, but debugging gets an infinite budget, as if it exists outside the reality of delivering value.
Eventually is a tradeoff between the amount of time you invest in making the system comprehensible and the amount of time saved in debugging. But that's only after you've thrown down table stakes (unit tests, basic documentation, high-level modularization with loose coupling, preservation of critical invariants, etc.)
Well said.
> But that's only after you've thrown down table stakes (unit tests, basic documentation, high-level modularization with loose coupling, preservation of critical invariants, etc.)
I would actually turn this on its head, because I would rather have a comprehensible system without those table stakes features than an incomprehensible system with them. Of course, there is a feedback cycle between unit tests/docs/invariants and comprehensibility, but I have worked in systems with pretty good tests that were still very slow to work in because of their complexity.
Imo everything comes second to comprehensibility. Without it you are hosed.
Having such a finely articulated process has also been helpful to demonstrating to me why micromanagement in the business world completely destroys productivity.
Management hates, hates, hates this process to the depths of their souls because they would prefer a prescribed process that outlines well defined time-boxed steps to be articulated, from the get-go for many diverse disciplines to contribute to equally.
To them, Steps 1-3 should already by known and are unnecessary. Needing to go through them either means the issue is unimportant, the developer is incompetent, or there isn’t alignment between teams.
Step 4 and 5 means you’re flailing. You need to provide action, not ideas. “narrowing the search space” means assigning blame; healthy teams work together, they don’t assign blame. Developers attempting to break down the problem are disrupting the harmony of the work environment. This necessary step should be avoided.
Step 6 is the only step that matters. It’s more important to get to this step as quickly as possible ill informed than to arrive at it with data to support your conclusions.
Following these steps are strictly necessary in many circumstances. They’re also impossible to follow under scrutiny.
I say this to make a recommendation: Any good leader, to a development team, should insulate and shield their team from scrutiny so that they can carry these steps out rigorously rather than succumb to pressure and micromanage productivity into oblivion.
One has to define what a bug is. For example, I have many times received a bug report for something that was perfectly normal system behaviour. The reasons for this are varied from users not understanding the system, the system behaviour being complex and causing emergent phenomena that is strange at first sight but completely fine when you start to analysing it to consequences of underspecified requirements, broken communication at the time of requirement gathering, etc.
It might still result in a bug, but this time a bug report for the original design.
Another very important thing missing is it is absolutely crucial to gather from the user what was the expected outcome. I can't begin to tell you how many times I had a developer "fix" the bug only to find out the fix is not what the user wanted to happen. Or that there wasn't even a bug in the first place and the fix actually broke the system for somebody else.
As to the forming and testing the hypothesis, the missing is that you can do it all by reading, understanding the code and simulating the problem all in your mind. I have probably fixed something like 70-80% of all bugs at this stage.
What you do is you form a hypothesis and then you start tracing the pieces of data through the code of the application (aided by your understading of it). If I can find something in the code that disproves my hypothesis, I have now saved time on setting up the whole test. But I also gained more understanding of the problem which I would probably not if I just let the application fail. Even if I actually found a solution, I can have better understanding how the piece of data interacts with parts of the system and I may now be aware how to write the fix or at the very least how to do it better.
If something's broken in production and the bug is hard to figure out, that often means stress. And stress is not good for helping you to solve problems. So take a few breaths, slow down, relax.
Slowing down to get into the right frame of mind helps get to the _right_ fix faster, because your panicked brain is not jumping to all kinds of speculative conclusions about what the solution might be. Be calm, observe, reflect.
Coincidentally, I wrote a blog post about this and my other steps for debugging not long ago:
If you narrow your search to most recent changes, you have a really good chance of finding the source of the bug. The culprit may even be you!