Learning from Your Bugs
henrikwarne.com
henrikwarne.com
Most of the requirements problems derive from lack of understanding about the tradeoffs in the current system or lack of understanding what the customer actually wants.
Most of the design problems are a result of excessive complexity. People often forget to simplify and refactor prior to adding a new feature. They end up having to keep a larger system in their brain when designing than they otherwise need to. Therefore they miss interactions that exist or forget to handle them in the new design.
Most of the coding problems would have been caught had unit tests been clearer and closer to 100% branch/code coverage. It is amazing how many bugs lurk in the last 5% of testing.
It is actually quite fun to tell a product manager that they are the cause for a bug :) Usually it is the developer that
Presumably, coverage starts with the low-hanging fruit (simple, linear code; heavily-exercised code like startup routines and main loops; etc.) and grows to encompass ever more unweildy, smelly, edge-case parts of the codebase.
Hence, by increasing code coverage, we're boiling off the solvent of run-of-the-mill code, such that what remains uncovered gets an ever stronger concentration of hackery, unintelligibility and black magic. By 95% coverage, the bits remaining are those parts people don't want to touch, so I'd expect the concentration of bugs to be highest there.
:)
But really, at that point the business often doesn't care. Not when a requirement bug could cause months of lost engineer time.
Our experience is requirement bugs are much more costly than a missed unit test resulting in a bug.
It is true that software engineers often treat bugs as unfortunate but unavoidable noise in the system; when one pops up, you fix it and move on. But it's really worth thinking them as an inherent part of the process of software development, one which you can improve and learn from, just like anything else.
For example today I was debugging an issue with Kubernetes (it was a really cryptic error about client-server not being able to negotiate a protocol or similar). I had never used Kubernetes before and I couldn't find anything on Stack Overflow.
The first thing I did was check the client and server versions (to see if they matched) - They did so that wasn't the issue. So then I had a strong 'gut feeling' that it was related to the client configs.
In this case, my second hypothesis turned out to be correct. 5 years ago I would probably tested my way through 20 different hypotheses before finding the problem.
To summarize my debugging strategy, it's simple; just pick all the low-hanging fruits first. The hard part is knowing what a low-hanging fruit looks like.
I'm just thinking out loud because I do what you describe here but I think the journaling idea could be valuable.
I'd go even further and say that I've often wasted time chasing down a hunch that was completely wrong.
It's a good memory exercise to try to remember when was the last time I hit that bug. At the beginning, I wasn't very good (and I couldn't remember the solution of the earlier problem - a journal would have been handy at that point) but after doing this for a while, now I almost always remember the solution to various regressions I encounter.
Another that I like to ask myself, is how I can use the compiler and its type checking to my advantage. As in, how can I make it more difficult for myself to make this bug again in the future?
But I agree with the author's point: Nearly every bug is an opportunity to learn. You should not just fix bugs, but also to understand why they happened.
Will give it a go over the next while. Cheers for sharing!