I’ve made a conscious effort to stop apologizing for bugs in my code
blog.danslimmon.com
blog.danslimmon.com
One is all-about-you. The other is a team-building action, recognizing the folks who normally get a lot of grief(denial; accusations; ignoring their input). Testers respond amazingly when they're heard and treated seriously.
It is so profound, let me tell you an anecdote. My test lead had gone to a conference (they have them!) He was in a bar at the end of the evening, shooting the breeze about all the type-A developers they have to deal with, with a group of attendees.
He told the group "I have a guy in my company, I find a bug, he thanks me! Then works closely to categorize and then fix it. No complaining; no yelling. He just says Thank You! and gets on with it. Says I saved him embarrassment if the bug had gone any further, and saved the company money."
Another guy in the group, from a different company in a different state, said "I know who you're talking about. I'll write his name on this napkin, you tell me if I'm right". He wrote my name and passed it over.
They had never met; never working the same companies at all. Were in different places in their careers.
But I had worked with both of them. One of them, 8 years before. But still, still I was the only developer either of them had ever dealt with, that responded positively to a bug report.
Think about it, and then go thank your tester.
I mean, in the end, the person notifying you of your mistake didn't create that mistake. All they did was take the state of the world from "you don't know about your mistake" to "you are aware of your mistake". Other than edge cases of plausible deniability, this can only be a good thing, right?
What is funny is sometimes I'll run into a problem customer that other developers complain about and I'll profusely thank that customer for finding the bug (even if they are being jerks about it).
After a few rounds of that and they think they're a valuable power user with lots of insight who is saving us all. (Doesn't matter if it is true or not.)
Suddenly they bring me bugs all happy as a clam that they found something that nobody else could (again doesn't matter if it is true or not), and they're all proud and in a much better mood and feel better about the product.
Would you accept such nonsense from your plumber, or your dentist, or your electrician?
Software engineering is one of the most fundamental building blocks of modern civilization, and yet it continues to be so fragile because of this kind of cavalier attitude.
Also, apologizing doesn't mean you have a moral failing. It means accepting at least partial responsibility, and being humble about it. Any good org knows that you don't just design your system with the hope that people won't screw up, you need to design it to work despite some amount of screwing up. But in either case, it is still both an individual and collective responsibility.
Maybe there's a reason you rarely hear one category apologizing and hear the other category coming up with excuses.
Failed you? This sounds either very entitled or sensational. Has software caused you major grief in any way financial or the loss of limbs or loved ones? While this happens, very few people are affected by bugs in such a way, at least confirmed.
Many people have been significantly affected by software bugs. Therac-25[1] is one I constantly remind myself of; it killed 5 people. Here's a list for further reading: https://en.wikipedia.org/wiki/List_of_software_bugs
I'm not sure what your definition of "very few people" is but we'd all do well to aim for 0 adversely-affected people as a result from using software we write.
> I'm not sure what your definition of "very few people"
I think my definition is about the same as every accepted definition of "very few", the number of people affected by such bugs falls within this definition, considering how many users software has.
> Many people have been significantly affected by software bugs. Therac-25[1] is one I constantly remind myself of; it killed 5 people.
5 does not fit the definition many in anyone's book.
Well, IIRC, there's Watership Down, a literal book, but I assume you meant that figuratively.
...
>I was asking OP if he was personally affected
So 5 people dead is insignificant, but a single person, adversely affected, is cause for concern?
So the analogous question would be whether someone who has worked as a plumber has ever heard a coworker plumber apologize for messing up a job. I'm hoping the answer to that is yes. A plumber apologizing directly to a customer would be more analogous to a company making a public announcement about a bug, which is a different issue than how responsibility is determined within a team.
"Just be more careful" has pretty much proven not to be a successful strategy to reduce defects.
We are pretty good at writing software with a very small number of bugs, it just costs a few orders of magnitude more than normal software.
Imagine if an electricians invented new types of wires for every job they did
I'm not sure why software is the way it is, maybe it will be more like a trade in a few decades...
In buildings with nontrivial electrical systems, shit hits the fan really quickly too. Just look at the absolute mess that is the fire prevention system in Berlin's new airport for example.
Copper, plastic, other things which I have no authority to talk about, were all complex at one point in time
You do make a good point about scope creep, I wonder if the early days of electronic had a similar problem because nobody knew what they wanted or what was possible?
Very interesting to think about IMO
Why wouldn't this be acceptable? They source that crown from a supplier, and apparently there was a fault with it.
This is an incredibly ignorant statement. Have you worked in that profession? Any professional engineering roles?
But that all doesn't really matter because the statement becomes meaningless when qualified with "Depends on the project[...]"
Besides all this incredibility, yes, I do work professionally in an engineering role.
Limiting a statement is not the same as a completely making it meaningless. Of course there are plumbing projects, which are more complex than a static blog. Besides that, I would also like to remind you of the little part of the phrase "I'd wager that". I did not know, that it is an "ignorant" statement, to express ones believes.
I would be interested in some experiments of average developers learning average plumbing vs average plumbers learning average development. I'd wager theres an order of magnitude difference in the results(time spent, quality etc).
>It reinforces the idea that any one person or piece of code can be blamed for a given failure.
That can absolutely be the case. How could it not sometimes be the case?
>Short of malice, this is never the case.
Never? Outside of somebody deliberately harming a system, there has never once been such an instance?
>It gives the impression that, when you wrote the code, you should have written it better. This is a counterfactual that rarely holds up to examination.
Lots of assertions in this piece with no supporting information. There are plenty of reasons substandard code is written by some individuals. Laziness is a big one. Apathy is another.
>It positions shame as the correct emotion to feel about bugs in your code: if you were a better engineer – a better teammate – the bug wouldn’t exist.
Call it whatever you want, but concern is certainly warranted.
>If you’re a more senior engineer on your team, the effects of these anti-patterns are magnified: people see you apologizing for bugs, so they think that they should be striving to write bug-free code. They may feel ashamed if their code has bugs.
People look up to experienced individuals within their profession. In fact, I would argue that it is even more important these senior individuals are able to accept responsibility and act accordingly; that's setting an example, something from which everyone will benefit.
Why not a little bit of both? Can we not think about system and data driven improvement while also at the same time apologize for our mistake?
In the UK...
Dentists: you have to have a BDS or BChD degree from a dental school at a university
Plumbers: have to get NVQ diploma as well as pass Gas Safe accreditation to fit boilers and heating systems, plus get a CSCS card to work on construction sites
Electricians: have to get an NVQ diploma, and to work in construction, they must also have an ECS card to work on a building site
Software developers: [this is left blank as an exercise for the reader]
In fact, there's a (very popular) school of though that the more credentials you have, the less qualified you are to develop software: https://blog.alinelerner.com/how-different-is-a-b-s-in-compu...
I mean in electrical engineering, idea-to-working-prototype involves a lot of design and assembly first. In structural engineering, you just can't build a bridge on your own. It goes on.
While in software development, I can cough up ten distinct features in a day if not more without forward design or testing.
This would be a reasonable comparison if 90% of software developers could reliably produce bug-free code. This would be a (maybe less) reasonable comparison if 75% of software developers could reliably produce bug-free code. This would be a middling comparison if 50% of software developers could reliably produce bug-free code. This is a completely unreasonable comparison given that there have been 0 software developers ever, in the history of software, who have been able to reliably produce bug-free code. So yes, software development is orders of magnitude more difficult than plumbing, dentistry or electrician(-ing?). If you think it isn't, please feel free to jump in and prove all of us wrong.
A hammer that melts in a furnace is considered a bug?
The possibilities and test paths a piece of code can possibly take exceeds a very large number in a complex software application.
Zoom should work and gracefully fail if there is not a single microphone device found on the client’s PC.
I agree but this point is orthogonal to the main point. Plumbers and electricians build to code in an environment where physical reality remains constant and they operate on expected inputs. A civil engineer is not held to account for a failure if gravity suddenly reverses itself. If I shock a circuit board with static electricity and it fails is that an electrical engineer's fault? If I put 20,000 psi through a household plumbing system and it fails is it the plumber's fault? These designs are only expected to operate in a narrow range on a small number of expected inputs / variables and that is easier.
Software can basically expect infinite permutations of inputs at any time and is expected to give good outputs and no bugs 100% of the time?
“Thanks for reporting this, I’ll take care of it” is accepting responsibility while avoiding a morality judgement.
If I make an off-by-one error, where, other than me, did it come from?
> It gives the impression that, when you wrote the code, you should have written it better. This is a counterfactual that rarely holds up to examination.
If I make an off-by-one error, I could have done better.
> It positions shame as the correct emotion to feel about bugs in your code: if you were a better engineer – a better teammate – the bug wouldn’t exist.
It is not about shame, it is about responsibility.
> If you’re a more senior engineer on your team, the effects of these anti-patterns are magnified: people see you apologizing for bugs, so they think that they should be striving to write bug-free code...
We should all aspire to do better.
> ...They may feel ashamed if their code has bugs.
See my previous comment on shame.
About the only thing that annoys me at work is people trying to deny responsibility for their actions (or lack of them.)
Someone once said "There is a difference between 10 years of experience and 1 year of experience 10 times". The best developers I know always take responsibility and try to do better.
Taking responsibility might involve any among:
- documenting the behaviour for future maintainers.
- writing tests.
- identifying and pushing the management to fix technical debts.
- digging up the root-cause to fix deeper flaws, if any.
- advocating for saner development and deployment practices.
- ...
aspiring to do better does not mean aspiring to write bug free code. One is a nearly impossible goal (not over time but certainly at first and certainly when balanced by other constraints)
> About the only thing that annoys me at work is people trying to deny responsibility for their actions (or lack of them.)
There's a difference between denying responsibility and apologizing.
What I do when I find a bug or someone points it out to me is say something like
"I see that. I thought the xxx did yyy but that's clearly not the case. Now that I think about it xxx may also do zzz, is there anything else you can think of that I should take into account"
Or more simply. "Thanks for catching that. I'll put it in the bug tracker. Is this a showstopper for your work?"
maybe git-blame s/b renamed git-sorry
Hence 'aspiring' not 'achieving'.
Guilt and shame are the enemy of responsibility and those who seek to inflict guilt and shame are toxic to any organisation. They encourage those who are focused on maintaining an image of never being wrong while punishing those who seek to better themselves and obviously make mistakes as they grow.
I managed to remove everyone from my life who inflict a guilt or shame (luckily not any family members) and life has got way better since then both personally and professionally.
Could have come from confusing documentation on the appropriate bounds, could have come from an unclear exit condition for the loop, could have come from contradictory requirements that had no real solution. That you're the agent at the end of a chain doesn't release the chain from responsibility.
> If I make an off-by-one error, I could have done better.
People often miss targets by small amounts. Tyson Gay is only behind Usain Bolt in the 100 meter world record by 0.11 seconds[1]. Is that all on Tyson not working hard enough, or are there maybe other factors?
> It is not about shame, it is about responsibility.
If it's about responsibility, then there should be a process for understanding why engineers end up writing bad code. Of course it can be that the engineer made a mistake, but it can also come from many other factors. Assuming that the person who wrote a bad line of code should "take responsibility" puts the cart before the horse in building a good process.
After all - it's the compiler that emitted machine code that did the wrong thing. Maybe the compiler is the real villain here.
> We should all aspire to do better.
I agree!
> Could have come from confusing documentation on the appropriate bounds...
If you are blindly following direction to that level of detail, are you a programmer or a typist? Maybe there will be corner cases like this, but they do not invalidate my point.
> Tyson Gay is only behind Usain Bolt in the 100 meter world record by 0.11 seconds.
This is not a comparable situation. For one thing, off-by-one errors are not off-by-0.11 errors, though the difference is more significant than that.
> If it's about responsibility, then there should be a process for understanding why engineers end up writing bad code.
Of course, and there is no shame in that. Part of answering that question, and in becoming a better programmer, comes from acknowledging and understanding the mistakes we make (this is an important part of aviation safety, and the last thing an instructor wants to see is a student who has a tendentious excuse for every lapse.)
And no-one is responsible for things they cannot control, unless they set them in motion themselves, or ignored the likelehood that they could happen.
>> We should all aspire to do better.
> I agree!
Yes! this is the most imprtant point - and there is no shame simply from falling short.
I have seen countless instances, both in and out of software development, where a manager:
1. Explicitly told an employee to rush through a task too quickly, and that some bugs/cleanup/whatever would be OK and handled at a later time.
2. A short time later freaked out because the rushed task was not done perfectly.
Of course only terrible managers behave this way, but there are a lot of terrible managers out there. In these sorts of environments simple errors are often ubiquitous.
Assigning personal attributes (responsible/irresponsible) like this is a sub-optimal strategy for improving human performance. It conflates what a person does with who they are. What a person does is malleable, and their performance can be improved with teaching, practice, better techniques, etc. Experienced teachers understand this and avoid turning a discussion about correctness into a discussion about character, as it's counter-productive, and reinforces the narrative that performance is based on innate character, not on hard work.
It is not obvious to me that there is anything very wrong with this. In particular, I think it is rather more reliable than going by what they say.
"I did not do X, therefore I could have done X" seems clearly false. How do you know you could have done better? And if you could, why didn't you? And if you actually didn't, what does it usefully mean to say you "could have"?
You "could have" done better but didn't, does that imply you didn't want to do better? Is that what you want to communicate - you wanted your code to be worse - because that isn't in your favour either. You wanted to do better and didn't do better, that's back to implying you wanted to but weren't able to, i.e. you couldn't.
Because it was an error! I could have done better by doing the thing that fixed the error!
I am not a religious person, but the Lord save us from those who have an excuse for everything.
> the Lord save us from those who have an excuse for everything.
Lord save us from people who think self-flagellation is at all useful or beneficial to anything. It's status-signalling and nothing else, you're in hypothetically the same situation as someone else (having made a mistake) but you want to claim to be superior because you're feeling the most bad about it?
In this post [1] we see you arguing that everyone makes mistakes, yet here you seem unable to recognize it as the obvious answer to your question. Perhaps not coincidentally, a failure to be semantically consistent is a common root cause of programming errors.
> So answer the rest of my questions - you could have, you didn't. Why not?
As a rule, I try to avoid responding to utter nonsense, though I must admit to breaking that rule in this thread.
> yet here you seem unable to recognize it as the obvious answer to your question.
I'm the one who is insistent that it was a mistake and that people make mistakes. You're the one insisting there's some loophole where you can both get it wrong and feel good that you could have got it right, at the same time. I'm saying that it's your position which is utter nonsense, a meaningless double-think designed to make you feel good, which reflects nothing about the world.
Like a kid saying “I ate ten hot dogs and threw up, but I could have eaten fifteen!” - nobody believes it, you couldn’t, you’re trying to save face and get credit for eating fifteen hotdogs without actually doing it.
Now, whether you can do better in future is a different matter - I’m not saying you can’t do things to avoid that mistake happening again, or learn from experience, or get better.
This is bizarre. Where on earth did you get the idea that I am arguing that one could have not made a mistake? The whole issue is about one's attitude and behavior after having done so.
I am not going to consider anything else you write here until you clear up that mystery, which seems to be the root of your confusion.
My thoughts are, it's all about accountability. Good development and, more broadly, learning, rely on effective feedback loops. "Did it work? Y/N and why?" repeated over and over.
When you blur accountability in an effort to be empathetic, it messes with this loop. You've changed the definition of "it" to not be fully what was in your control, and then as a result don't have a clear action you can take to do better next time. This, to me, is how you wind up with 10 1 year experiences.
To summarize, even if a bug is only 10% your fault, be accountable for it, because that's a clear signal towards something you can improve. Accountability doesn't mean falling on some sword, you can also own improving the things that lead you to make this mistake. The difference is, you're taking action to improve things, versus distributing blame (which I find often leads to "what can you do" type mentalities, leaving all the broken things there for the next unlucky person)
Bugs are solely my responsibility: there is no one else to blame. And bugs mean that my customers paid money for a product which did not work as intended (certainly not as I intended it to work). I feel it is definitely right to apologize for bugs!
I disagree with the author: if I worked at a workshop and dropped a hammer onto my customer's foot, I should apologize. No matter what the reason: I could be clumsy, or you could say that no one could be expected to never drop a tool — still, an apology is in order, because I did drop that hammer, and it made my customer's day worse.
Therefore, I always apologize for bugs.
What I did, however, learn to do, and it is quite important, is not to apologize when you are not sorry. An example is when a customer is unhappy with the pricing, or complains about a missing feature. Many people will begin a reply with "I'm sorry, but…", and it's the wrong thing to do. I only say I'm sorry when I really am.
You write some code and a code reviewer points out that you used an API incorrectly, which you did because the documentation and test environments are out of sync with the production API. You will, of course, modify your code to use the API correctly, but will you "apologize"?
Now again, you call the API incorrectly, but your company doesn't do code reviews and they just ship the code. Your code ends up causing a lot of headache for everyone after it runs in production for a month doing the wrong things. Do you "apologize"?
Note, what you did is exactly the same in both scenerios. Should you apologize? Are you the only one who needs to apologize? Should everyone apologize? Like, do you all sit in a room together and go through a strange routine where everyone around the table takes turns saying "I'm sorry".
I'm surprised by all the disagreement over this post, so I wonder what people would think of these scenarios.
If it was feasible for me to catch the erroneous API behavior through testing, before code review, I would see no reason to not apologize.
Your second scenario is not even remotely ambiguous, of course I'd apologize if I caused other people that much hassle.
Stop tying your identity to your work.
They are asked: "Would it be preferable to serve ice-cream cake, or traditional cake at your wedding?"
Alice writes "ice-cream cake" secretly on her answer card, but Bob is still thinking.
After some thought, Bob writes "traditional cake" secretly on his answer card.
Both answer cards are revealed. The answer do not match, so the couple does not win a prize.
Alice criticizes Bob, "you should have known this, we love ice-cream, we had ice-cream on our first date!"
Bob believes he is not at fault. They argue. At one point Alice says "stop tying your identity to that question, just admit you made a mistake".
Bob still believes he is not at fault, and that he made no mistake, because it was never within his power to be certain what Alice had written on her card.
---
And now I ask: Is it within your power to be certain how the hardware and software of a modern computer will act?
My point is I will never be able to be certain that my code is bug free. That will never be within my power. I can only ensure that I follow a process that gives a good-enough probability that my code works correctly. What that process is, and what "good-enough" means, is largely determined by my company and team. I do follow that process, I do what is within my power. Should I apologize for not doing what was never within my power? Should Bob apologize for not doing what was never within his power?
I do accept the responsibility of ensuring "my code" (however the company wants to define it) is working well with the rest of the system now and in the future. Apologizing is completely orthogonal to accepting responsibility. Apologizing alone is not enough to "accept responsibility", and one can accept responsibility without apologizing.
PS - Alice and Bob eventually learned to avoid making moral judgments immediately in every situation.
Accepting responsibility is understanding the other person's reasons for hurting, which means listening to them.
Apologising is trying to make the other person to stop hurting, and stop them from talking anymore so you don't have to hear more or spend more time on it.
You can modify your code to use the API correctly, but if your team doesn't get the documentation fixed or the test environment to sync back up with the production API, your team is not solving the issue.
Your code can cause headache for everyone after runs in production for a month, but unless your team begins to do code review, you're not solving the issue.
In a very literal sense, the idea of the team finding an issue and resolving it (apology or not) is extremely important and one of the few ways for an organization to improve rather than decay over time. The apology is almost a formality.
I've been part of such retrospectives, and no blame is given, and no apologies are given. There is no hesitation to lead the investigation into "your own code". "Your code" might end up being what needs to change, but it was not alone in causing the fault in the entire system.
Another though experiment (stripped of all moral judgement):
Alice writes some code which runs as part of a larger system for 10 years. No problems are identified in the system.
Bob writes some code. After Bob's code is integrated with the rest of the system, problems are identified in the system.
Changes are made to Alice's code which resolve the problems.
Who is at fault? You probably find it hard to identify who is at fault without knowing more details. You need details so you can form your own personal moral judgments. Perhaps a better question is: Can we ever objectively identify who is at fault? And does it matter?
Of course, it is also possible for management to shirk such responsibility and push that responsibility (without corresponding process ownership) onto the ICs. It's quite common in low performing organizations.
We're typically not apologizing for the act itself, we're apologizing for the effect it had. We don't mean "sorry for writing code with bugs in it", for example. We mean "sorry the bugs in the code caused you problems."
I agree that you have to consider more than only the actions of the individual. I believe, if there is fault, it lies with not just one person, but also the circumstances. In cases where the circumstances are determined by management, company culture, legacy code, etc, the fault lies with many if not everyone. Thus, if one should apologize, then everyone should apologize.
Another example: If my wife wants me to invest in stocks and I do so after doing a lot of research and preparation, and that stock goes down, do I apologize? Do I apologize because of things that happen after and independent of my original actions? Or maybe my wife should apologize? Maybe nobody should apologize? How much depends on the stock market? Out of our control and comprehension.
The HTTP request can time out. The firewall of the user has blocked the request. The server might be turned of for maintenance. The server might have an expired certificate.
But there really are some people who constantly make mistakes, and those that rarely do.
There are people on my team where, when there is a bug, I just start looking at diffs of what they have committed because more likely than not, they caused the problem.
I think it's important to reinforce an attitude of trying to become better and people who give a shit when they make mistakes tend to also be the ones who make less of them.
If you can tell the bug from looking at diffs, why was the bug not caught in code review or by tests? Most obvious bugs should be caught this way.
It's like giving a minority report, honestly.
Even if these people suck at coding and are still committing code without someone else double-checking their work, they're not the real problem.
> * If you’re a more senior engineer on your team, the effects of these anti-patterns are magnified: people see you apologizing for bugs, so they think that they should be striving to write bug-free code.
Imagine writing this. Imagine actually sitting down at your keyboard and writing down that people should not be striving to write bug-free code.
I agree with you, it's outrageous. What an awful article. I get the point (you should have robust and multi-layered quality systems) but just the complete rejection of accountability is embarassing.
The point is to not have a "shame culture" for writing bugs, because they are normal part of the workflow. Instead focus on efficiently identifying and fixing bugs (efficient debugging is an important skill that's unfortunately often neglected).
Otherwise you end up with people writing overly defensive code, and get into a situation like the Stalin-era airplane designers who each made their own parts just a bit stronger then necessary because they feared ending up in Gulag or against the wall if it was "their part" that broke. End result was planes that were so much more heavier than planned that they were essentially useless for the tasks they were designed for.
John Carmack put this better than i can [1]:
> Left to themselves, most people have a tremendous ability to ignore their flaws, and it hampers their growth. A bit of shame is often a positive motivation. I am ashamed of a lot of code I wrote last year. I have reasons why it is the way it is, some of which are defensible, but some are just “WTF was I thinking?” If you don’t have nagging bits of guilt about your recent body of work, it might well be a benefit for someone to point out problems in terms that break through your defenses.
But there's no need for a "formal apology" to your team either. The user might deserve an apology by the PR department, but this shouldn't single out the actual developer who wrote the bug.
> It reinforces the idea that any one person or piece of code can be blamed for a given failure. Short of malice, this is never the case.
Never the case? Never?
There is an explanation to this discrepancy. Risk = probability of fault * cost of fault. Programmers usually work with the data. Data has a pretty very unique property that it is easily and cheaply copied and restored. It also means data can be replicated more easily, which allows to build systems with large amount of redundancy. Therefore, price of a software bug is usually orders of magnitude less than the price of breaking something in the physical world. This allows to drive up probability of fault parameter and allows for much faster evolution and less regulation.
Many structural engineering projects are built with a safety factor greater than 1, which allows for - as you'd imagine - safety. In software, it could absolutely be done that every function has explicit validation checks for each input, and that more safety, error handling, and checking is written. The economics just don't make that viable though except in the most extreme situations where this is required. In aerospace, with the exception of the recent Boeing debacle, there's been very few software related issues (that I know of) relative to the number of CPU cycles run and flights taken - I'd be willing to be there's far more mechanical issues that cause grounding and maintenance cycles.
Now look at civil engineering and infrastructure. How many roads do we drive over daily filled with potholes? How many bridges get patches and repairs every few years because of concrete cracking and rebar rusting? (I live in Quebec, so there may be bias in this statement). Maintenance like this is just paying down tech-debt - the same as in any field.
Software absolutely can be built to a high level of robustness (and it is) - it's just not economically sensible in most cases.
The damage caused by a software bug can range from nil (even negative, since a bug might have desirable effects) to incalculable. Consider a bug in software used for landing SpaceX boosters... Or a bug in a trading platform. Or a bug in an automated medicine dispenser. Or...
You can compare the work of a programmer to the work of a civil engineer, but it's true that it's difficult because most bugs cost little, much software is not positioned to cause much damage if buggy, software bugs are so prevalent, and it's so difficult to write bug-free code. But the cost of replication of software has nothing to do with any of this.
Replication is the cornerstone of fault-tolerance. When logic is written in high-level languages with some minimal good practices, most SW bugs result in crashes rather than data corruption, hence data replication plays important role in protection against bugs as well as HW problems.
> Consider a bug in software used for landing SpaceX boosters... Or a bug in a trading platform. Or a bug in an automated medicine dispense
I specifically mentioned: "Programmers _usually_ work with the data. <...>" In the examples you listed above software manipulates physical world rather than pure data. I thought it was too obvious to explain.
“I really should have checked the x-ray better before I extracted your front tooth. But you know, I’m not sorry. It was an external constraint — there was a queue of patients waiting”.
I can’t believe how low we let this profession sink. I hope in 15 years we will look at thies era like we look at the Wild West today. Exciting from a distance but nobody would really like to live like that.
I believe we need to get the "context" before judging any situation.
Building software should be an engineering discipline with all its perks and responsibilities just like building bridges. We rarely see it and it propagates to the craftsmen who seriously believe that good quality software is impossible to write under the slightest of constraints. Like business pressure.
We had this ability in the past when programming was way more difficult, tedious and error prone. Look what IBM did with their “Clean Room” approach. I’m sure we still have it at many places but we choose to spread the myth that it cannot be done. This call to stop apologizing is the next step towards not even trying.
I think everyone would not deny this fact. Engineering is more of a discipline.
But if you see things practically, this issue of "bad" code may not happen for various number of cases. For example, unclear project requirements, use cases etc., That is where 'context' of what happened helps to better understand the situation.
Building bridges are not done agile and it has a learnings from decades or even centuries. Computer Engineering is mostly agile where you push changes based on changing requirements.
Like on that dentist's case, my hope is that as liability persecutions and lawsuits due to faulty software increase, that will eventually be sorted out.
Should we expect for them to go into a full-on dogeza Japanese kneeling bow apology? Of course not. But there are many cases where a person glaringly half-assed something, cut corners, quietly committed something directly to master. And yes, this person should feel some kind of regret, or should be brought to stop such behavior because it is extremely destructive and is totally avoidable and caused not by overworking or accident but by persistent carelessness and neglect, so it cannot just be cheerfully swept under the rug.
... where a developer was given 30 JIRA tickets each "sprint" and whose "performance" was evaluated entirely on how many they completed.
How does the old saying go? The road to where is paved with good intentions? You could engage in moralizing against "carelessness" and "neglect" and presuming that a teammate somehow cares more about watching Netflix than doing their job (which they presumably outperformed other candidates to even get), and maybe that will make you feel better. You are the good, careful, model employee, and that other one -- they're careless, slovenly, apathetic and lazy. But if that's the case, how did they get past the door in the interview process? If the whole company is like that, why do you work there as opposed to a company where the bar is higher? Something does not add up. In all likelihood, your explanation is a rationalization and not the most obvious answer, which is that your process could be improved but it hasn't been because such investments are not viewed by your companies executives as improving the long term bottom line to be worth the short term investment cost.
Take a look at history and figure out how companies in the industry have solved this problem before by building more bulletproof runbooks, processes and tools. These companies, as they approach enormous scale, necessarily have to determine how to deal with employee reversion to mean. It turns out, surprisingly, that process eats good intentions and care for breakfast. You'd be surprised at how quickly those good intentions become useless if your company is successful and you hit hypergrowth and scale. It's ironic that for a profession where it is so tractable to automate away mundane or repetitive tasks, where we study spacetime complexity in data structures and algorithms, that we have so many practitioners that default to witch hunting and moralizing and seem unable to apply spacetime complexity or procedural analysis to their own software development lifecycle. I expect to see this trend change as our still young industry continues to mature.
The problem is that the person(s) for whom I had to add these baby bumpers not to do input_array[0] will - and has - screwed up much bigger on everything that is slightly more complicated than that, too. Because if you are an engineer and you don't care, then no amount of padding around you will make you curious, careful, detail-oriented, non-lazy and capable of tackling hard problems that life throws at you every day at a tiny startup.
But you first need to believe it is in your own power to write less buggy code. If you can identify patterns that frequently produce bugs, you can try to avoid them (e.g. copying & pasting: very buggy!). You can identify, isolate, and document footguns and complex API. And you can be proactive in simplifying systems when feasible to do so.
Don't beat yourself up if you accidentally have a bad commit, but don't excuse sloppy behavior. Demand better from yourself. Don't produce garbage software.
They're still going to apologize to the customer who got the wrong drink. Because sometimes mistakes are acceptable, expected on occassion, and still worth apologizing for when they happen. I don't always say sorry when I do something wrong, I sometimes say sorry when what I've done has a bad impact on someone.
You did not test it? Apologize.
You were sloppy? Apologize.
In most other cases I can agree with the author.
All I can add is that, as a freelancer, I always explain (like they are 5) what went wrong. My customers don't care about the details but they like it when they know I made an effort of creating good software (bugs included).
Not on every case. I once had a customer who literally removed the "Testing" part of my offer because it seemed to expensive to him.
As testing often takes a considerable amount of time I do list it explicitly in an offer and don't "hide" it in the numbers for the other tasks.
However, I've always believed an apology is important to show humility to my team mates. As a manager, I do not want to appear entitled. Of course, as soon as the error is uncovered, we should prioritize it and fix it. But I do not believe that proceeding to fixing an error is orthogonal to apologizing for it.
We are trying our best to build a company without a guilt culture. In such a position, is apologizing a bad move as a leader? What would you do?
I think context matters too. The industry you're in (what are the consequences of bugs) as well as the part of the codebase (presentation or data integrity) change the acceptability of bugs.
1. How will you fix it?
2. How will you prevent it from happening again?
"I made a mistake when i dereferenced that null pointer"
and:
"I'm sorry i dereferenced that null pointer"
I absolutely agree with you that the ritualistic part is pointless, and perhaps harmful if it is performed instead of actually fixing the problem (see also "thoughts and prayers").
But i wouldn't want to stop doing the acknowledgement part.
I very rarely apologize for a bug at work. I don’t expect others to apologize to me either. Apologies should be reserved for personal/relationship things, like forgetting to invite somebody to a meeting.
> It reinforces the idea that any one person or piece of code can be blamed for a given failure.
Why is that a fallacy?
Surely, the person making the mistake is the proximal cause for a given bug, and surely all other things being equal personal attitude matters (i.e. a more careless person is going to make more mistakes), but if you zoom out to the big picture you see other responsibilities and more importantly you see opportunities for improvement that vastly exceed the playoffs of "just putting more care" into each commit.
The intent of a "no blame culture" however is to find a new sweet spot on the myriad of tradeoffs an engineering organization does, as an many paradigm shifts we do, it is accompanying with a certain amount of bullshit-sounding maxims meant to be thought provoking and dislodge the readers from their suboptimal local maximums.
So what is the responsibility of the key-puncher if it isn't turning requirements into robust, correct code? Many comments here seem to suggest that programmers are just there to produce some sort of fuzzy first-stage approximation of what the code could be.
In that case, just employ an automated code generator and push the burden onto reviewers.
> So what is the responsibility of the key-puncher if it isn't turning requirements into robust, correct code?
How can it be a responsibility to never make a mistake? Nobody can never make a mistake.
> Many comments here seem to suggest that programmers are just there to produce some sort of fuzzy first-stage approximation of what the code could be.
Many comments here seem to suggest that producing high quality code is a team effort supported by a company or department which values high quality code. If you had mythical no-mistake employees you could do away with most everything else. Why even have tools, tests, revision control systems if you're not making mistakes?
And if you are using tools, tests, etc. as part of not making mistakes, why not reviewers, bug testers, quality analysts, as well?
A code reviewer is able to see that someone is building SQL from raw input, the QA isn't. And if I see that a developer writes that kind of a thing and another person approves it, the first question I have is if they actually read through the code or just glanced at it.
If you're using my code and hit a bug, I am, genuinely, sorry. The goal state is craft software that works properly. That is not an unreasonable goal!
Of course, mistakes will happen, but you should be sorry, and you should think about what would have avoided it (better programming methodology? better testing?).
Aim to create high-quality software that delights its users!
To me it's a similar situation to if I, say, accidentally spilled your drink; something of that nature.
(I guess different people could attach different amounts of significance to the word. To me it simply means, "I feel sympathy that this unfortunate thing happened, and I wish it had not.")
Why doesn't your dentist mess up 1 out of 5 interventions? They have gone through years of strict training and there are giant liabilities in place.
In development? A lot of self-taught developers, who can always jump to a next gig because of never-ending demand. This sets up the scene for a totally different scenario - never-ending stream of bugs.
If you want your team to have less bugs, then as a manager make sure you have a process in place that optimizes for that. Start with your hiring process and add requirements for code quality practices. Then build a process that includes strict commit guidelines, documentation, code reviews and QA process. Most importantly, add time to estimates that ensures all of the above is checked properly. After all that is in place, define in a clear way some sort of liability for bugs.
Your dentist goes through all of that, why not your developers doing arguably equally important work?
To me it seems that failures happens pretty often, generally due to incompetence, limitations in the science regarding the problem or insufficient resources. Seems very similar to bugs to me.
The question is how bad is failure, obviously if someone's health is at risk, you have a much bigger incentive to have processes on the safer side.
Maybe you feel "bugs" are more frequent in software because the software you're referring to has an incentive towards innovating rather than stability.
I'm sure if you studied software used in critical systems, the failure rate would be more in line with a procedure at a dentist. You can't justify the kind of cost that kind of development has for most apps.
The world needs enough software, and the world's requirements for quality are low enough, and the average developer writes reliable-enough code, that in most cases money can be made without special attention to quality.
In some domains, quality is paramount. I imagine that quality and bug mitigation are the primary concerns of someone who manages, say, an X-ray machine firmware project, or avionics software. Software for these purposes that is not extremely reliable cannot compete in that market. For other kinds of software people are paid to write -- like social networks for pets -- reliability is a much lower priority.
Certain kinds of software needs to be rewritten constantly to meet the demands of highly-evolving markets. Games are an example of this. Reliability is important, but not paramount, because the underlying business can only continue to compete by at some point stopping work on the reliability of existing games, and moving on to developing new games.
I'm sympathetic to the respect you have for the value of high quality software. But the reality is, the value of quality varies dramatically within the software industry -- much more than it does within, say, dentistry, or medicine, or aviation.
Fewer bugs isn't necessarily the primary concern of those using and buying any particular kind of software.
I mostly agree with the rest of your post.
But I disagree about it being something special to leaders.
And the way he says it, makes it sound like maybe he thinks leaders shouldn't apologize in general. Which I think is a fairly common belief that leaders use to justify not admitting they made a mistake. And is 100% different from not apologizing for bugs. Since as he points out, bugs are not mistakes, they are a normal part of development.
When the manager makes a significant mistake it's not like a bug, it's a poor judgement that probably affected everyone. And sometimes if it's a very negative affect then an apology is warranted. But at least, it's usually necessary to acknowledge when you had people going in the wrong direction.
Something you definitely don't want is to sound like it's an inconvenience for you that they're reporting bugs. Bug reports are extremely helpful and you should be grateful that they're taking the time to report them to you.
As a company, by some PR representative, yeah, sure, maybe, but individual developers should not apologize.
Working at Instagram, probably not, since it wouldn't be my job to handle support.
It seems to me that most comments with a strong opinion (including the article) have a specific scenario in mind, involving assumptions about the dev's intentions to write good code, the bug's effect, how easy it would have been to avoid the bug, and who is this person we're considering to apologize to (coworker / manager / customer / QA who found the bug). In some situations apologizing feels right, in others it doesn't, so there's not much sense in generalizing from a single scenario to the general case.
I think ultimately it comes down to matching expectations: almost all people involved in software development expect bugs. The question is how much effort do they expect me to put into reducing the quantity and severeness of those bugs. When I go to a dentist I expect them to make every possible effort to avoid harming me; generally this is not the expectation of software developers going about their day. A feature that takes one hour with reasonable effort at ensuring quality could easily take a day or a week if you make _every possible effort_ to ensure it is bug-free. Different pieces of code (at different times) could have different amounts of impact, so it takes a wider understanding of the code's context to decide how much effort I'm going to put into chasing bugs. When in doubt, I match expectations with the relevant stakeholder(s). Then the situations where apologizing feels right are either those where I didn't meet the expectations, e.g. I was in a hurry to push so I didn't take the time to properly test and I should have known better (not very frequent); or situations where I didn't take the time to properly match expectations (though sometimes that's on the other party, but it takes two to match expectations).
One other thing regarding responsibility - I think that taking responsibility and apologizing are different things. Sure, you can take responsibility by apologizing, but there are other ways too. When something breaks in my apartment and I call my landlord, they (usually) take full responsibility and make sure the problem is fixed, but so far they never apologized, and it would feel funny if they did.
But I read this as though the author is saying apologies to fellow teammates shouldn't be necessary, which I do at least in part agree with.
It's an empathetic response that says "I am putting myself in a different context and trying to understand how it feels". This makes it perfectly appropriate to say "I am sorry" a lot more than we do: when learning about a friend's bad-luck circumstances, a customer's poor experience or adding a bug to your code.
If someone points out a bug, I say thank you. People make mistakes. I am a person. I make mistakes. This is not controversial, it is expected. We should still strive to avoid them and are allowed to be annoyed with ourselves when we make them, but it doesn't make us bad people.
Should you apologize for bugs? No. But if you're doing professional quality work, (as opposed to prototyping or researching,) a large portion of your time (as a developer) should be writing automated tests (like unit tests) to test the code you write.
Code is _always_ a flawed approximation of an ideal system. You can move bugs around, but you cannot get rid of them: software _is_ bugs.
My instinct is always more like, "oh, nice catch!"
The company could apologize to customers if needed. But it would be insane to make the individual developer apologize to customers.
I also take the position of "criticizing the idea/code, not the people" when it comes to other people's work. I rarely have a problem with criticizing myself.
Especially when it comes to "your" own code. You should be able to apologize for your mistake.
> It reinforces the idea that any one person or piece of code can be blamed for a given failure. Short of malice, this is never the case.
You can be attributed to your action. If you can be attributed to the good things in your work, then you can be attributed to bugs. Unless you are claiming that you have no control what so ever about how your code turns out, at which point, what are the different from you and a typewriter?
The thing is, you can be attributed for an instance of error. You can be blamed for the error you did. But you should never be categories as "the error".
> It gives the impression that, when you wrote the code, you should have written it better. This is a counterfactual that rarely holds up to examination.
Yes. There are instances that I could have written it better.
> It positions shame as the correct emotion to feel about bugs in your code: if you were a better engineer – a better teammate – the bug wouldn’t exist.
> If you’re a more senior engineer on your team, the effects of these anti-patterns are magnified: people see you apologizing for bugs, so they think that they should be striving to write bug-free code. They may feel ashamed if their code has bugs.
I take the approach and culture of "be strict with yourself but be kind to others". When I talks about other people's bug I always criticize the bugs itself and talks about problem and process in general without tying it to them. But when I talked about my code, I will also express what I could have done better or what is the pros and cons of my coding decision.
The thing is there's different between people striving to better themselves, and people blaming others for not being better.
You should strive to be better, while at the same times not being a snob or looking down on other for not being perfect.
In every day's life there's no permanently good people or bad people. There's bad action that people do. You should still be able to attribute a certain instance of event to a person. You just don't have to hold grudge and judge them forever based on it.
I am probably reaching it here, but I feel that the author is the kind of person that can't take criticism of his idea well because he feel that his idea is himself.
Your idea and action is yours. But your idea and action is not who you are. If you think it is yourself, then you become blindly defensive and either never acknowledge that you did it, like the author did, or never change.