Hacker News Reader

If an organization is having developers triage vague bug reports directly from end users, they're already in trouble.

Not only are developers a very expensive resource, they're generally not good at doing what you describe here. Nor are they usually well set up for it as their systems/devices are often tainted with development and debugging tools.

We can't make end users deliver bug reports that are detailed and reproducible, but (whenever possible) developers should only have to worry about reports that have those qualities.

Any organization that's grown beyond a couple founding engineers needs to have a layer of QA or Customer Support that's responsible for everything that you describe. It's a layer that not only pays for itself but also keeps both users and developers happy.

I think you're taking my post a little bit literally. The "customer" doesn't always have to be the end-either. If a QA member comes to me and says they found a bug, I also should not just go back to them and say "It works for me" and then not proceed any further. (Although, in defense of the literal interpretation of "customer", for small startups, devs often double as customer support for engineering-related issues.)

There's a real difference between a Customer and a QA member, though. You can't demand leave the responsibility to make a good report on a customer because they're external to your organization. But you can do exactly that with the QA member because it's precisely their job to identify and describe defects.

You can indeed say "it works for me" to the QA member, and it's their responsibility to identify the environment or series of steps that reliably produce a failure. From there on out, you can no longer say "it works for me" because it doesn't: you finally know how to make it fail and now it's your job to figure out why.

And like I said, nascent startups with just a few founding engineers are a necessary exception. But the QA or Customer Support hire should come very early! Too many organizations stall on that, and they waste tons of opportunity and productivity by doing so.

I remember reading about a company (game company I think) that got tired of unreproducible bug reports, and added an automatic RAM test to their software. The number of bugs that could be traced to RAM faults was significant.

I wish I could remember who it was, but my Google-fu fails me today.

Edit: found it - http://www.codeofhonor.com/blog/whose-bug-is-this-anyway

"It works for me" is not an answer, but it is the first step in a reasoning process. "It works for me, it clearly doesn't work for the bug reporter, so what's different between their setup and mine?" That then leads to questions about steps to reproduce.

> "It works for me" is not an answer, but it is the first step in a reasoning process.

Exactly. "It works on my machine" has over time become the easiest way to get a death glare out of me. To rehash the old quote, we are not shipping your machine!

"It works for me" is a great first step. More specifically, it works for me on Windows XP, Vista, 7, 8, and 10 TP (since they're all ready in virtual machines) is an even better one. Followed up with the usual questions of What operating system are you using? 32-bit or 64-bit? What's the full install path? What antivirus product and version is running? What software firewall, if any, is running? Any other apps running? New install or upgrade? Etc. I have some of this built into the About box of one of my apps and am adding in the rest shortly.

Quite a few "bugs" are no fault of the developer/publisher. But learning where those issues are and letting your app alert/inform the user what's going on as they install to an invalid location or break their TEMP directory thinking they're protecting their machine or remove IE from Windows using a broken 3rd party utility or use an antivirus product that gets more wrong than right, etc... can help cut down on the bug reports, help the user from making 'mistakes' they shouldn't be allowed to make, and just make the whole process easier.

That is nice in theory, in practice many, many users are stupid (for the lack of a better word) and the bugs they report are exactly how the program is supposed to run, or caused by things that we cannot reasonably deal with, typically client computers that are too underpowered and have too much adware installed.

it isn't that black and white. Many times it's simply a misunderstanding, the developer isn't actually testing the right thing. Saying "works on my machine" is the opening to a dialogue that involves "can you show me what you did?"

Sometimes it can be as simple as initiating something via keyboard vs. mouse.

Are you implying it's a matter of getting your customers to use their brain?

I freelance and therefore I can be a jackass without direct monetary repercussions, perhaps that's why I don't view it that way.

Because I struggle to think of a lot of situations where I've been unable to replicate an issue a Client has reported where it hasn't turned into a miscommunication/misunderstanding of some sort.

No, not at all! If you've been in the business long enough, you learn that people have different preferred ways of working. Some will use menus, some will use icons. Some will use keyboard shortcuts. Some will use drag-and-drop. If the bug description doesn't specify how they performed an action, and you assume your preferred way while they used another, you can easily be unable to replicate their problem.

I don't remember any specific instance where I had this misunderstanding, but a coworker had it happen and will never forget it.

In the context of web development, one of the biggest offenders (in my experience) is coders who attach a "validate this form/submit it via AJAX" handler to the form submit button's click event, instead of the form's submit event. If I try using Enter to submit the form, their code doesn't run.

I misunderstood you, I thought you were implying customers needed to use their brain, but looking closer you did say keyboard and mouse and not keyboard and chair :)

I personally see it all the time, but I interact directly with my customers and it's always possible I'm terrible at communication. They'll say things like "X doesn't work on page Y", and It'll end up being a problem with a single link they clicked on because the data for it is such that it causes a problem (that sort of thing).

They don't know that's the issue either, you have to work with them on it.

> Was the user mistaken?

99999 out of 100000 yes. Sorry but most bug reports, even from QA departments are noise at best, nuisance at worst. And as I posted here before: I am close 40, have 15 years experience as a professional in this industry, I have seen it all and yes it gets harder for you to break through to me because I will presume you have no clue what you are doing because so many before you didn't. Sorry if you are the one special snowflake.

This is not to say my software is bug free, of course not! But unless you can describe the steps you made clearly and also have an expectation/happened instead , your bug report is useless. Again, this doesn't mean I expect reproducable bug reports but I need something solid to work with.

Your experience certainly doesn't match my experience; I've worked with brilliant QA people who were able to diagnose problems both when deviating from spec and when performance is a feature that isn't being properly met. And when someone has not been up to snuff, in almost every case I've been able to teach the people you arrogantly dismiss as "noise" and make them better able to help us all create better things through that education.

Your attitude, however, does match my experience: it is why people think software people are assholes. So be more mindful, please, from up there on your high, high horse. For yourself and for us as a profession and a craft.

Deciphering bad bug reports is absolutely a waste of your time as a developer, but that doesn't imply that the user is mistaken. In nearly all cases, the user is correct in sensing a bug but incorrect or inarticulate in their description of it.

It's almost always a good idea to understand what's behind the report. It's just that somebody in a different role should be doing most of that work.

That's what CS engineers and test engineers are for!

Small startups are often just a handful of people, and devs can and do pull double-duty as QA and development, and sometimes customer support for engineering issues! Freelancers also don't have customer support teams. For larger orgs, however, you're right. And also, like I said in another comment, the "customer" doesn't have to be your customer -- your "customer" could be the QA guy who says he found a bug, or your boss, or another engineer. Whoever tells you they found a bug, you should not dismiss their report with "well, it works for me."

> 99999 out of 100000 yes

If there's one thing I can't stand then it's people spewing fantasy numbers out of their ass without anything to back them up in order to give their argument some conjured up weight. You may well be right that a majority of bug reports are unhelpful, but please, if you need to use numbers, use ones you can back up with something.