Hacker News Reader

Comment by eminence32 | Hacker News Reader

I recognize that this is annoying from a user perspective, but I do understand it. Not all bugs are easily reproducible (and even if they are 100% reproducible for the user, it's not always so easy for the developers). Also sometimes you make a change to the code that you think might be in a related area, and so sometimes the most "efficient" thing is just to ask the user to re-test.

When I close an old bug that is not actionable, I do feel bad about it. But keeping the bug open when realistically I can't really do anything with it might be worse.

View on HN

Back in another part of my career I worked a lot with putting Macs on ActiveDirectory. And there was a common refrain from Apple about bugs in that implementation: "works on 17!".

The joke is that Apple owns the 17.x.x.x class-A range on the Internet (they got in early, the also have a second class-B and used to have a second class-B that they gave back), and what engineers were really saying is that they could not reproduce on the AD systems that Apple had setup (lots of times it was because AD had been setup with a .local domain, a real no-no, but it was in Microsoft's training materials as an example at the time...).

> keeping the bug open when realistically I can't really do anything with it might be worse

I've heard this from others before but I really don't understand the mindset.

What's the harm in keeping the bug open?

makes it very hard to lie with your metrics

I used to think that there is no harm in keeping the bug open. I think if you honestly feel that you have the time and resources to go back to the bug and fix it, then by all means keep it open.

But I find that sometimes I can tell from experience that the IR is not actionable and that it will never be fixed. Some examples:

* There's not enough info to reproduce the issue and the user either can't or won't be able to reproduce it themselves. Intermittent bugs generally fall into this category. * The bug was filed against some version of the software that's no longer in production (think of the cloud context where the backend service has been upgraded to a newer version).

Sometimes the cost to investigate a bug is so high relative to the pain caused that it just closed as a WONTFIX. These sometimes suck the most because they are often legitimate bugs with possible fixes, but they will never be prioritized high enough to get fixed.

Or sometimes the bug is only reproducible using some proprietary data that I don't have access to and so you sometimes have no choice but to ask the bug filer "can you still reproduce this?".

Computer systems are complicated. And real-world systems consisting of multiple computer systems are even more complicated.

I think asking someone if they can still reproduce an issue is valid. Especially if it was trivially reproducible for them, and now it isn't, that seems like a fine resolution, and the bug should be closed.

But in the other cases, closing the bug seems to me to be a way to perturb metrics. It might be true that you'll never fix a given bug, but shouldn't there be a record of the "known defects", or "errata" as some call them?

For your specific scenarios:

- lack of information on how to reproduce or resolve a bug doesn't mean it doesn't exist, just that it's not well understood.

- For the "new version" claim, I've seen literal complete rewrites contain the same defects as the previous version. IMHO the author of the new version needs to confirm that the bug is fixed (and how/why it was fixed)

- I agree there are high cost bugs that nobody has resources to fix, but again, that doesn't mean they don't exist (important for errata)

- Similarly with proprietary data, if you aren't allowed to access it, but it still triggers the bug, then the defect exists

In general my philosophy is to treat the existence of open bugs as the authoritative record of known issues. Yes, some of them will never be solved. But having them in the record is important in and of itself.

> It might be true that you'll never fix a given bug, but shouldn't there be a record of the "known defects", or "errata" as some call them?

Yes, fully agreed. But closing a bug doesn't preclude that. A closed bug isn't refutation or denial of a defect. It's just an indication that there is no plan to fix the bug. Not every bug system works like this though. My bug tracker works like this, and I should have more clearly described what a "closed bug" is in my earlier posts.

Bug reports are not known defects, at any kind of scale half of them will be already fixed, misunderstandings, bad data in, or related to an unusual setup.

Closing the bug is a way of saying: sorry this doesn’t look too important and we don’t have time to look at this given the other more important things (bugs/features) we plan to work on.

If it’s closed as stale after 6-12 months (multiple humans will have seen it) OR triaged by a human and marked as won’t fix I think that’s reasonable.

> at any kind of scale half of them will be already fixed, misunderstandings, bad data in,

Here you're referring to a class of bug reports that's uninteresting for this discussion, because they're invalid (i.e. they don't represent an actual bug). We're talking about valid bugs that have not been fixed.

> or related to an unusual setup

Unusual, but ostensibly supported? Then there exists a bug.

These rules are in place because that first class of 'bugs' is the majority of bug reports when you are at scale.

What does it cost you to keep the bug open?

Their team's bug close metrics

Attention very time you look at the bugs list.

That's what labels are for.

Surely a few years worth of open but unverified bugs would cause some issues with reporting and such.

What is the use in keeping it open when no one will ever look at it again after it goes stale? It still exists in the system if you ever wanted to find it again or if someone reports the same issue again. But after a certain time without reconfirming the bug exists, there is no point investigating because you will never know if you just haven't found it yet or if it was fixed already.

See my reply to eminence32 - bug tracking serves as a list of known defects, not as a list of work the engineers are going to do this [day/month/year].

The primary purpose is not usually a list of known defects and many ‘bugs’ are not actually bugs but feature requests or misunderstandings from users (e.g. RFC disallows the data you want my html parser to allow).

> The primary purpose is not usually a list of known defects and many ‘bugs’ are not actually bugs but feature requests

IME there are separate mechanisms to track feature work, bug trackers are for... bugs.

> or misunderstandings from users (e.g. RFC disallows the data you want my html parser to allow).

Again, this is a class of bug report that nobody is arguing should stay open.

The people who filed them would disagree and many would vehemently argue that their bug is in fact a bug, and is the most important bug and how dare you close it.

Craig shows up under your bed.

> What's the harm in keeping the bug open?

Conversely, what's the harm in closing the bug? (As long as you don't lock or delete it, I agree that's bad.)

People focused on the work often interpret "open" to mean "requires work" and "closed" to mean "no planned work" in which case keeping an unreproducible bug open is dishonest because it falsely implies that someone might continue to work on it.

Whereas people focused on the problem often interpret "open" to mean "not fixed" and "closed" to mean "fixed" in which case closing an unreproducible bug is dishonest because it falsely implies that it's no longer a problem.

Neither seems right or wrong as long as everyone on the project agrees which interpretation you're using.

How is that worse? Leaving it open signals to anyone searching about it that's it's still an issue of concern. It will show up in filters for active bugs, etc. Closing it without fixing it just obfuscates the situation. It costs nothing (except pride?) to leave "Issues (1)" if there is indeed an Issue.

To some people "open" means "not fixed" whereas to others it means "more work planned". I've worked on projects with both interpretations and it's fine as long as everyone is on the same page.

> It costs nothing (except pride?) to leave "Issues (1)" if there is indeed an Issue.

In our case we omit bugs we couldn't reproduce from the issues list due to practicality, not pride -- our software has tens of thousands of unreproducible bugs and having them show up in reports would drown out planned work.

And it's not like anyone deleted or locked the unreproducible bugs, they are either tracked as "open but unreproducible" or "closed because unreproducible". Either way they're still in the database in case more information comes along, but still filtered out of the vast majority of dashboards.

> Not all bugs are easily reproducible

Apple did not say they couldn't reproduce it. Neither did they say that they thought they fixed it. They refused to say anything except "Verify with macOS 26.4 beta 4".

> and even if they are 100% reproducible for the user, it's not always so easy for the developers

It's not easy for the user! Like I said in the blog post, I don't usually run the betas, so it would have been an ordeal to install macOS 26.4 beta 4 just to test this one bug. If anything, it's easier for Apple to test when they're developing the beta.

> the most "efficient" thing is just to ask the user to re-test.

Efficient from Apple's perspective, but grossly inefficient from the bug reporter's perspective.

> realistically I can't really do anything with it

In this case, I provided Apple with a sample Xcode project and explicit steps to reproduce. So realistically, they could have tried that.

I suspect that your underlying assumption is incorrect: I don't think Apple did anything with my bug report. This is not the first time Apple has asked me to "verify" an unfixed bug in a beta version. This seems to be a perfunctory thing they do before certain significant OS releases, clear out some older bug reports. Maybe they want to focus now on macOS 27 for WWDC and pretend that there are no outstanding issues remaining. I don't know exactly what's going through their corporate minds, but what spurred me to blog about it is that they keep doing this same shit.

A company like Apple should have complex enough tools to perfectly capture system state at the time of the bug so that they can reproduce it

That’s easy enough. The hard part is doing so without capturing a bunch of email, messages, and other private data that happens to be in memory at the time.

Ignorant question, if privacy didn’t matter and they had an atomically identical machine, would there still be plenty of edge cases where it was the printer or the Wi-Fi causing the issue?

In any case I would have said it sounds difficult on every front

I should be more precise. Capturing the system state isn’t too hard. Turning that into a reproducer may be quite hard, because of things like you say. There are certainly a lot of bugs that such a capture would make easier to figure out, but it wouldn’t be a panacea.

I don't work at Apple, so I can't comment on that. But that doesn't always help. There's been plenty of times where I have a full HAR file from the user and I can clearly see that something went wrong, but that doesn't always mean I can reproduce the issue. (I recognize a HAR file doesn't represent the complete state of the world, but it's often one of the best things a backend developer can get)

Reminds me of this Raymond Chen Microsoft blog post:

https://devblogs.microsoft.com/oldnewthing/20241108-00/?p=11...

It always helps. Even if you can't determine the root cause you can at least add an extra assertion check or logging statement at that point so that next time the bug gets triggered you'll at least get more useful diagnostic data and can get a step close. Iterate until you find the root cause.

Reply on news.ycombinator.com