Hacker News Reader

Comment by andrewguenther | Hacker News Reader

> This bug was extraordinarily rare, and because we have no metrics/remote log collection, there was an initial period where we had to spend time adding logging and collecting user-submitted logs to try to track it down.

Without telemetry, can you actually back up the claim that this issue was extremely rare?

View on HN

Some details on how this assumption was made would be nice, but I think it's pretty obvious that any developer involved in a project can make a reasonable assumption of how rare a bug is depending on the technical details on what is required for the bug to happen. For example, if we say for the sake of argument that a hypothetical bug requires you to have more than ten contacts of the exact same name and these also need to share the same country and area code, one can make the assumption this use case is very rare without knowing the exact number of users that this applies to, just based on common sense regarding how the application is normally used.

edit: The linked github issue says:

> The TL;DR is that if someone had conversation trimming on, it could create a rare situation where a database ID was re-used in a way that could result in this behavior. It was very difficult to track down, with earlier phases involving getting additional logging into builds. Once we had some more information, it did in fact become our top priority, a fix was made, and we got it out as quickly and as safely as possible. The fix itself should make it so that database issues like the one that caused this bug can't happen again.

>for the sake of argument that a hypothetical bug requires you to have more than ten contacts of the exact same name and these also need to share the same country and area code

This is only rare if you have a small social circle. My circle has multiple first name-collisions of at least 5 participants, but my circle is not very big and the area code is also quite small.

Some countries do not use area codes for mobile phone numbers, which are used for Signal, meaning country code is the only area limiting factor.

>for the sake of argument

Yes.

And for the sake of the same argument I made a counter argument, stating that some initially believed to be rare circumstances are actually not that rare.

"For the sake of argument" does not mean "I invite you to debate this", it means the exact opposite: "let's assume this is correct/we agree/etc. for a while and debate what comes after". So in this case you are doing the exact opposite of what the phrase "for the sake of argument" is requesting.

https://idioms.thefreedictionary.com/for+the+sake+of+argumen...

> I know you want to go to Stanford, but just for the sake of argument, let's talk about what some of the other schools you got into have to offer.

This does not mean "I invite you to make a counter-argument as to why no other schools than Stanford are worth going to".

It sounds to me like NullPrefix accepted the hypothetical bug, for the sake of the argument, and then addressed the rarity assumptions made about that bug. "For the sake of argument" doesn't mean "No debating" it means "Let's assume some thing x is taken as a given and go from there."

The whole point was that "given this assumed rare bug, one can know it's pretty rare". Whether the example is actually rare enough is completely beside the point. If it's not rare enough for you, replace it. The argument comes next: given a rare bug, a developer can make an assumption of how rare it is. Debating the example of rareness is splitting hairs.

There is no argument, and it did not require a counter-argument, because that is not the point. He could have used literally anything else, it just has to be rare.

Well he just wanted to make an example. One could also construct an example, where the bug only occurs for people with a rare sequence of unicode symbols (e.g. U+2600 U+2601 U+2602) in their username and have a specific date (e.g. 05.04.1920) as their birthday.

Your argument depends on Signal implementing username support, because we do not support unicode in phone numbers.

My argument depends on an alternate universe in which signal supports it, everything else being equal, because why not.

Does that alternate universe support phone numbers with unicode characters too?

You are arguing against a hypothetical situation of your opponent's creation. This is like when Ross tried to beat Chandler at Cups

I go to bed now and dream of a universe in which NullPrefix is Ross and continues to argue endlessly.

I'll argue with anyone about anything. For free.

> I'll argue with anyone about anything. For free.

No, you won't.

Obviously that was a hyperbolic statement.

Don't argue with me!

I am curious how different that alternative universe needs to be for my argument to be invalid.

It's not that your argument was incorrect. It's that it is tangential to the parent comment. hnarn was not making the point of something being rare, they were demonstrating that a developer can estimate the rarity based on the conditions that trigger it. The rarity itself is no matter.

Arguing over small semantic or circumstantial differences is often considered impolite

It is a hypothetical scenario for the purpose of an example. It could have been literally anything, the point is for it to be rare.

Yes, but the fact that scenario is hypothetical doesn't counter my argument that whatever the developer thinks is rare is actually rare.

I do not know why I keep replying to this... but regardless, here it comes: your counter-argument is irrelevant! It does not require any counter-argument, because it is hypothetical! He could have said anything else that is rare, something that you think is rare, too. Just assume he said something that you think is rare as well. It is a hypothetical scenario, so you can do that; in fact: you should have done that, because all that is important is that it is rare, whatever that is. Again, argument over what he said is rare or not is not needed. Just assume it is rare, or assume he said something you think is rare.

I hope this gets the point across. Yes, maybe what he said is not rare, but that is besides the point.

> The fix itself should make it so that database issues like the one that caused this bug can't happen again.

I've said that a few times, and been wrong about it. :D

I don't know what they mean by "where a database ID was re-used", but I guess it had to do with caching, and cache invalidation is one of the hardest part in computer programming.

> cache invalidation is one of the hardest part in computer programming.

Yes, famously it is one of the two hardest parts of computer programming, the other being naming things, and off-by-one errors.

> a hypothetical bug requires you to have more than ten contacts of the exact same name and these also need to share the same country and area code

Sounds like the village a part of my family tree comes from.

Why would anyone think that re-use of ID was a good idea?

I'm not defending the practice of re-using a database ID, but it's absolutely something that can happen (or change) as an oversight rather than an active choice from the developer.

Perhaps that’s why it was a bug?

> just based on common sense regarding how the application is normally used

Just based on assumptions of how the application is used.

"falsehoods programmers believe" comes to mind. The uniqueness of names in your example could be false in some (or many? No idea) cultures.

> it's pretty obvious that any developer involved in a project can make a reasonable assumption of how rare a bug is

... is it? The fact that a bug exists means there's a logic gap. You can try and patch it with theory, but that's just adding assumption to a scenario created from broken assumptions. Also, the job of telemetry in incident reporting isn't to be vague - its to add precision.

There's probably a ratio of bug-report-to-occurrences that they're used to for difference kinds of bugs. Ex: If user-visible security bugs have good report rates, say 100-1000 leaks per 1 report, and 10 reports, then 1K-10K incidents. This is harder in b2b, but in b2c, PM's should have a feel for it..

You’d be surprised how difficult it is to estimate the frequency that someone sees a bug. The only way to have a “feel for it” is to base it on… other data.

Say there is a bug that happens in the photo taking flow – you’d need to know how often people take photos in Signal. You’d think you could spitball something for that, but it is actually really hard. But if you log how often photos are taken, then that is a great starting point.

But further, lower level logic errors like this, especially ones involving race conditions, are even harder to pin down. That is why on iOS you can log “faults” which are non-fatal but very not-expected events:

https://developer.apple.com/documentation/os/logger/3551617-...

They generate reports with stack traces that you can use to a) judge the prevalence of an issue and b) see where it originated

If you know the reproduction steps and you can look at the code, you should be able to make an estimate of rarity.

Even with telemetry, would you be able to back up the claim that that the issue was rare? All too often, people add telemetry to something in an attempt to try to find things like this, but they end up leaking things elsewhere and not accounting for them. Bad statistics are more dangerous than no statistics, and all that.

If this is the first instance of this bug being reported, then yes it's rare.

The rarity of a bug is independent from reports of it.

That also depends on the severity of the bug. :)

No, those are directly correlated.

The reports of it are correlated to the rarity, except in instances where reports are not generated, such as apps that do not collect error telemetry.

Users can make reports manually, and you would even expect them to do so at a high rate in bugs like this. This implies a high correlation between bug reports and rarity.

Telemetry does not matter in this case. If a bug occures often, then either the issue ticker will get this bug often, or there might even be a "known bugs" section.

I would like to think so. I used a program that is full of bugs, yet those are not reported[1], nor have they been fixed. :(

Edit:

[1] I did report one (or two?, it has been some time ago) after some time (could have been months, because I could do without it as a compromise for that time being). They did not respond nor did they fix it (so I did not report any other bugs I have found). I recently re-installed it to give it another go, and the easy-to-fix bug still occurs. They just do not give a damn. Their website has been rebranded and everything, but this bug has not been fixed for over a year.

Edit #2:

Wow, I was way off with the "over a year". I checked the bug report of the one I remember reporting. It does seem like some other people have reported it too. Well, it is a bug from 2018, after all. Still open, no word from a developer. :D It is a bug of a crucial feature.

In any case, if for some reason I had to use this software again, I would never report any bugs because it is pretty much futile. They do not answer, they do not categorize bugs, they do not seem to check the bug reports at all. It would be a waste of time.

Then why not report them?

Because the bug happens when you're trying to get something done and you don't have 5 spare minutes to interrupt your flow in order to file a bug. You might make a mental note to report it later and then forget. Or, if the bug is annoying enough, you might just never use that program again and you don't care if it gets fixed. Or your wifi glitched at the same time the bug happened and you've no idea if it is a bug or the glitchy wifi. Or you don't know it's a bug until later and by that time you've no idea what you were doing to trigger it.

There's a million and one valid reasons people don't report bugs.

Even with, what telemetry would allow you to determine how many pictures (and presumably content) you sent to whom?

You can fix the bug but emit a metric on the original code path to estimate the impact after the fix is made. It wouldn't tell you who sent what to who, but would give you a better idea of impact for an incident report or postmortem.

I read it as bugs of this severity are extremely rare. Which claim doesn’t require telemetry.

Reply on news.ycombinator.com