Hacker News Reader

Comment by burkaman | Hacker News Reader

Personally I don't have a problem with automatically sending the type of telemetry that makes detecting this possible, but Mozilla could also discover this issue by in-house testing on a wide variety of devices, and/or prompting users to submit telemetry after a crash.

It's not feasible for a smaller developer, but it wouldn't be unreasonable for Mozilla to have a big lab bench with like 100 different devices on it that are just constantly running a script to start Firefox and load a suite of test sites.

View on HN

I don't think you could really approach the kind of coverage automatic crash telemetry can give with a bunch of test devices. You will absolutely find a non-zero number of issues with the test devices, but it's not really the same.

Without substantially increasing the cost and work to maintain the test pool, you'll be testing exactly one combination of OS version/patch level/settings, one internet connection medium/carrier, one set of browser config options, etc for each device in the pool.

The investigation into any fault isolated to a single device in the test pool has to start with suspicion of the health of the specific device. The test S20 crashing doesn't tell you that every S20 will crash in the same way, it could be an isolated local fault. Yes you can throw more devices but that's further increasing the cost and effort of maintaining the test pool.

It's not remotely feasible to test more than a tiny fraction of the variety of devices, configurations, and sites that a product like Firefox is going to encounter in the real world.

I agree you'd never catch everything, but if it's crashing on the latest Samsung flagship phone you could probably catch that.

User volunteered crash telemetry is better than nothing, but automatic crash telemetry is going to have a more representative sample. A compromise would be to grab just a stack trace and phone model / firmware version etc automatically, and a full dump with user affirmative consent; but if that's not compatible with your privacy stance, asking everybody for everything is reasonable but limiting.

Android has an immense variety of devices. Without a representative sample, you wouldn't know which devices to buy. Regional variants sometimes have a huge difference and sometimes not; this bug seems to be tied to non-US Samsung flagships, which you might not buy if you're US based and don't know what people are using.

Reply on news.ycombinator.com