Ask HN: How do I know my software is working for my clients?
Every time I release an update, I get complaints about various new features not working as expected, which is normal.
I try my best to address these issues with software updates, but since the software is growing more and more complex, this approach is getting less and less successful.
My way to deal with software updates is to listen to as many reports as possible, figure out common problems and try to address the biggest problem with a successive update.
However, I'm starting to realize that people only reach out and complain if they're actually having a problem (why send an error report if the software is working correctly?) Because the user-base is growing bigger, the chance that a person is experiencing a one-off problem -- a problem specific to their setup, environment or handling -- increases, while the chance that theirs is a common problem decreases.
So I know that _some_ users are experiencing a bug, but I don't know if it's a common bug, a rare bug, a design error or if the feature is even working for anyone at all!
I'm unsure about what to do here. The error reports I get are vague because of my closed-source "if you have a problem, it is the developer's problem" business model/mentality, client-side log files up to this point haven't been detailed enough, although I am working hard to improve that part a lot.
Is collecting anonymized user-data the answer to a problem like this?
TL;DR As my software grows more complex and the user-base increases, I start having more uncertainties about dealing with updates, stability, vague bug reports and "fixes".
Have any of you had similar problems? Do you know of any good resources dealing with software stability? Or does this sound like a clear indication of <insert common mistake>?