If I want to survey the effectiveness of Post Office in the era of Internet (email, messenger, etc), I'd definitely interview mainly the visitors to the Post office to check their their purposes, experiences and activities there. For example, you'll probably find someone posting a postcard inside the post office since that someone is probably a visiting tourist.
If you wanted to know what parents think of public schools, would you survey only the parents who currently send their kids there, or would it also be informative to know the opinions of the parents whose children educated elsewhere?
And even if someone has never used X, it doesn't mean they haven't done extensive research about X, before concluding X doesn't meet their quality bar.
Before deciding whether to send my son to an SFUSD school, I had a call with the principal of the school and asked questions specific to my son's situation. Do you think my conclusions are worthless for understanding the quality of the service?
Also, while I get your point, you're in the very unique position of informed non-users. That's a valuable category if you can access it, but it's the most difficult. (How can you determine which library non-attendants have done their research?)
Not useless by any means
"If you're looking to understand the quality of a service, non-users are worthless"
given the choice between somebody who interviewed the principal and a somebody else who's son actually attends, I'd prefer the latter every time. Wouldn't you?
"If you're looking to understand the quality of a service, non-users are essentially worthless."
I'm suggesting that, in absence of some out-of-band signaling mechanism, the expectation value for usefulness of individual feedback is orders of magnitude greater for users than non-users.
Put differently, if I happened to be interested in this SFUSD school as well, is there some list of people like you I should interview?
Put differently, if I happened to be interested in this SFUSD school as well, is there some list of people like you I should interview?
I'm suggesting that, in absence of some out-of-band signaling mechanism, the expectation value for usefulness of individual feedback is orders of magnitude greater for users than non-users.
If you got to the post office to find out about people's experience, your sample will be biased: you will mostly be interviewing people who use the post office much more often than the median post office user.
I go to the post office 1-2 times per year. People like me will be both:
- underrepresented in your sample, and
- be the least likely to have adapted to any difficulties getting to know how the post office works
Some folks who are heavy users of parcel services will have stopped using the post office completely. They will be totally unrepresented in your sample, even though you'd like to know why they use UPS instead of USPS.
Having a relevant Facebook group is about the best case scenario. I guess email might work? Maybe letters? I can't imagine calling or canvasing.
On top of all that I still think the data from actual users is still going to be more valuable overall, even the negative stuff. I'd prefer somebody who attended the school and had a bad experience than somebody who had a bad impression and went elsewhere.
But... I'll agree it's not worthless. It just strongly depends on the type of data you want to collect. For impressions and onboarding, it's a different story. Here's the goal of this study:
> NYPL regularly surveys its patrons to understand and improve how the Library fits into—and adds value to—their lives. We strive to identify the unique power of public libraries, pinpointing precise mechanisms of positive impact, so that we can preserve and strengthen that impact.
If you believe that public libraries serve some purpose, you have to start with that, rather than just using gradient descent to get to somewhere. Otherwise why wouldn't libraries evolve into hospitals, or podcast studios, or brothels?