Hacker News Reader

Comment by praveen9920 | Hacker News Reader

Here's the ground reality of "Surveys" and "Census" in India, The numbers are different from actual by a huge factor. There are multiple cultural and economical reasons for this, especially in villages.

My mom used to do census work in some of the villages in India, where people would give out questionable and unverifiable details based on benefits offered by the Govt. For example, a kid is reported to be staying with grandparents ( which is very common ) and also with the parents ( in a different village). Both of them avail benefits from Govt for the kid. India has no infrastructure to verify the validity of the data even with Aadhaar. It is a common occurrence to find people with multiple Aadhar or Voter cards.

So, the surveys do not give an accurate picture of India.

Edit: Typos

View on HN

While incorrect data being used for beneficiary fraud happens, census is india is limited in scope by law, and is not used for targeted benefits delivery - there is very little reason to lie here, since the data isn't correlated back with, for eg, your ration card.

NHFS (National Health Family Survey), used here, is a sampled survey (not a census), and has a detailed guide[1] on how they go about accounting for these biases, and ensuring their results remain representative despite sampling. afaik, the results from NHFS have the same limitation as a census - and it is not used for targeting benefit delivery.

You can look at the Questionnaire to get a better idea of how it was administered. It doesn't include any identity numbers (such as ration card or aadhar card), and starts with this:

> My name is _______. I am working with (NAME OF ORGANIZATION). We are conducting a survey about health all over India. The information on family welfare and health that we collect from households and individuals will help the government to plan health services. Your household was selected for the survey. I would like to ask you some questions about your household. The questions usually take about 25-35 minutes. All of the answers you give will be confidential and will not be shared with anyone other than members of our survey team. Your participation in the survey is voluntary. If I ask you any question you don't want to answer, just let me know and I will go on to the next question or you can stop the interview at any time.

While incorrect answers are a real thing, surveys can be designed around these problems, with both statistical analysis and better survey design.

[1]: http://rchiips.org/NFHS/NFHS5/pdf/NFHS%20data%20quality%20as...

[2]: http://rchiips.org/NFHS/NFHS5/schedules/NFHS-5Household.pdf

>> My mom used to do census work in some of the villages in India, where people would give out questionable and unverifiable details based on benefits offered by the Govt.

> While incorrect data being used for beneficiary fraud happens, census is india is limited in scope by law, and is not used for targeted benefits delivery - there is very little reason to lie here, since the data isn't correlated back with, for eg, your ration card.

Would that matter? That seems to imply a certain amount of legal savvy that I'm not sure should be assumed of anyone, including "villagers" who I assume are typically uneducated.

If I were engaging in beneficiary fraud (like the OP describes), I probably would tell the same lie to every government agent that comes by, unless I had a compelling reason not to. Staying consistent is safer and also far easier than sorting out the minimum amount of lying required to pull it off.

Try explaining this to common Indian villager. Ground reality is they see the same govt employees collecting any data and they have their own assumptions. Govt employees themselves do not do very good job of explaining anything.

> NHFS (National Health Family Survey), used here, is a sampled survey (not a census), and has a detailed guide[1] on how they go about accounting for these biases, and ensuring their results remain representative despite sampling. afaik, the results from NHFS have the same limitation as a census - and it is not used for targeting benefit delivery.

I assumed so. Its not the question of the process set for NHFS. On-ground reality is completely different from their assumptions.

> On-ground reality is completely different from their assumptions.

The people designing the survey are very well aware of the reality, imo. The survey is based on not just what the answers are, but also what the surveyer sees at the household. They have multiple questions to ascertain the actual number of people living in the household, including questions like "who all slept here last night".

Initial samplings of the survey are done with control groups (two separate teams) across different intervals to ensure that the methodology is good enough, and survey results are reflective of the ground-truth.

One way to validate the results would be to measure them with another methodology, and it turns out that the relevant result (India's Total Fertility Rate) has a fairly similar value (2.0 v/s 2.1)[1] via the SRS (sample registration system, 2020) which is conducted by another body (census of india) via a somewhat different methodology (tracking life-events by a local body, and using birth/death registrations alongside).

It's easy to give up and blame lying survey responders or sampling biases, but there are ways to solve for these, that are being used here.

[1]: https://timesofindia.indiatimes.com/india/fertility-rate-dec...

Are they consistently biased one way? Multiple Aadhar would seem to bias down, while the double-child would bias up.

The first censuses in the US were basically "a dude rides into town, asks how many people live there, then leaves" and got surprisingly close to recent estimates.

I just wonder if this is a matter of "there's error everywhere, it's basically stochastic, the # is close enough"

India is a vast country with different cultures and states with varied benefits, so it is hard to say which way the bias is. The only thing I can say is they are not accurate.

That's the super power of statistics though.

Getting accurate enough numbers from unreliable sources.

Garbage in, garbage out.

Incomplete data might be usable if the underlying distribution is the same, and there is reason to assume it is the same. But that does not help if the data is bad in the first place.

I'll give you a concrete instance where that was not true in a "stochastic sense".

In the mid 2000s the Indian government wanted to test its water at the local level. A lot of time and money went into purchasing Water test kits. They set up large agencies and funded offices to be built all over the place for water points of access to be tested and the data to be written down aggregated and sent off. For quite some time the Indian government believed they had IMPECABLE water, how could that be because ... they didn't. When these offices were audited in the early 2010s it was found most of them hadn't even OPENED the test kits and were just faking the data, much like you might on your intro chem lab.

Source: Me I worked on operationalizing that data collection and the engineering instrumentation of the test kits to literally track people to make sure they just did their job and what a cluster fuck that all was.

What is my point though, before believe government oriented data I'd have to know something about their methodology and how it's verified. Fraud does exist whether explicitly or emergent based on some factors cultural, bad incentive structure .. ect and you can't depend on stochastic hand waving to say it's close enough. Having said that you probably COULD depend on stochastic MCMC type analysis to tell you the likelihood your measured sample data is real.

>>> is a common occurrence to find people with multiple Aadhar or Voter cards.<<<

Voter cards yes, but AADHAR : No. I haven’t heard anyone still able to have two Aadhar number which are based on biometrics (Fingerprint + Iris Scan).

As for the accuracy of survey data, at least in new village, I know it to be nearly correct. Not grocery incorrect.

Probably true to some extent. but Im sure there are "agencies" out there which provide "Aadhaar card" services which does some shady sh*t if you need duplicate aadhar.

> It is a common occurrence to find people with multiple Aadhar or Voter cards

This is false. Voter cards perhaps, but not Aadhaar cards. It is technically not possible because the cards (identities) are tied to biometric data (fingerprints and retina scans) and would be immediately caught by the system if the same person were attempted to be registered in the database again.

Edit: In fact, because of this many lost kids who were tried to be registered by orphanages have been found and returned to their homes!

https://m.economictimes.com/news/politics-and-nation/about-5...

Reply on news.ycombinator.com