Hacker News Reader

Comment by causi | Hacker News Reader

What I found interesting is that his analysis did not pick up the (now strongly evidenced) degree of inaccurate reporting from India. Indications are that India's covid deaths are between six and seven times higher than reported yet it looks pretty good on the Benford chart.

View on HN

It really depends on how you're manipulating data.

If you're just making up numbers, Benford's law could pick it up (unless you're aware of Benford's law and create numbers to follow it).

However, if you're just counting a sixth of all deaths, then your numbers will follow whatever pattern they were following before, just lower across the board.

The general form of Benford's law also fails in situations where numbers are capped for some reason. You'll wind up with a strong grouping around that particular number.

All in all, Benford's law can never prove anything. At most, it can give a good reason to look into a situation.

This is an excellent point.

The CDC estimates that in the U.S., there are about 32 cases of undetected covid-19 for every 10 that are detected. That ratio (derived from testing of pregnant women, car crash victims, blood donors, etc.) was from before Omicron, when there was for a while a shortage of testing capacity. So, if the ratio in India is 60 to 10, while that is worse than in the US, it's not that much worse.

In any case, Benford's Law helps to detect fraud (made-up numbers), not necessarily just inaccuracy (a low detection percentage, for example).

They're talking about deaths, not cases. For a case to go unreported is one thing, for a death not to be is much worse.

From my coworkers in india, I think that India's reporting has been less to do with inaccuracy and more to do with their entire healthcare system just crumbling under the pressure of covid. Far too many people getting (and dying from) covid to accurately measure and capture the true impact. It was real bad.

Yes, lots of people died in rural areas and were buried without ever getting near a doctor in the Delta wave, so they weren't recorded.

Benford would likely detect manipulation, but not systemic underreporting. If they're just missing half the cases and accuracy is essentially arbitrary, then it probably wouldn't show up as violating Benford.

Not saying India did this, but I would imagine someone aware of Benford's law could tweak numbers to look valid with it taken into account, right? The more folks talk about "I used Benford's law to spot malfeasance", the less useful it becomes.

It depends how much the tweak implementer knows about Benford's law. If you look through the figures in the Appendix of https://econwpa.ub.uni-muenchen.de/econ-wp/othr/papers/0507/... , a lot of people get the first and sometimes second digit behavior right yet fail to properly falsify the less significant digits. Of course, this sort of analysis might fall apart once you've summed 20 or 40 or 300 days of data.

Benford's law is sort of like the Bechdel test. It's trivial to evade if you know what algorithm for which to optimize; it's value is in demonstrating the surprising number of places where it fails.

I'd you divided all your measurements by 7, I'm pretty sure both datasets would still align with Benford's curve.

I'd you divided all your measurements by 7, I'm pretty sure both datasets would still align with Benefords curve.

Reply on news.ycombinator.com