How an arcane statistical law could have prevented the Greek disaster | Hacker News Reader

How an arcane statistical law could have prevented the Greek disaster

economicsintelligence.com

96 pointszalthor31 comments

Open articleView on HN

Other fun I've had with Benford's Law.

1. Spotting odd things in MPs' expenses: http://blog.jgc.org/2009/06/its-probably-worth-testing-mps.h...

2. Spotting odd things in BBC executives' expenses: http://blog.jgc.org/2009/06/running-numbers-on-bbc-executive...

3. The Iranian election: http://blog.jgc.org/2009/06/benfords-law-and-iranian-electio...

4. New Age mumbo jumbo: http://www.jgc.org/blog/2008/02/any-sufficiently-simple-expl...

Benford's law does not apply to some of those things! In particular not to expenses.

Benford's law only applies to things that experience exponential growth, things that obey a power law distribution, and things that are totals of random processes.

Expenses are none of those.

Benford's Law has for some time (years, in fact) been used to look for anomalies in expenses and other corporate financial data: http://www.journalofaccountancy.com/Issues/1999/May/nigrini

How could expenses not have a power law component to their distribution? Sometimes you're buying an expensive thing, and sometimes a cheap thing, and you don't buy something between $999 and $200 dollars nine times as often as you buy something between $199 and $100.

If you have a 40$ meal limit on your expence reports you would expect to see a lot more 3X$ meals than 1X$ meals.

Yes, you will certainly get artifacts at the top of what you can expense. You will probably also get distortions at the bottom where you'll have some threshold of "not worth expensing." There will almost certainly be distortion around the boundaries, but as long as you have a big enough range you should be able to see a Benford's law effect in the middle.

It's going to apply to anything that has a probability distribution that is monotonically decreasing at any rate, exponential or not.

This is pretty obvious: the probability of the number 1 will be greater than the number 2, and so on up to 9.

The probability of the range 10-19 will be greater than the range 20-29, and so on.

The probability of the range 100-199 will be greater than the range 200-299, and so on.

And so on... The probability of the ranges starting with "1" will always be higher.

Also, I have no idea why anyone would think this is "arcane". It's very simple and obvious and a good example of where intuition can go wrong.

It applies to most quantities that can be represented in multiple units or where no unit is a natural one.

Benford's law would not have stopped the groupthink, willful ignorance and corruption which actually drove the Greek disaster. It's not like Greece's problem with tax collection was a well-kept secret, it's just that nobody wanted to face the facts.

I seem to remember at the time there was widespread reported skepticism about a number of countries' data. It was just accepted that the central European elite wanted to expand to sell their wares to these dodgy countries and prevent conflict. It shows how dependent the lender is to the lendee and how both sides don't face up to the reality.

Also, Greece’s original accession to the EU was motivated in part by geopolitical considerations; Greece applied for membership shortly after a democratic government replaced seven years of harsh military rule.

The Tim Harford article mentioned in this submission was submitted and discussed a couple of days ago:

http://news.ycombinator.com/item?id=3008848

The similarly mentioned Ben Goldacre article was also submitted, but that got no discussion:

http://news.ycombinator.com/item?id=3007964

A detailed discussion by Terry Tao of Benford's Law (and more) can be found here:

http://terrytao.wordpress.com/2009/07/03/benfords-law-zipfs-...

The issue with using Benford's Law with analysis is that as soon as people realize that you're using it, data can easily be manipulated to make it compatible.

The previous article by Tim Harford mentioned that Madoff's financial data was Benford compatible. So I doubt that using it would have actually prevented the Greek or any other disaster.

The article says the opposite, though it's hard to put a lot of credibility on something starting with "according to statisticians"...

According to statisticians, it is almost impossible to manipulate data in a way that a certain outcome is guaranteed and Benford’s Law is met at the same time. Hence, tax authorities in several countries are using Benford’s Law as a default testing device.

I re-read the article and you're right. The article seems to imply Madoff as an exception rather than the common case, but I still feel that if data is being manipulated at such a vast scale where an entire countries economic activity is misrepresented, you will be able to find people as smart as Madoff to 'fit' the data as per requirements.

You certainly can find people smart enough to make sure that their fraudulent statistics will meet Benford's Law (or any other data model you wish). The article points out that Greece didn't find people this smart.

Not meeting Benford's Law is a good indicator that there has been some dodgy dealing. However, meeting Benford's Law is not a good indicator that everything is above board.

You're looking at it from the opposite point of view than the one I'm making.

The reason Greece didn't find people smart enough to satisfy Benford's is that Benford's wasn't being used in the audit.

If it was common knowledge that audits utilize this Law, you can bet that Greece's data would have been compatible.

Nice geek porn, but the title and the premise is basically nonsense.

EU politicians knew for many years that Greece was scamming things. Maybe not so much when the Euro started, but well before it all blew up. All kinds of circumstances, however (such as the need for Greece's vote on some other hot issue of the moment) made sure that no one had the will to really do something about it.

Statistics is nice and dandy, but it doesn't help a dime in fixing the broken mess that is EU politics.

ps. Fun trivia: if the EU applied for EU membership, it wouldn't get in because it's too undemocratic.

It's nice that they made sure the data set contained the key data to their collapse. The pensions of retirees. Would the results have been the same if it weren't included? I don't have access to the paper but I'd be curious what their justification for their choice was.

Among other things, they looked at the total level of debt, the cash reserves of the government and the pensions of retired civil servants.

Radiolabs had a segment dedicated to such analysis. It appears the IRS/accountants use Benford law patterns to audit tax evaders.

http://www.radiolab.org/2009/nov/30/from-benford-to-erdos/

Are there numbers from other nation's economies/ledgers that someone's run by Benford's Law?

Not sure if you skimmed the linked article and missed this or not:

statistics from the Czech Republic, Sweden and the UK meet the Benford distribution particularly close and hence appear unsuspicious. All those countries have no interest in joining the Euro and therefore do not have to meet the convergence criteria. On the other hand, statistics from Latvia, Belgium and Romania (the latter notorious for its corruption) appear very suspicious, as well.

And from Tim Harford's article in the FT: http://www.ft.com/cms/s/2/171aaa36-d8f1-11e0-aff1-00144feabd...

Romania, Latvia and Belgium also have abnormally distributed data, while Portugal, Italy and Spain have a clean bill of health.

can you apply Benford's law to only 130 samples?? that's not a large set of data according to my book

As I read it, they examined 130 different time series. The article did not say how many observations were in the sample for each time series.

"They looked at 130 different values per country and year", they don't say how many years were studied but I assume the greece book fixing lasted only a few years so we are looking at ~500 samples. I honestly don't know if that's enough or not, on wikipedia they show examples of the distribution on less than 100 samples so I guess I'm on the wrong.

The article says:

> According to statisticians, it is almost impossible to manipulate data in a way that a certain outcome is guaranteed and Benford’s Law is met at the same time. Hence, tax authorities in several countries are using Benford’s Law as a default testing device.

I'd be very interested in evidence to support this. This doesn't sound correct. "Almost impossible?"

Arcane ?

My thoughts exactly. Between reddit and HN I see a Benford's Law post every month or so.

Benford's Law is pretty well-known in mathy circles.

Although in this instance I agree, I think the bigger point is that you shouldn't call anything "arcane" on HN. Because it'll be well known to someone and that someone—who is probably someone who could contribute a lot to the HN discussion—will think poorly of you.

Come on, this is not even a real law, and the data are really shaky to justify the title of this post. 5% significance threshold means that Luxembourg or Austria satisfy the criteroa. So what? The EU is a large political organization, not a NYSE index. The real trouble for greece is not the fake accession data (Italy had similarly manipulated data), but the squandering of public money that followed.

I get it, geeks love to play with empirical laws, but this paper is at best interesting trivia. I could go on analyzing what could really prevent the disaster forever ...

Reply on news.ycombinator.com