Chi-square goodness-of-fit, Benford’s law, and the Iranian election | Hacker News Reader

Chi-square goodness-of-fit, Benford’s law, and the Iranian election

incanter.wordpress.com

31 pointsmariorz18 comments

Walter Mebane uses Benford's Law on the second digit as a more reliable system to detect electoral fraud: http://www.umich.edu/~wmebane/pm06.pdf Furthermore, it is interesting to note that he pointed out in 2006 that the results of the Mexican presidential election seemed fraudulent and he recommended a recount. But at that time the mainstream media seemed quite happy with the official winner, so they didn't talk about it, even when over two million Mexicans protested in Mexico City over the election results. The mainstream media decided to ignore all the statistical evidence of fraud, and just label the other candidate a a "sore loser".

Commentators are “explaining” the Iran elections based on their own illusions, delusions, emotions, and vested interests. Whether or not the poll results predicting Ahmadinejad’s win are sound, there is, so far, no evidence beyond surmise that the election was stolen. However, there are credible reports that the CIA has been working for two years to destabilize the Iranian government.

On May 23, 2007, Brian Ross and Richard Esposito reported on ABC News: “The CIA has received secret presidential approval to mount a covert “black” operation to destabilize the Iranian government, current and former officials in the intelligence community tell ABC News.”

On May 27, 2007, the London Telegraph independently reported: “Mr. Bush has signed an official document endorsing CIA plans for a propaganda and disinformation campaign intended to destabilize, and eventually topple, the theocratic rule of the mullahs.”

A few days previously, the Telegraph reported on May 16, 2007, that Bush administration neocon warmonger John Bolton told the Telegraph that a US military attack on Iran would “be a ‘last option’ after economic sanctions and attempts to foment a popular revolution had failed.”

On June 29, 2008, Seymour Hersh reported in the New Yorker: “Late last year, Congress agreed to a request from President Bush to fund a major escalation of covert operations against Iran, according to current and former military, intelligence, and congressional sources. These operations, for which the President sought up to four hundred million dollars, were described in a Presidential Finding signed by Bush, and are designed to destabilize the country’s religious leadership.”

--> http://www.counterpunch.org/roberts06192009.html

Yep, a full blown infowar is what we have here.

That's what I don't understand: Weeks before the election several polls (including ones by Western newspapers) predicted a 2 to 1 win for Ahmadinejad. These predictions were all fairly uncontroversial as far as I could tell.

Now suddenly everywhere I look there's stories about young photogenic Iranians getting shot at while protesting the election results. I keep hearing about claims that the election was rigged but so far I've not seen anything concrete.

The only way I can make sense of this is by assuming infowar / agent provocateur / other covert tactics employed by the USA. Iran has been a thorn in the side of America's middle-eastern foreign interests for a long time, and poking holes in it's democratic process is an important step.

I can't even visit Reddit at the moment, every second story is about Americans getting angry at Ahmadinejad.

Sorry, did I miss something? Protesters are being shot and killed in Iran, right? American progressives haven't deluded themselves to the point where they think this is just as likely to happen in today's America as it is in Iran, have they?

i think the reason some people are throwing around the cia/black-ops hypothesis is that all they've ever heard about iran is in the last couple years. which basically meant pursuit of nuclear weapons and a holocaust denying Ahmadinejad. but i think it might help to put the whole thing in a broader context.

what we're seeing right now is another chapter of the iranian revolution. it's been a long story that it started in '79. and these protests themselves are morphing from simply an electoral protest to one that is a continuation of the student protests during khatami's presidency a decade ago.

america has had zero to no presence in iran, much less any tangible way of 'destabalizing' the regime. the money the cia has been throwing at this problem for the last decade has been at things like VOA (voice of america radio broadcasts inside iran) or tacit moral support for minority separatists in iran.

this is not to deny that the west has huge stakes on the outcomes of this, as evidenced by the US government's intervention with the twitter downtime thing and the significant media coverage, or maybe even fake twittering and blogging.

but given the choice of whether an autocratic paranoid regime has rigged the vote to suppress its young population OR a secret cia operation is pulling the levers of what by all accounts seems like an organic grass roots social uprising--i choose the former.

All of the election polls that I'd seen predicting a 2-1 win also had 40-60 percent of the respondents not responding. That they all broke for Ahmadinejad seems... unlikely.

Hello,

I am putting together information on the Iranian elections for a report I am doing on the political risks of international investment. The information that I have collected thus far indicated that the highest rate of non respondents in the indicated polls was 43%. That was the rate for the Washington Post poll. Would it be possible for you to share the polling data with me for the polls with non respondent rates higher than 43%. That data would be EXTREMELY helpful to me.

If 40% were undecided (leaving 40% / 20%), that still means that at least three quarters of all the undecideds had to vote against Ahmadinejad for him to lose.

I've not seen any polls with 60% undecideds.

>[polls predicted Ahmadinejad victory]

I've seen the argument that people in Iran tend to not have "unpopular" opinions if asked in polls. (I don't know if the population would have been that nervous, considering that only four candidates were allowed out of more than 400.)

Anyway, the opposition party seemed to have a good few last weeks of campaign. Also, there are some bad smells... the uniquely short time to publish results, the way different minorities switched totally from the previous election, etc.

In non-democracies, there are lots of conspiracy theories. (As the joke goes, the population live in a conspiracy theory!) So maybe there weren't outright cheating and the election was "fair", but I'd rather buy a bridge than bet money on that.

This recent news flash seems to give more solid evidence of electoral tampering:

"Iran's Guardian Council says the number of votes collected in 50 cities surpass the number of people eligible to vote."

Since the Guardian Council is a highly conservative organization, it can be reasonably surmised that any possible fraud would have to be rather significant for them to make a statement about it.

To read a highly detailed analysis of the election, and signs that it could have been stolen, read this recent piece from The Economist: http://www.economist.com/world/mideast-africa/displayStory.c...

"Chatham House and the Institute of Iranian Studies at the University of Saint Andrews said that in two conservative provinces, Mazandaran and Yazd, the number of votes cast exceeded the number of eligible voters."

"The study also said that the official results suggest Mr. Ahmadinejad won the support of 47.5 percent of those who backed reformist candidates in 2005. The study's authors called that figure "highly implausible."

This is the original study from Chatham House & The Institute of Iranian Stuides, St. Andrews Uni:

http://www.chathamhouse.org.uk/files/14234_iranelection0609....

Wow -- incanter looks terrible. All that code to run a chi2 test and a plot?

The plots and chi-square tests are a single function each, it's the data preparation that takes up so much space. Alas, most statistical analyses involve a lot of preparatory steps, which are rarely shown in the final write-up.

Since I recently did something similar in R:

   d <- read.csv(file='~/stuff/earlh/Iran_2009.csv', header=T, sep=',')

   lastDigit <- function(v){
   	v - 10*floor(v/10)
   }
   
   digits <- lastDigit( c(d$Ahmadinejad, d$Karroubi, d$Mousavi, d$Rezaee))
   hist(digits, breaks=10)
   
   #chi2 gof
   tab <- table(digits)
   n <- length(digits)
   
   model <- chisq.test(x=tab, p=rep(0.1, 10))
   model
   
   # hand generated -- check our work above
   ts <- 0
   for(i in 1:length(tab)){
   	ts <- ts + ( tab[[i]] - 0.1*n)^2 / (0.1*n)
   }
   qchisq(p=1-0.076, df=9)

and to be more specific:

  (def regions (sel votes :cols "Region"))
  (def ahmadinejad-votes (sel votes :cols "Ahmadinejad"))
  (def mousavi-votes (sel votes :cols "Mousavi"))
  (def rezai-votes (sel votes :cols "Rezai"))
  (def karrubi-votes (sel votes :cols "Karrubi"))
  # -or-
  attach(d)

  (def ahmadinejad (map first-digit ahmadinejad-votes))
  (def mousavi (map first-digit mousavi-votes))
  (def rezai (map first-digit rezai-votes))
  (def karrubi (map first-digit karrubi-votes))
  # -or-
   digits <- lastDigit( c(d$Ahmadinejad, d$Karroubi, d$Mousavi, d$Rezaee))  # not even attached -- could drop d$

etc. While it's obviously a matter of taste, it looks horridly verbose.

Don't mistake explicitness for verbosity. Soon we'll see someone posting a Perl oneliner to do the same, claiming R is also verbose. Conciseness is not a goal, clarity is. if you have to expand the operations in your mind, you may well spend more time thinking about your code then when it's written fully, perhaps factored into a properly named function.

It could be done more concisely by using data structures and the (map fn struct) function, but it was expanded for clarty.

JMP can do such analysis in a more GUI-oriented manner if you prefer. Here's a 30-day trial: http://www.sas.com/apps/demosdownloads/jmptrial8_PROD__sysde...

Reply on news.ycombinator.com