Are Oklahoma Students Really This Dumb? Or Is Strategic Vision ...

From the answers to the questions I am inclined to believe in the "made it all up" theory. But at a 99% confidence level I can reject the possibility that it was made up with a simple simulation of the type described.

How am I so sure? Well we're given all of the probabilities for all of the answers. We can therefore calculate what (assuming independence) the likely odds are of different numbers of correct answers. We're also given the number of students who got each number correct. Here is the result (I'll give the program that I used at the end):

  0 right: 46 vs 32.56
  1 right: 158 vs 147.61
  2 right: 246 vs 273.73
  3 right: 265 vs 278.71
  4 right: 177 vs 174.46
  5 right: 80 vs 70.58
  6 right: 22 vs 18.74
  7 right: 6 vs 3.23
  8 right: 0 vs 0.35
  9 right: 0 vs 0.02
  10 right: 0 vs 0.00

The argument is that we should be suspicious because those two sets of numbers are too similar. The tails are not heavy enough. Now I admit that graphically it does look close. But if you look, the tails _are_ heavy. Is that difference significant?

We can use a confidence testing to answer that. The null hypothesis is that the distribution comes from a simple simulation and should be close to the theoretical numbers that I just gave. The alternate hypothesis is that the tails should be heavier and the middle lower.

Let's look at the middle because it is easier to calculate. Theoretically 0.55244 of the test population should get one of those two numbers correct. It is claimed that 511/1000 did. How likely is that?

Well in the null hypothesis each student is an independent observation. Each observation is a 1 or 0 with average 0.55244. The variance of an observation is readily calculated and turns out to be 0.2472500464.

The number of people getting one of those two answers after 1000 observations is therefore approximately normal with average 552.44 and variance 1000 * 0.2472500464 = 247.2500464. The standard deviation is the square root of that, which is 15.724. That puts the observed value more than 2.63 standard deviations out. at a 99% confidence we can conclude that the numbers were NOT generated by a simple simulation of the type described.

For the curious, here is how I calculated the theoretical distribution of answers:

  #! /usr/bin/perl -w
  use strict;
  use Math::Polynomial;

  my $ans = 1;

  # Number from the % of students who got each question right.
  for my $x (qw(.28 .26 .27 .1 .14 .61 .43 .11 .23 .29)) {
    $ans *= Math::Polynomial->new(1-$x, $x);
  }
  
  print "$ans\n";
  
  __END__
  
  (5.295902595984e-07 x^10 + 2.07154949414e-05 x^9 +
  0.000345419801607472 x^8 + 0.00322944050567802 x^7 +
  0.0187445906666514 x^6 + 0.0705826597827996 x^5 +
  0.174464060957845 x^4 + 0.278709766012616 x^3 +
  0.273731283943492 x^2 + 0.147609870020204 x +
  0.0325616632239042)

An analysis of the post: http://blog.smellthedata.com/2009/09/analysis-of-pollster-fr...

"communist and republican" ... lol those kids got brainwashed!