It's basically a specialization of the base rate fallacy. If, say, idiots can still be right by chance 10% of the time, and experts are right 90% of the time, but only 10% of the population are experts, then lucky idiots will make up fully half of the population of people who are right.