This is the sort of thing that CrowdFlower does automatically, except it takes the quality control aspect to a new level. It takes into account the history of the worker (were they accurate workers in the past? etc.) and also handles automatically rejecting workers who fail too many gold questions.
Speaking of, that's one of the cool bits about CrowdFlower, too, is that you can define "gold" questions. These are questions to which you know the answer. Workers will see these as if they were any other task. We then give them feedback about how well they're doing. If they keep missing the gold questions, then we mark all of their work for your job as "tainted" and their answers don't go into the aggregate calculations, which tell you what the users agreed on the most.
So, for instance, in a job where we pull in tweets about, say, #kanyewest, and ask workers to judge the sentiment of the tweet (positive, negative, etc.), we could take some obvious ones, like "Kanye West is a big jerk!" and mark it as gold with an answer of "Negative". Anyone who says "Positive" is probably yanking your chain. So we trust them less.
Of course, gold questions don't work too well for subjective questions, like writing short paragraphs.
I hope this gives you an idea of what CrowdFlower does in terms of Quality Control. I'm only the front-end guy, so I'm not too knowledgeable about all the math behind the scenes. All I know is that there's lots of it. :)
(I'm a front-end developer for CrowdFlower. I don't represent the company's views. Except in regards to bacon. I represent those.)