It seems like going through CrowdFlower could help a lot. Has anyone used it? I guess it worked for the author.

At my last job I was involved in a project where we were going to use CrowdFlower, but it ended up being canceled. I was always curious about how good their results are.