For example, we used this feature to dogfood our own FAQ page: http://pickfu.com/4UQM7U
For example, we used this feature to dogfood our own FAQ page: http://pickfu.com/4UQM7U
the FAQ example is a bad example: I can always answer "yes" and then supply as reason "every questions I had was answered". No need to read the FAQ page, of course.
I think if you want to insert any kind of validation system you would have to count the time it took for the turk to answer your question and make sure we are in that range for every hit.
I'm sorry you had a rough experience with that specific music-listening job - we've never had a more time-consuming question submitted. Most are in the order of "which site do you prefer?", like this current one: http://pickfu.com/AGSIBT. These get answered far faster by the MTurk workers.