How do they handle bogus results from MTurk? I know people that tried to use it and had to stop because 95% of the "workers" just put meaningless data in hoping you don't notice and they get paid.
It depends on the task, but if you can get them to do multiple tasks in one, you can ask them to do a task you already know the answer to. If they get the known answer wrong, instant reject.
Couldn't you just solve that by sending the job to multiple workers and comparing the results?
Think about it.
That's an obvious solution, but you might start approaching a point where it's actually cheaper to hire a dedicated person for it.
That doesn't solve the problem of a person being wrong.