Very interesting. What were your challenges with respect to the machine translations specifically? Also, can you post or share the MTurk ads you use/d?
This ad has been the most effective for me:
Translate a bit of English text into another language.
Please email me at andrew@gaiagps.com to tell me what language you want to translate to, and I will email you the text to translate. The potential languages are Spanish, Portuguese, German, Dutch, Swedish, Italian, French, Russian, Japanese, and Korean. No other languages are needed.
Do not use an automatic translator. I have checked all of them. You shouldn't email me unless you are a native speaker. If you do not have Chinese relatives or a degree in Chinese from a university, then you cannot do the Chinese translation. The same goes for all the other languages.
I have gotten hundreds of responses, mostly from Indians who claim to do the translation, but they use an online translator. If you are one of these, don't bother - you are just wasting your time. If you are a special guy in India who knows Russian or some other language, then send me a really convincing email, or I'm just going to delete it with all the spam.