"For MTurk, we aimed to select the best available crowd-workers, notably by filtering for workers who are classified as “MTurk Masters” by Amazon, who have an approval rate of over 90%, and who are located in the US."
It's interesting that the best available MTurk Master crowd-workers located in the US are paid about six cents per task.
(Ha! As if. Costs are only going to come down)
If you tried to pay people to do it properly then presumably it would cost much more.
It doesn't feel surprising that GPT is cheaper and more accurate than a human who is randomly clicking buttons. Custom software can already do that.
I didn’t do it because I can’t imagine a worse hell than having to solve so many captchas, but it opened my eyes to how creative botters can be.
I can imagine someone doing a similar setup for mturk, where all you need to do is remotely solve a captcha on your phone every few hours/minutes while ChatGPT does its thing. I also wouldn’t be surprised if that already exists.
Maybe openAI put safety barriers on their model... But surely the botters have a model without that by now?
When people don't read articles others can see that and correct misconceptions. Officialy approved bots will not be able to receive or act on this feedback.
1. The comment with the summary is made like any other comment. (In the case of Reddit, that’s perhaps mainly because the mods have no other option, but the point still stands.) Because of this, other users can downvote the bot comment if it is incorrect, as well as respond to it with corrections. Exactly the same way as you’d interact with a person in the past if their summary comment was incorrect; downvote them and/or correct them.
2. Include a disclaimer stating that the summary may be inaccurate. Encourage people to correct and/or downvote bad summaries in the disclaimer