It's not like you need a PhD to confirm if a search result sucks. Results are supposed to be good enough for the average user, so you need common sense and some on-job training to do a good job.
And having 10 common-sense guys churning through user feedback @$11/h is better than 2 geniuses @$55/h.
As one comment says, it probably beats fliping burgers.
They cannot rate all results anyway, so why not get the best possible evaluations?
Shouldn't quality data beat quantity if you extrapolate from the evaluated samples to the entire index?
Spammers target the lower part of the bell curve. If you let that part evaluate the quality of the results, do you think that they will recognize spam?
For example, here are some queries in my history - would the average person understand what I'm looking for and what a good search result would be?
"neovim worktrees"
"django models through"
"pandas econdb"