I've been doing it manually: looking at resumes, code samples, and asking sample questions to judge for myself. Not sure there's any automated way to do that that will work any better.
Otherwise your platform is really a way for random people to get money (nothing wrong with that) while the actual maintainers are busy writing code, reviewing tickets, etc.
I know monetizing open source is terribly difficult so I don't want to sound too negative.
Ideally, this is the plan.
Take a look at Lyft’s post-ride survey. Rate your ride, plus a set of chips that let people express what stood out about the driver, plus a comment field. Very simple & quick to complete.