And if they fix it? Pay them. Simple.
Stop fooling around with abstract puzzles that have zero relevance to a battle-tested production environment.
> Stop fooling around with abstract puzzles
I beg to differ on the simplicity here.
The very best candidates don't want to think about your stuff at all, unless they're paid ahead of time, since they've already made tons of $$$. In effect, they're already getting paid by your competition or they themselves are your competition.
It's a great example of Akerlof's "market for lemons".
The "satisficing" solution IMHO is to take [calculated] risk on available consultants/contractors and keep the better ones on retainer [to hire]. Yes, in a sense, I'm suggesting you pay [more] to interview people. That way you stand a chance of beating the average.
Finding the "best value" in a candidate is a trap... you only need good enough.