Testing in an environment completely different from 'production' isn't going to give a super strong signal on collaborative problem solving imo.
How can you tell? Is this always based on how you experienced the interview, or do you ask candidates afterwards how they felt?
(no insult intended, but) you made an empirical statement, and I'm curious where your evidence is from.
I failed 100% of them with various, strangely random feedback by HR/recruiter.
I've come to conclude it's the hiring mechanism of social incompetents / unempathetic sciency types. And it was always a dev from another group than the one I was applying for.
No thanks.
I recently started pointing that out, in a roundabout way, when I run interviews.
Why? If a candidate starts arguing about xml versus JSON, the candidate misses the point when I bring up xml. (Hint, I only discuss xml because it allows discussing widely known concepts that are very general and have little to do with xml versus JSON.)
Or, if a candidate says to me, "why can't we use async-await" in a particular question, the candidate also misses the point of the question. (Hint, I'm trying to test knowledge of concepts that's easy to test when doing something without async=await.)
Interview questions are always contrived and end up as an "if you want to work here, you are going to do what I say" exercise. If a candidate can't work within the constraints of a short interview exercise, then what will happen when the constraints of the job come into play? We never get to use our favorite patterns and APIs all the time.