I remember one time I was hiring for a JavaScript developer. The most important technical competency was a brief working knowledge of the DOM. The actual position required modifications to pages for A/B tests and jQuery was either too slow or broke in production when other developers used it for their test code.
Of everybody that passed through HR to me I sent out a notice that there would be a minor code assessment and they could not use jQuery. 6 or 7 people instantly dropped out. There were 22 people left that I actually talked to which only three could pass the code assessment.
The assessment was never designed to be challenging. Candidates were using their home computers had access to any reference material. There was no hurry. They could look things up. I also told them if they got stuck just ask me and I would point them in the right direction without any penalty, but only 2 candidates tried that.
The assessment was an hour long phone call. At the start of the call I would send out a static HTML page for the candidates to open in their browser and for them to write an answer in whatever tool they wanted but it had to execute in the console of a browser on that page. Tasks were things like take this item out of the page and reinsert at some other location or change the color of a particular paragraph to red. Stupid simple stuff that you would expect any UI developer to do easily. After all these were beginner skills and this was a senior level position.
I did ask to see the code they wrote only to make sure they completed the tasks. I would drop their code in my browser console and run. I never looked at code style or sloppiness. It’s a job interview where people are rushed and nervous. Their code is allowed to be sloppy.
Only 3 candidates were able to pass this code filter. The people that did pass either did well enough to pass or extremely excellent. The people that failed, maybe 19 of 22 people, all failed epically.
That experience really shocked me. I did everything I could think of to make this relaxing and take the edge off because there is always pressure during a job interview. No matter how much of a lifeline I threw out almost nobody would bite. They would sit there silent on the call or they would attempt to stall like that would somehow make code magically appear. Sometimes I would remind candidates they could ask me questions because I cannot evaluate a blank answer. Other times I would try nudge them to get started or guide them into progress, which only seemed to hurt more than help.
Very bizarre experience.