Relying on coding challenges or automatically-generated code metrics is deceptive because it tests if people can solve coding challenges, not solve real production issues or work on production apps, quickly and/or produce maintainable code. (It's about as useful as asking "why are manhole covers round" frequently asked questions interview trivia questions as proxy signals for intelligence.)
So, unfortunately, finding top coding talent is fundamentally an irreducible problem because it takes a little time & money to examine if someone is a badass coder by other badass/es reviewing some actual work and/or working with them on side-project/s.
100 years of industrial psychology. See page three.
- General mental ability (Are they smart)
- Work sample test (of actual work they might do, NOT HAZING/Quizes).
- Integrity. (Varipus tests available for this)
Things that do not work.
- Reference checks (r 0.26)
- experience (years) (r 0.18)
- Years of education (r .10)My favorite HR signal which doesn't seem to qualitatively work:
- non-rigorous certifications
The best interview styles are not labeled as such explicitly, whenever possible, because it makes for artificially-awkward pressure, alters expectations and ups the hype and hazing.
Disclaimer: I work as a Director of Engineering at Toptal and I've held a few hundreds of those live interviews myself.
Disclaimer: she interviewed me. (The experience, by the way, was almost identical to the one I had interviewing with AirBnB recently).
However, it's also important to know that the 3% was not arbitrary. We didn’t start by saying, “wow, we should set the acceptance rate at 3%, because that sounds nice” or we somehow knew that the bottom 97% were bad. 3% (and every value you see on our screening process) is simply the result of a numerator and a denominator (successful_applicants over all_applicants). We designed the process to be the bare minimum criteria for us to be extremely confident that we are admitting someone into the network who will thrive here.
When it comes to technologies being in demand, it really ranges all across the board: from the standard PHP, Ruby, Javascript etc, over more bleeding edge like React, React Native to niche things like OpenCV are all there.
Full disclosure: I am currently employed as Head of Community for EMEA @ Toptal