So true. A test will only measure its set of criteria. What is a test for drive — desire to understand and learn? What is a test for how a person will respond to challenging and overwhelming prod scenarios? What is a test for if they will burn out in a month because they want to prove how rockstar of a programmer they are?
My most successful hiring has always been based on a conversation with 3-4 practical questions. I even worked in a company that had testing down to a science with all the psychometric nonsense and in the end, it just hired many sociopath-adjacents.