Adaptive scoring algorithms and ways to let employers create custom projections of our multidimensional data is something we're thinking about. If you have ideas on how to build this, I'd love to hear them! james@cspa.io
The issue you're having is that you're trying to test over 7 dimensions, with the majority of those dimensions being irrelevant to an individual taking the test to signal mastery of 2-3 of them.
The standard method in testing to deal with issues of this nature is to provide people with the opportunity to test themselves on modules, rather than taking the omnibus test so that they can spend the majority of it signalling that they aren't good at things they don't care about. If they're good at everything, they can sit for all of the modules. If they aren't, they aren't wasting their time.
Additionally, this frees candidate time up to let you test people in more depth in the areas they're actually professing skill in. From that point, you're open to do interesting things; evaluate question difficulty/time to completion and adjust the questions provided as the test unfolds to match the expected skill of the applicant for instance.
As it is now, you're getting buy-in from corps that little have no skin in the game, but I don't see how the sample test you have provides more signalling value than an applicant having a CS degree or a list of github projects they can discuss.
Adaptive testing (https://en.wikipedia.org/wiki/Computerized_adaptive_testing) is something we're looking into, and would be extremely interesting (and challenging!) to build.
The algo to take Core + the best other section, right? As it stands, the test difficulty makes it pretty easy to hit 2 perfect scores, so you'll be competing against those pretty consistently.
How does the CSPA help my interviewing process if I'm a network guru and knock all those questions out of the park, but apply for a ML job where my knowledge is paltry? How does my CSPA testing help me differentiate from the above guy as a network guru when I also aced the ML section? This seems like an obvious area the test can outperform other testing metrics.
As far as I can tell, the only value for the current scoring system is to weed out people who are just plain abysmal at everything. Eventually people are going to game your exam and pop up very close analogues onto google. At that point even the floor function is dead with the added deadweight cost of the exam still being an application requirement at BigCo.
Maybe I'm just blinded because I think having accurate skill radar charts that test takers and employers could use for self improvement and prospect evaluation, respectively, could be an absurdly large value add.
You're right, so far most of the test takers are entry-level or changing careers. To accurately assess specialities like ops/IT, we'll need separate, dedicated subject tests (or adaptive tests).
That said, no one has yet gotten a perfect 1600 :) . The highest is something like 1380.