Why were you unable to keep StockFighter running?
All three of us are software developers. We like developers. We spend a lot of time talking to developers, in particular about their careers.
We had no trouble finding clients, but a lot of trouble convincing clients to take the CTF model seriously; virtually all of them wanted us to function like a traditional recruiter, offering 10 candidate profiles for every open req. That's a job that is 90% saying "no" to people.
It was dispiriting watching developers we knew to be strong get rejected for positions because they didn't have the right resume, or didn't solve some pen-and-paper programming puzzle quickly enough. It was a drag for all of us. It made us unhappy.
Life is too short.
Sorry about that horrible timing.
I mean, change some puzzles/tests or simply recommend someone to the job if you knew he had the skills?
Wasn't this what you were solving? Matching positions to great devs.
Anyone that has spent time working outside of firms that hire this way (which includes stockfighter's founders), realizes that the technical interview today is very expensive, throws away a lot of candidates, stifles all axis of diversity, and ultimately it's not really all that predictive of on the job performance. A CTF is not an ideal predictor either, but I'd argue it's better.
Stockfighter found that their best candidates by the CTF were receiving terrible ratings by the companies they send them to, because they lacked the shibboleths required to pass the interviews. They'd only have success if the company filtering after being presented a candidate was minimal, and then they saw that yes, the job performance of the candidates that were good at the CTFs were at least as good as those that came through the traditional recruiting process. Apparently there wasn't enough appetite for that kind of process, and thus, Stockfighter made no sense as a company.
That's a pity, because I share their view that technical interviewing is broken. I just don't see how we can systematically improve it without the kind of data that you can only get at huge companies that hire thousands of engineers a year.
I understand and appreciate the mentality that your comment comes from. But the premise of our startup wasn't Steve Blank-style customer development, where we started with a vague idea of the space we wanted to be in and then worked with customer prospects to figure out exactly what product we would end up building.
Rather: the company we started was an experiment, a very opinionated one, to see if we could port a hiring model that had worked extremely well for us at one company to a larger set of companies.
I think there's a way to make that model work, but I've become convinced that "contingency recruiter" isn't the way to deliver it.
The last "innovation" I've witnessed in the space was the Hired model, and everyone is trying different versions of that now. It's a hurtful model that abuses the inefficiencies and lack-of-transparency in the two sided market. (For anyone wondering what I mean: When you sign up on Hired they ask you what your preferred salary range is, then, as of last time I checked, they show _that exact value_ to the companies looking through prospective candidates. So guess what the first offer you get is? Not what a "fair" market rate is, but exactly what you typed into the sign-up form.)
I do a lot of talking to recruiters as part of helping them use our Stack Overflow Talent product, and honestly it feels like taking crazy pills half the time. Having to answer "why?" to "You need to at very least explain what your company actually does, or maybe the tech stack you use" makes me feel insane.
I loved Starfighter, but thinking back what I liked most about it was the content you and Erin and Patrick wrote about it. Your point about "life is too short" is very very poignant.
I guess the whole point of this comment is for me to say I completely respect that decision, you all did the right thing. Trying to convince people who would rather just throw more money at a wall than try something unique to use your unique product is an awful feeling.
I'm very glad to hear your new project is easier to make money off :)
I'd better do a stupid coding test online that have to schedule a phone call during working hours with a guy who just want to get away from the phone call and could barely present his company and what he's recruiting for.
Standard interviews: multiple choice trivia, red black tree programming puzzles.
Innovative interviews: Custom code exercises with custom unit tests written by the company, ssh exercises where a server is created automatically per candidate and he has to connect to it and configure/fix/change some stuff.
After many interviews (both given and received). I have definitely settled that:
1) I'd rather write a stupid program on HackerRank than deal with an annoyingly scheduled phone interview (where I might be expected to spell that code over the phone WTF). Automation is nicer than the average interviewer. A proper code editor for coding is better than the average shared editor.
2) No matter what we try to do: 20% of candidates won't show up + 20% won't be able to code a program to print numbers from 1 to 10. There's nothing we can do about that (the sad state of interviews is probably all because of that).
3) For the first initial screening, very simple exercises are all that's needed (see 2). Hacker Rank is perfectly appropriate for that.
s/the recruiting/any
or, simpler:
s/in the recruiting space//
Did you think you could change the way people hired overnight?
There probably is a way to do what you set out to do, but it seemed obvious to me from the moment you announced it would take years of slogging and building a track record to change such ingrained processes at companies.
Revolutions on that scale are never going to happen in a few months. The reason I thought you guys had a shot was that you'd be stubborn enouh to stick to it in the face of resistance.
I have thoughts on how I'd do contingency recruiting differently if I had to do it over again, and I have thoughts on what I think a better model for this would be than trying to do contingency recruiting from day one.
What I don't want to do is write them up in HN comments that will further sap me of any will to do a real writeup.
I think you need to find the few rare companies that are interested in this recruitment/selection model.
9x% of places are doing standard resume review and interviews. They don't have the will nor the expertise to evaluate CTF (or any similar) contests and candidates.
Matasano?
We'll probably put it back on the air at some point in 2017. It's down for a lot of little reasons (inefficient AWS configuration, occasional need to babysit, the need to adjust a lot of little messaging things around the site so as not to make people think we're still recruiting off it, &c) that add up to enough hassle that it's easier to have it be down for now.