ammon

All candidates did both screens (quiz and fizzbuzz). The correlations were calculated against the same population. Now, I agree that survivor bias could affect the quality of these results (we know nothing about the significant % of people who dropped out). But it's not really possible to solve that problem outside of a lab. I don't think it's an argument to not do analysis. For now we're simply trying to minimize the dropoff rate, and maximize correlation. The quiz was better at both.

The problem is that you don't actually have a clear measure of who you're rejecting. Every company uses a similar argument to support their (very different in every case) interview process. The fact that it allowed you to build a great team does not mean that you could not have built a better team by giving other options as well.

A lot of people feel this way, and I see no reason why it can't be an option at more companies

Well, that's hard. Standardizing evaluation (critical to reducing bias) requires that everyone (or a lot of people) see a similar problem. And when someone is working on their own time, I think it's important that there's not working code easily accessible online. Perhaps this can be overcome by asking candidates to talk through their code after they are done

Yeah, we agree. We don't plan on asking everyone to do hours of work on their own. A lot of people don't have the time. However, we're also seeing a bunch of applicants who are good programmers, but become really stressed in interviews and do badly (they freeze). These people have trouble getting jobs. What we want to do is offer them the option of doing a larger project on their own, rather than our final interview. I think this might help a lot of people (clearly we have to test this)

Author here. We do do the interviews blind to earlier screening results. That's clearly vital. But you're totally right that job performance is the real thing that matters

Hey! Author here. Yes, we did not know that the screening steps were meaningful, so for the first 300 applicants, we interviewed everyone, even people who performed badly. We then looked for correlation between screening step scores, and programming interview results. Doing well on the fizzbuzz problems was not very correlated.

By dropoff I mean people who left during a step, and never logged back into our site.

Hey, another Triplebyte founder here. This is something we've talked a lot about. To set a balance, we talk to people about a project that they've worked on in the past, and also have them pick a problem from a list, and work on it with us over screenshare. I'm sorry this did not go well for you (I'm curious to look at our notes from your interview, but I don't know who you are).

We're focusing on being consistent and fair. I understand (and am sorry) that we mess up and miss good people. But looking exclusively at experience creates other opportunities for errors (I believe more opportunities). We don't require perfection on any of the problems. We're looking for process, and try to help people (for example, on both of the problems mentioned, we generally talk about good approaches before the person starts coding). But again, we definitely mess up. Sorry about that. We're trying to do it less as we improve the process.

I see the problem. Experience is certainly valuable. But a resume also shows age and status. We're working to eliminate age discrimination (and other biases). That's part of why we do blind phone interviews.

We're open to remote workers. There's a little less demand (not all YC companies are hiring remote workers), but we have no problem with it.

Which ones? We're thinking about offering all the questions in a bunch of languages, and letting people pick.

No, it's not. We think that understanding basic code should apply to everyone. That said, we're not trusting the fizzbuzz for now. We're going to talk to the first several hundred people who apply, then look for correlations

Founder here. Yeah, sorry about that. We totally intend to work with people without visas (it's actually where the idea started). We're just doing a phased launch. Talking to lots of people on the phone is hard to scale. We'll be removing the restriction as soon as we can.

EDIT People without visas are free to register now. They just can't book phone calls. We'll let everyone who registers know when they can book a call.

Hey, founder here. Yes, we totally agree that things other than technical ability matter. Some of these things (productivity, ability to work on a team, communication ability) we think we can evaluate. Others (company culture) we can't. As you say, however, getting better at identifying good engineers (outside of any company's specific requirements) is valuable.

I was unschooled from 1st grade through high school. Overall, the experience was great for me. I had time to do a bunch of different things (boat building, blacksmithing, metal casting, programming, ice boating, climbing and hiking, firework making). I'd simply not have been able to do all these things to the degree I did if I were studying 6 hours a day. That said, it made applying to university hard. I ended up going to a pretty bad school, the only place that would have me. I had gaps to fill once in university, but managed that quickly (earned a 4.0 my freshmen year).

However, I know people for whom unschooling was a disaster. They feel they were neglected and missed critical education. I will not unschool my children.

Oh, and I can't spell. Many unschooled people seem to be bad at spelling.

Choosing to fight a man with a knife who is trying to rob you is stupid and dangerous, regardless of your training. I'm not sure what this says about the thesis.

I'm an erstwhile (amateur) blacksmith, now a programmer. Funny to see the two together. Blacksmithing and programming are both areas where practitioners make their own tools. Compilers and tongs can be bootstrapped.

We had a database problem. All settings changes were failing. It's now fixed.

Apple does not want beta products in the app store, which makes sense. But I've not seen an outright ban on testing. Testing allows us to make the app better! In any case, we've not been rejected.

Glad you like it. We got the testing/config system working in a day. Of course, we've spent time tweaking and adding features since

For what it's worth, the payment servers were one of the only parts of our system that did have (manual) tests in place at that time (because it took 24 hours to do a restart). The bug in question was in DB timeout code, and all tests passed (until several weeks later, when the DB had problems). Now, clearly coverage was insufficient, and I do agree that adding more testing (as we now have) is a good idea. But some bugs are always going to slip past. Code reviews are what could have caught this bug (which we also now take more seriously).