I'm happy to answer any questions about this (I expect it to be controversial). When we started Triplebyte one year ago, I was pretty skeptical of bootcamps. Doing credential-blind interviews and seeing what some bootcamp grads can do, however, has won me over. Clearly there are a lot of bad bootcamp grads (and probably a lot of bad bootcamps). But the model is working really well at the top.
We never provide negative info to companies. If you fail our interview, you are free to apply to whatever companies you want on your own.
The practical advantage of going through Triplebyte is that you'll pass more interviews. We match you with companies where you will pass at a higher rate. Because you only have time to interview at a fixed number of companies, this will give you more options, a higher salary (from more competition), and a better fit. We also help candidates negotiate (A big help if you're afraid to do this), and help people with bad resumes (say self-taught programmers) get in the door at top companies. Traditional recruiters do just the opposite (they filter heavily on resumes)
Employers don't actually (in most cases) have a very good grasp of what qualities they select for. It's function of the engineers doing the interviews and the engineering culture, and most companies are not aware how much this differs between companies. We model what each company looks for by sending them candidates with specific attributes and reading their feedback (we get honest feedback from companies after interviews, which is pretty rare). We're then able to see how feedback for the same candidate differs between companies.
We have specific guidelines / process that we use to measure each. For example, professional coding is a focus on writing clean code, that is well designed on the micro level (good names, good modularity on the function / class level), and good testing. We measure this using a rubric as we watch each engineer code. Low-level understanding is knowledge of how computers work, under the hood (bits, bytes, character encoding, operating systems, networking, etc). Again, we have a rubric (and these topics are covered at multiple points as each engineer goes through the process). Our measurements are not perfect, but companies really do vary widely in how much they care about different areas (understanding how computers work say, vs. having a great coding process). By mapping this and matching we save everyone pain. No one had tried to do this before, and I am pretty excited about it.
Well, the engineers who go through our process are in a good position to verify the effectiveness of the matching. Granted, the bar to reach that point and check the quality is higher than it is for Pandora (readers can't go check right now what their matches would be). But I don't think that's an argument against trying to do a better job matching engineer with companies. This is an important area that's been largely overlooked.
The categories that you mention (applied problem solving and professional code) really are important. Companies differ widely in how much care about those two things (solving problems in the interview effectively vs. showing clean, well-structured code and good testing process). When an effective but iterative (and sometimes sloppy) programmer interviews at a company that values process highly, the result is wasted time and pain for everyone.
I am sorry that you had a bad experience with us. Evaluation is a really complicated thing. The bar that we use for evaluating the take-home project is to treat it as real work, e.g. would a teammate feel good if you were working with them on this task, and you came back after half a day with this. Because we can't see process, all we can do on the take-home track is judge of the finished result is professional-level programming. We do indeed pass people who do not complete the project on the 1st call, but they need to be on a good path (where we think they can finish by the 2nd call).
I do not know who you are (and would of course not post details here), but from what you say, it sounds like we did not think that you were on track to finish, and has some concerns about the design that you selected (we track whether a number of milestones in the game have been reached). It is totally possible that we were wrong. We much prefer to see a working front-end / back-end combination that has missing features than we do just a front-end or just a back-end.
Again, I apologize for your bad experience. I hope we can make the take-home interview better in the future with some tweaks.
I totally agree with this. Companies should give (constructive) feedback when they say no. They do not partially out of a fear of being sued, but also because they often don't know really why the reject people. The default state at most companies is rejection. If no one really liked you during the interviews, at most companies this will result in a rejection.
We're doing this differently at Triplebyte. We give everyone we don't work with (who does our final interview) a several hundred word personal email, with an explanation and advice on how we think they can improve.
So, you may be right. Perhaps you're great at telling good programmers from bad. But almost no one does the analysis to really know if this is true (false negatives are the huge unknown). And most interviewers are not as good as you. The consistency between interviewers at the same company is low (we measure this). The consistency between different companies is low. We (interviewers) can do better than this!
This only works if the numbers are in a known range (say sequential from 1 to 100), and you XOR in the index (plus 1) as well. Then each number is XOR'd two times, except the duplicate, which is XOR'd 3 times (and thus remains at the end). The fact that the code is wrong shows why this question is a very bad interview question.
EDIT
The given code works to find the only non-duplicate item in a list (perhaps that was what was intended)
I am sorry that CS education has not worked out for you so far. I want to emphasize the "so far" part of that. I hope that that's hyperbole at the end of your post. If not, I have to encourage you to reach out to friends and family, and maybe take a break from interviews. You can get in touch with me (email in my profile).
There is a huge amount of randomness in interviews. I usually say this as a bad thing, but in your situation now, it can actually be a good thing. Interviewers look for a WIDE variety of traits. Most ask algorithmic questions. But if you do enough interviews, you will find a company that values the skills that you have (I am assuming here that you can program productively). Smaller companies particularly have more variance in what they are looking for. I encourage you to just treat this as a numbers game, and get applications out to as many small and medium companies as you can.
This sucks. But it can and does get better. After you have a few years of experience under your belt, companies will look at you in a very different light.
EDIT:
Also want to add that we made Triplebyte to help people like you. We'd love to have you apply.
Fair point. But is it not much more common to encounter a programer who cannot break a complex system into loosely coupled components? Or who does not use consistent naming conventions? Or who is smart but lazy? I'd rate all of these as equally important things to try to evaluate in an interview.
I am a strong believer in looking for strength in an interview. Someone really rocking complexity analysis is a strong positive (shows that they are smart and pedantic in a good way). But so does clean, smart code and great loose coupling.
You're not being compared to people who lack hundreds of hours of programming projects. You're being compared to people who have that same experience, either at a worse school, or on their own. Credentials correlate with being good, absolutely. But many, many more people lack credentials than have them. This means that there is a large number of great programmers who don't have them (I think a larger number than who do). There are also bad people with good credentials. Strong filtering on credentials harms companies who miss good programmers, and harms programmers who can't get jobs. Using credentials as one factor among many in a screening step, however, makes good sense (although we don't do this at Triplebyte because we want to force ourselves to get as good at possible at directly measuring skill).
So, I see where you are coming from (I actually love academic CS). But the VAST majority of the programming work out there does not require any Big-O analysis. It just does not. It's used as a tool in interviews to (essentially) look for rigor. The problem is that this harms people who are rigorous as hell in low-level details of JS and V8 (something I'd posit is actually more useful to many more companies), but never studied academic CS. There's nothing wrong with valuing the skill of complexity analysis. But there's a mismatch in how common it is to look for this.
We've not seen very much focus on design in interviews (very few companies talk about this when telling us why they like/dislike candidates). We do see a lot of focus on interest in what the company does. I think that pitching design skills as a passion for making the company's product better is the best way to go.
A large number of bad things influence interview decisions (credentials, targeted practice, how well you know the specific algorithms that come up again and again in interviews). I hope that more programmers getting better at interviewing skills will help move companies toward measuring actual programming skill.
Early startups can do this. The problem for larger (valuable) startups is that the grant of stock would be taxed (as income). Options get around this. (The tax law makes the rather dubious claim that options granted at the market value of the company have zero value.)
Sure it will. Keeping options is strictly better (for the employee) than not keeping options. Vested options are compensation for work that has been done. Everything you say can be true, but that does not mean that companies should not let their employees keep their options.
We moved the blog to medium about a month ago. Since writing this, we've tested a bunch of new things, and now have about 20% better precision on the quiz (20% increase in the percentage that we predict will pass and go on to pass). There are some other interesting results (one is that easy programming problems are about as effective at screening as hard programming problems). I'll be writing more about this, with the new data/results in the next few months.
Interesting follow up to our blog post! This goes into a lot more details about why startups might prefer product-focused people, and larger companies might need technical people.
The discovery of product motivation in interviews is limited to “are you interested in this product?” [...] Technical interviews discover technical programmers, even if YC startups prefer product programmers.
Interesting point. Could the explanation be that larger companies (with more demand for technical programmers) define interviewing culture, and skew all interviews in this direction?
If you have more microservices than users, you may not have an engineering problem, but you definitely have a product problem.
So, we ask at the beginning to talk about a programming project that you've done it in past, but not about where you've worked. We don't ask for a work history until the end (by which point the notes and scoring that go into the final decision are over). We ask at the end so that we have the data to write posts like this.
I can't give company names, but we've not seen much of a pattern. It really seems to have to do with the backgrounds of the founders, and who the early successful hires were. This sets an engineering culture going forward.
We asked about visa sponsorship. We did not ask about salary. The general pattern with visa sponsorship was that small companies don't want do it, out of concern about the success rate (only 30% of H1B applications last year won the lottery). Some large companies will sponsor visas, but they set a higher bar (they have to be really excited to take on the extra work). The exception is larger companies with teams outside of the US (several YC companies have offices in Canada) to take advantage of different immigration restrictions.
I think this is the best solution. It does raise issues of how you keep a consistent bar between the various options, but I think it's better than the alternatives. Currently we let people do a project track where they do a project on their own time, or an interview track where they answer interview questions. We also give them a choice of interview questions in a number of areas, to try to increase the probability that we see a strength.
Several reasons:
1) people who go through us skip the screening steps and go straight to a on-site interview at the YC companies.
2) We've sent a bunch of people to most of these companies, and have a good idea what they are looking for. We help you avoid failing interviews for the reasons mentioned in this article.
3) Perhaps you have no trouble getting responses to your resume, but a lot of (strong) programmers do. We help them.
4) We help candidates negotiate offers.
Good point. There is a short list of big companies that are taken as positive signals (Google tops the list), while most others are negative. I think that this gets at a contradiction.
I think that recruiters do make sense (clearly). I hope that we can use our position as recruiters to be better than most companies at figuring out who's a good programmer, and better than most candidates at figuring out which companies are good places to work (we're candidate constrained, so we have no incentive to send people to bad companies)
EDIT: I thought that sillysaurus3 was talking about a work trial period (days or a week), but it seems they are talking about asking question during that interview that are similar to actual work. I retract me objection :)
If only it were that easy. I am not at all opposed to work-trial tests. They are almost certainly more accurate. But a high percentage of programmer will never apply to a company that requires one. Read any HN comment thread about the topic. It's very controversial. Lots of programmers are against it. Work-trials require MORE time upfront from the candidate, and of course still often lead to rejections (by design a hiring filter rejects most people). A failed work trial burns an entire week of vacation. An unemployed programmer (who is being paid the work-trial) may like it, but someone working another job may not.