Triplebyte expands its recruiting platform beyond YC, signs up Apple, Facebook
techcrunch.com
techcrunch.com
Upfront, I didn't pass a TripleByte interview I had (one of the few companies I haven't passed).
My interviewer showed up late initially, then took a break and showed up 10 minutes late after the break. Further, the interviewer nit picked super irrelevant details, and acted exceedingly smug and condescending. Some of the stuff he told me I was wrong about was related to my research. Even after attempting to explain it several times, he just said, "No, you're wrong, you don't know what you are talking about."
I then literally brought up the paper and sent it to him, before he said something along the lines of... Oh, well I guess that is right.
Overall, it was one of the worst interview experiences I have had, and I don't believe they are good way to recruit. Hell, I even passed all their coding questions with flying colors. It was the silly video conferencing interview with a smug engineer who really made the interview fall apart.
Yes, it was.
> I thought their interviews were standardized.
I don't know if you can standardize peoples personalities... The real issue here (in my opinion) was the guy was not a good interviewer.
> Can you tell us more about the experience, and what they asked?
I honestly don't remember, they asked about active record (as I do some web dev), but also stuff related to describe on the spot some SQL queries and why one would be faster than the other. All of that appeared fine, the real issue was our debate related to computer vision (where my research was done). Sorry, I can't really dive more in, but it was probably a year ago.
Oof. I feel like computer vision is a particularly misunderstood field. In a way, it reminds me of statistics - I feel like people purposefully obfuscate things to make their results sound more impressive. The great thing about computer vision is you can almost immediately see through the smoke and mirrors - ask for a live demo. I'm amazed how many people refuse to show me a demo of something that is inherently meant to be viewed.
Did they give you homework challenges? Did you clear those but then bomb the interview?
I'm not an expert in all areas. At the end of the interview we have a section where we let the discussion go into whatever technical area the engineer wants to talk about. It sounds like you're an expert in an area where I am not. In those cases I try to ask questions and push deep, but (depending on the topic) that can be hard.
edit: removed discussion of the specific topic discussed. Sorry, folks below are right
Expertise is something our model handles less well (it's much harder to standardize). This certainly results in us failing some great people (and it sounds like that may have happened here). I'm happy to talk about this more. Give me an email at ammon@triplebyte.com.
On the other hand, sharing my info publically is not professional. I shared what I viewed as unprofessional, without specifics regarding interviewer or even questions. He shared my technical expertise and much more specific details regarding my question(s) and interview.
That kind of goes along with my original point, it seems abrasive and leaves a sour taste in my mouth.
Quite honestly, it probably damages their reputation more by responding. Why would someone want to interview somewhere, where they know if they criticize, their interview could be made public.
I took the original comment about the interview with a grain of salt(these things are inherently subjective). But it's fairly obvious now that they're as unprofessional as claimed. Hope no one from Apple or Facebook sees this.
That being said, I think the best way to improve the process is to try to standardize it by area of expertise. I'm sure that is done to some degree, but I think more pronounced may be better. I think of roles within companies are too static and don't actually represent role(s).
Regardless, best of luck. I didn't mean to be a dick, but I felt obligated to share my opinion in this case.
First round was multiple choice questions, relatively straight-forward. Second-round was skype-call and just felt incredibly subjective. I was asked questions around building out memcached to support arbitrarily-sized values, and I got the same "smug" vibe you sensed.
The interview style was very "Him: How would you do X?" "me: Well that's not a simple problem, there are a lot of solutions each with tradeoffs." "Him: Okay so name one" "Me: So you could do X" "Him: BUT THEN Y [GOTCHA!]" "Me: Yes, that's one of the tradeoffs of X"
It wasn't clear to me what the heck he was even looking for. Was he hoping I'd list race-condition problems? Had he not even considered race-condition problems? Was he looking for a theoretical solution or a real-world solution? Also he kept going on random tangents ("That brings me to an interesting question, how would you shift a gigabyte of memory 1 bit?"). He seemed very concerned with efficiently bit-packing the header in this problem, which seems silly to me when we're talking about storing gigabytes.
My understanding was that triplebyte was seeking to be the SATs of engineering, however SATs do heavy validation with test-retest reliability and such, I had no particular reason to suspect triplebyte's interview was any more objective than any other company's.
It doesn't matter how good the interviewer feels the candidate is, or whether a design was picked from a decision tree. All that matters is whether the candidate can do the work at actual companies.
I think people here are reacting to irrelevancies during the interview process -- questions which cannot possibly be reflective of a candidate's real-world competency. (When was the last time you shifted a gigabyte of memory? And even if you did, that's not what companies are going to employ people to do. So why ask the question? Are you sure it isn't trivia?)
We've recently moved to a new interview processes organized around this idea of max skill. It's working great in terms of company matching and predictive ability. However, it seems we may have underestimated the cost to candidates of being asked about areas where they are weak. There's more negative feedback here than we've seen in previous HN discussions, and I think that the interview change may be behind that. I'm taking that to heart. I think we can probably articulate it better (that we measure in a bunch of areas and look for max strength). We're also running an experiment now where we ask engineers are the start of the interview which sections they think they'll do best on. I'm excited about this. If engineers can self-identify their strongest areas, we'll be able to make the process shorter and much more pleasant!
So, the bit shift question: that come up down one branch of a system design question that we used for a while (we've since moved to a more targeted version that is more repeatable). The (sub)issue involved adding a binary flag to a large data blob (this came up as part of a solution to a real-world caching problem). Adding a single bit flag to the front of a 1GB blob has a problem. To really add just one bit, you'd have to bitshift the entire 1GB. This is clearly not worth it to save 7 bits of storage (ignoring that that would not be saved in any case). You can just use a byte (or word), or add the flag at the end. When candidates suggested adding a bit flag at the front, we would follow up asking them how they'd do it (to unearth if they were using 'bit' as a shorthand for a reasonable solution, or if they really are a little weak in binary data manipulation). This was one small part of our interview. By itself it in no way determined the outcome of the interview, or even of the low-level systems section. Plenty of great engineers might get it wrong. But I don't think it was unfair.
Of course it's unfair. The candidate isn't actually programming a solution when they're talking to you. They're on a tight time crunch, under a microscope, in front of an interviewer. The answers to your questions will literally make or break their future with you. Did you specify to them in your original question that the entries in the cache are 1GB large? If you assigned them the task of implementing a solution to your caching question, they would immediately notice using a 1-bit flag is a poor design decision.
The point is:
Plenty of great engineers might get it wrong.
That says quite a lot more about Triplebyte's question than the engineers. A wrong answer doesn't mean they're weak in bit manipulation or that they decide to implement poor solutions. It says they're suffering from interview jitters. They're weak in the artificial environment you've constructed for the purposes of the interview, which may or may not correlate with their actual ability.
This may sound like useless theorizing, but unfortunately a massive number of excellent engineers are awful in an interview setting. But if you give them a problem to actually solve, they pass with flying colors.
Triplebyte does give problems to candidates to solve, but it sounds like you also care about whether they can pass your interview (by demonstrating sufficient max skill when prompted) instead of whether they can implement solutions to the problems you assign to them. This rules out candidates who would otherwise do very well, which is the type of candidate you're trying to find.
I know that you're saying the verbal section of the interview isn't the whole process, but are you sure it's an effective one?
It might be positively misleading. A candidate who is very strong in the area you're looking for is also likely to be someone who will get your questions completely wrong, because they're not programming. They're talking. So it sounds like you're selecting for people who can talk well: those who can show strength during your interview when prompted verbally. Is that the right metric to find talented candidates?
If you were to put together a pipeline where e.g. you give candidates an XCode codebase and say "There are bugs in this codebase, and <specific missing features>. Implement as many fixes or improvements as you wish or have time for, then send us the code," you would have a mechanism which selects for candidates who are ~100% competent, since that's exactly the type of work they'll be doing on a day-to-day basis.
Some candidates wouldn't want to do that, so perhaps there should be an alternative for them. But it'd be vastly more effective than quiz-style questions during a timeboxed interview.
It's possible to come up with endless reasons why it might be a bad idea to set up a pipeline like that. But all the companies that have set it up have been shocked how well it works when they rely solely on that test. Instead of an opportunity to show strength during an interview, the candidate is able to directly answer the question "Can they do the work?"
EDIT: From one of the other comments (https://news.ycombinator.com/item?id=13834231):
> I applied through their project track. It was described as a low-pressure way to write your code ahead of time and talk about it in the interview. The interview was, instead, about making changes to my project while Ammon watched. (Also, there was a request to derive a formal proof while Ammon watched. I didn't get it.) After which I got a rejection saying that my project was great but my interview performance was so poor that they wouldn't move forward.
It sounds like Triplebyte almost has the pipeline described above, but it won't work if you watch the candidate or ask them to do more work. The project alone has to be set up to be a sufficient demonstration of skill.
The other reason I dislike this type of interview question is that if the interviewer never proposes a superior alternative at the end, you get no opportunity to challenge them. How do we know my solution didn't solve problems the other party didn't see?
For context, it seems the "preferred" solution was to build a wrapper around existing memcache. However as a real-world engineer the solutions I was solving for (90% of your clients won't use this wrapper at a company over 100 people, so we want to avoid key collision with people who aren't using this driver) were not the theoretical ones (How could we make the header only 4 bytes!) that the interviewer was evaluating on.
Plus on top of all this, I have no idea if the person interviewing me is unaware that memcached supports atomic increment, knows but doesn't care, or is deeply concerned about preserving this functionality. There are dozens of facets that an individual could consider "important" in this type of problem, and there's no objective basis for most of these concerns without a context of the user (because that's what all engineering comes down to after all).
My guidance to companies in this type of situation is: A great engineer can do 80 hours of work in 15 hours, but not necessarily do 30 minutes of work in 29.
Large take-home projects (and trial employment) totally are better ways to evaluate engineers than interviews. Unfortunately, they require major time commitments from the candidate that many engineers are not able to give. Most (about 80%) engineers select a regular interview if given the choice. I do think they are a good option to offer, but they can't (unfortunately) replace interviews in the majority of cases.
We've also tried debug sections (where we give the candidate a program with bugs in it and ask them to fix test cases). This works great as a portion of the interview (but misses some people with other skills, so it can't be the entire interview).
I do think trial employment can be a great thing. But it's not a universal replacement.
I'd be interested in seeing the argument for why this is a good thing, beyond the fact that it enables a company like triplebyte to exist.
To me it seems like these concerns always boil down to the same thing: tests that try to quantify something that may be unquantifiable. I suspect this is because it's not cost effective to facilitate a process that digs well beyond engineering trivia.
That's really, really bad statistics.
In general if you're going to provide a critical comment I think it would be better for the community here to expound on it a bit so everyone can understand your argument.
This is directly in conflict with this comment from Harj:
> The metric we optimize for is our onsite success rate i.e. how often does a Triplebyte candidate onsite interview result in an offer.
If you want passing your interview to mean that a candidate will pass their onsite interview because your interview is tailored to deliver people who look good in an onsite, you're acting as a recruiter, trying to give companies whatever they say they want. This model provides a lot of value to companies but zero value to candidates, since anyone who passed your interview would have gotten the job anyway.
If you want passing your interview to mean that a candidate should pass their onsite interview because they would perform well in the job, you're acting as a credential, telling companies what they want. This model provides value to companies and to candidates (assuming you can tell who will perform well). These aren't the same model.
Your candidate-directed advertising leans heavily towards the second model ("just show us you can code!"). That makes sense, since that's the model that provides value to candidates. It's disappointing to hear Harj say that what you really believe in is the first model, and disconcerting to see you openly disagree with your cofounder about what your company is trying to do.
Assuming that, would you accept the position to be consistent?
How do you rate job success?
Also, candidates who do too well could be too good for the job, since Google supposedly likes having incredibly overqualified people maintain do-nothing internal apps.
What I didn't like was the chats with the companies that followed. It felt like the whole Triplebyte interview process never happened and we were starting from scratch..
For me personally though the main value was in getting contacted by companies that wouldn't have otherwise contacted me. I would have been willing to go through technical phone screens for those companies if needed, but not having to do so was an extra bonus.
Which is what? A short chat with the recruiter and an 1 hour phone interview.
With TripleByte instead you get something like, an online questionnaire, 1 two-week project and one 4 hour video interview. (Please correctly me if I don't have it exactly right).
How is this an advantage if you still have to do a call with the recruiter/company, and then on-site interviews?
> the main value was in getting contacted by companies that wouldn't have otherwise contacted me
That alone is a solid reason to use Triplebyte.
I do remember doing a two(or maybe one) week project, and could swear that my final interview was more than 2 hours. Maybe the process changed since I last interviewed?
Almost all other fields are capable of trusting third parties to certify competence. They interview for fit, but with a basic understanding that the candidate is qualified.
Employers waste less time giving interviews, and benefit from a more accurate filter than they could develop in-house.
Employees waste less time giving interviews, and will find out early (i.e. by failing college classes, prompting them to switch majors) if the work isn't for them. Good practitioners are unlikely to be denied their dream jobs by the unreliability of the interview process, and truly incompetent practitioners aren't allowed to keep trying until something sticks because (below some threshold) they'll never get in the door. They're given the benefit of the doubt for making it through the main gate (undergrad/bar exam/whatever) and from there, judged on the basis of their performance in their actual role, not how well the study for a song-and-dance routine that approximates the role.
Triplebyte's value proposition is to be such a trusted third party for software engineering, since universities aren't. If companies don't trust it, and view people who passed Triplebyte with the same "incompetent fraud until proven otherwise" lens they apply to everyone else, then Triplebyte isn't adding value for anyone in the ecosystem.
I think you can realistically expect the skills test to be culled from interview processes: in other fields, it never even appeared.
Employers in pretty much every other field complain that credentials are close to meaningless. As a result, they heavily emphasize work experience and demonstrable work accomplishments in hiring. This creates a problem for newcomers who have difficulty bootstrapping experience. Software engineering is unique in that you can at least get some idea of a person's ability through a day of interviewing (whiteboard interviews may not be a perfect measure of competence/ability, but they're way better than what is available in most fields), and don't need to use experience or credentials as a gauge of ability (though they use them as screening criteria to sift through the many applications they get, which is where Triplebyte comes in handy). The fact that employers directly assess the skills of job candidates is a benefit, not a drawback, for the field of software engineering.
>Triplebyte's value proposition is to be such a trusted third party for software engineering, since universities aren't. If companies don't trust it, and view people who passed Triplebyte with the same "incompetent fraud until proven otherwise" lens they apply to everyone else, then Triplebyte isn't adding value for anyone in the ecosystem.
Triplebyte is adding clear value to the ecosystem by finding candidates who would have otherwise fallen through the cracks and never gotten an interview in the first place, and matching them with potential employers. In my case, for example, Triplebyte has provided value to both me and Asana by matching us up together. Because of my nontraditional background, had I applied directly to Asana, my application would likely have never made it past their resume screen, and they would have missed out on a great candidate and I would have missed out on a great opportunity.
I wouldn't say it was particularly bad, but it was worse than average compared to my other interviews.
To be fair, they were super friendly and responsive to my critique, and I found their interview notes/follow-up helpful and accurate. Still, if you're going to provide interviews as a service, you should work hard to make the interview a positive experience for the candidate. I'm loath to apply to any company using Triplebyte for now, but I'd probably do it if I really wanted the job.
The challenge was to code up a regex parser in three hours then discuss in an interview.
During the interview I was asked to add (IIRC) a Kleene operator. I repeated back his explanation of what a Kleene operator is. I explained how that definition would impact my choice of how to implement it. During the implementation, I made repeated references to that same spec. I got it working.
Then, he told me that it didn't work, because a Kleene operator means something completely different than what I understood. He apparently wasn't listening the whole time because I repeated back his spec several times when implementing it and he never corrected it!
(Perhaps this was some subtle test of "see how they react to impoliteness"?)
More importantly though, it was rejected for not being an elegant state machine implementation of a parser, which made it hard to extend. Which is fair, in a way. I knew, abstractly, that that was a better way to do it and I would have gladly read up on the concept and written my implementation that way. But with the overhead of setting up the codebase, docs, and tests, I would have exceeded 3 hour limit that they trust applicants to hold themselves to.
Apparently, the right way to proceed here would be to learn state machines, severely exceed the 3 hour limit, and then lie and say it took me 3 hours. Is that what they're selecting for? Or perhaps for people who already know state machine implementations?
Ouch. That is truly awful. I'm sorry that happened to you. The irony, their site claims:
"The existing hiring process is broken. We’re building a new kind of interview that evaluates tech skills, not credentials."
If that's their mission there's really no excuse for this. I will be sure to avoid them. Thanks for sharing.
That session was overall certainly energy-intensive, but then no more than a good interview session with someone who knows what they're doing interview skills wise. But at a more general level only Triplebyte would know about interviewer variance and whatever secret sauce they have to maximize SNR during the screens, so I can't speak about that.
Ammon is a smug dude, no doubt about it.
Edit: to be clear, I passed the interviews.
I mailed the guy (forgot who he was, but his email address showed up on one of the pages) since I had some questions. He responded once and never again.
Then they had the audacity to email me again a few month later to schedule a call to give feedback for their platform.
Yeah sure, I am going to take extra time from my day to provide you feedback for no return. If they had sent it as a web survey, it would have been probably better.
Finally, they can only schedule interviews on two days a week. Really?
My interviews with triplebyte were definitely some of the hardest ones I had during my search, but they also focused on things I consider important in developing software in a way that no other interview pipeline I've been through has. It may be that my overwhelmingly positive experience with them is not reflective of most, but I think that's unlikely given the amount of work they put into standardizing their process.
Being on the side of engineers in the hiring process does not mean they can take it easy on interviews, and my experience would suggest that whatever personal intensity you may feel in an interview with them is very likely to be outclassed by any decent sized sample of interviews at a big company.
I'm definitely bullish on their model, and while the process isn't perfect I'm really excited to see them moving the needle on interview quality in the industry. I sincerely wish them the best with the expansion.
It's been a great experience so far, even though several of my favorite companies in the pipeline have gone incommunicado before I could even talk to them :-/!
Bigger companies like Facebook don't need to let engineers know they exist, a huge number of engineers have already applied to them at some point. What they've realized is the way to hire more is to find the good ones that are sitting in the resume review stage of the hiring process, not getting looked at because they don't have stand out credentials.
One thing that's struck out to me in a surprising way about the average startup hiring process compared to larger companies is speed. We've found the larger companies are actually faster to move on the first step of booking a call to speak with our candidates. This completely reverses when it gets to bringing someone onsite and making offers though. Bigger companies take longer to do both and this is where startups have a hiring advantage.
How do the types of engineers Apple or Facebook want compare to that?
I totally understand if this is part of the secret sauce and the answer is "just use it".
What founder wouldn't want a Child Prodigy on their team, someone who's 'going to found a company when they're older'? Someone with hustle and chops but happy to be a junior member.
Likewise who'd want an 'idiosyncratic' Academic? Ignoring the fact that massive tools like Scala are some academic's project. Or that Google was founded by two PhD students...
Obvs the former is going to get many 'yes please' and the later 'umm, maybe'.
I think this is the same trap as the old "smart and gets things done" post from Rands. Basically it just implies the programmer reading it is a super-genius, so now they'll want to keep reading your blog in case you compliment them some more.
In retrospect I kind of realize that Triplebyte is really more targetting as a service for more senior positions (at least for now) but I think this is a great move for Triplebyte to get a bigger marketshare and wish them luck.
Yup, I was one of those people whose resume kept getting sucked into the review black hole due to lack of stand out credentials. I understand companies are swamped with applications and need some way to screen applicants even before phone interviews, but it was rather frustrating knowing I would do perfectly fine if I only I were given an interview.
Triplebyte was an immense help here (just today I accepted an offer from Asana that I got through Triplebyte). I'm really a fan of your philosophy and what you're trying to do to improve tech hiring, and strongly recommend people (especially those without great conventional resumes) to give Triplebyte a go.
I found out later I didn't pass the interview because my solution to the coding exercise didn't work at the end of the hour. We had continued past it and I thought I was safe because I was complimented on its organization and straight-forward interpretation. I finished it that night:
https://github.com/blitmap/coffeescript-snippets/blob/master...
https://github.com/blitmap/coffeescript-snippets/blob/master...
I had fun but I felt like they got more out of it than I did.
I don't work with Apple or Facebook, but in the past four months recruiting part-time I've had 15 placements. Over 65% resume submission to on-site rate. And over an 85% offer close rate (close counts if I get 3 offers for someone and they choose one). I can match relatively well without needing to put candidates through a day of tests and I save candidates time by sending them to selectively chosen companies (safety, fit, reach). And I spend a ton of truly understanding each unique process of the companies I work with.
What makes Triplebyte actually unique? What are they changing about the industry? Are they actually reducing bias or is this a gimmick? Are they that different from a recruiting firm with strong lead generation?
Seconded. I don't use LinkedIn or recruiters. When I was looking, I was applying manually to several companies, and setting up phone screens and so on was very exhausting and timing things was complicated. Triplebyte allowed me to combine the primary stages for a couple of companies and helped a lot with prep.
It's the technical interviewing portion that's a pain to have to re-do over and over again. Especially if it involves travelling across the country to do. Engineers are ultimately looking at company and engineering culture to choose between.
The other thing is that for some engineers, they might perform well in one on-site versus another for many reasons such as the questions asked, interviewer rating, or something as trivial as mood. Seems like Triplebyte giving people one-chance makes this difficult.
Ultimately, I feel the main crux of hiring/interviews/finding the right talent is training. If the industry is over-fitting on people who can pass whiteboarding, then why aren't there more startups focused on this aspect? Not just passing interviews (e.g. outco.io), but actually focused on training systems design and algorithms. Universities don't do that in undergrad or grad school.
(1) We don't submit resumes to companies. We give them a profile of the candidate which describes their technical skills and their work history but without any mention of specific schools or companies. For a company to move forward with a Triplebyte candidate, they have to trust in our screening process more than credentials.
(2) Companies agree to not do any technical phone screens or coding challenges with our candidates. Instead they do an initial pitch call and then move to an onsite if there's candidate interest. This saves engineers a lot of time spent in repetitive phone screens.
(3) The metric we optimize for is our onsite success rate i.e. how often does a Triplebyte candidate onsite interview result in an offer. Since our candidates don't go through the regular technical phone screens, any improvement we can make on the industry standard of a 20 - 25% onsite success rate saves the company engineering time spent doing those phone screens. Currently we're averaging 2x that rate across all our companies.
I can see where that's valuable to the companies. What value are you trying to provide to candidates? I already know how to apply to companies, pass the phone screen, and wash out of the interview. If I knew how to pass the interview, I'd still know how to apply and pass the phone screen. What is Triplebyte supposed to help with?
> If you are someone who behind the veil of ignorance can pass phone screens, then you will wash out 75% of the time through normal interviews.
Veil of ignorance, huh? I'll run down the ways I've successfully gotten a job:
- Amazon (CreateSpace), by winning a contest they hosted. Multiple times. There was also an interview for this, though not of a problem-solving nature.
- eBay (Milo), by passing their online hiring challenge. There was no in-person interview for this; rather, they had me come in and work for a day.
- NCC Group, by passing their two challenges. There were in-person interviews for this too, largely consisting of them asking me if I knew how to do things and me saying "no". (I was told afterward that the reason my interviews had gone so oddly was that I had never provided them with a resume.)
Triplebyte themselves told me that I was exceptionally strong in "academic CS" -- the first time around. When they asked me to reinterview for their benefit, they highlighted it as a weak point.
My rate of success in applying to companies that rely on an interview instead of a project or other objective demonstration is 0%, not 25%. But I feel safe in saying that my interviewing problem doesn't lie in the fact that phone screens are hiding my basic incompetence from innocent companies.
With that said, do you accept new candidates for the NYC area?
Perhaps just responding to their customers (employers) desires. It seems companies these days, companies that are not Google, are looking for Google-caliber people even if they just need someone who knows how to code and is hard working.
When I interviewed with TripleByte I had just come out of a 3-month bootcamp, and spent the prior 3 yrs as a teacher. Most companies did not look twice at my resume and it was very tough to break in. TripleByte didn't look at all and judged on ability instead of credentials. I haven't seen that from other recruiting / sourcing organizations and give them a lot of credit for it.
I tried going through TripleBytes process and answered the programming quiz. Then TripleByte changes their policy stating that I must do a project before I can proceed. I stopped the process there.
Then after awhile when I try to login and try again after a few months it says that I am in some kind of pending state which I can't get out of. I gave up on the service after that.
I don't get it. If someone's done has an incredible body of work behind them, why do you want to hide that from yourself?
Isn't it a great way to pick the best people out of the pool of applicants? 'Ah this person designed the new IR in Google's V8 - we should definitely talk to them'.
When I speak to potential hires the first thing I ask is 'tell me about the projects you've worked on - what have you built in the past'. Am I doing it wrong? Someone could be great at general programming and pass a coding test, but if they have no experience in my field what are they going to do for me?
It would be great to allow programmers to show their portfolios, but that would likely enable interviewers to google them, giving away their backgrounds.
I have a few interesting projects in my portfolio that I am proud of, a reasonably high CS GRE score (a test which is unfortunately not administered any more), and a moderately good topcoder history. However, after being told by an HR person that they would not hire me because of the college I went to (a for-profit college) I started to view the relation between hiring and credentials as illegitimate. So for that reason I hope TripleByte catches on.
Plus, people with a good body of work need less help getting interviews. TripleByte has less value add in that case.
Relying solely on those signals though acts to the detriment of skilled people whose best work hasn't been done at prestigious, name brand companies.
Our approach to removing credentials from our screening process is to prevent ourselves being biased by them and forcing ourselves to build a process that can find strong engineers who don't look good on paper. This is a win for the companies we work with as it expands the pool of talent they can hire from.
If I need someone to build me a compiler, there's no point sending me any number of job applications who did brilliantly on a coding test, if they have never built a compiler before.
(Of course sometimes it's great to build someone up from scratch, but you can't do that for an entire team all the time).
Ah, all makes sense then.
Ok, then shoot me an email, and we'll build the compiler. I'd be happy to find a position that actually involves compiler work at all.
tl;dr Credentials carry signal, but recruiters at companies have gone overboard, giving that weight above all else. We push against that.
Triplebyte has been really helpful for me there, and got me interviews with companies that care more about problem solving ability and are happy to hire someone like me who will learn on the job.
Someone who designed the IR for V8 would no doubt have no trouble directly getting interviews with whatever companies they want, and people look you who screen applicants based on experience shouldn't have much trouble finding such applicants. General problem-solving / coding ability on the other hand is not quantifiable on an ordinary resume / linkedin, and Triplebyte's system provides a good screen (within the limitations of a few hours of testing) to quantify that.
These guys are the real deal, so it's pretty neat that they're expanding out to big-names! I wish them the best!
I applied through their project track. It was described as a low-pressure way to write your code ahead of time and talk about it in the interview.
The interview was, instead, about making changes to my project while Ammon watched. (Also, there was a request to derive a formal proof while Ammon watched. I didn't get it.) After which I got a rejection saying that my project was great but my interview performance was so poor that they wouldn't move forward.
I complained that this wasn't actionable feedback, but the only way they ever responded to that complaint was "I stand by that", from someone other than my interviewer. Am I wrong to consider "you do poorly in interviews" hopelessly vague?
They contacted me, much later, to ask me to be a test subject for a new interview. New interviewer asked me about hash tables, and I responded to his questions with this information:
- Hash tables are the generalization of an array to being indexed by "whatever you want" rather than an integer; they have similar performance characteristics to arrays.
- If a hash code is larger than the size of your hash table's backing array, you would generally handle that by storing the item at index (hash_code % array_size).
- When two objects have the same hash, one strategy is to store them in "buckets", linked lists of everything present in the table at that hash key; another strategy is quadratic probing (where when the index you want is full, you repeatedly square the index until the space you're looking in is empty). Quadratic probing has the downside that when you delete an entry from the table, you have to leave a placeholder in the backing array saying "something used to be here".
- If a hash table gets too full, you generally create a new backing array of double the size and rehash everything into the new backing array. The size doubles rather than increasing by some constant amount so that the amortized time requirement for inserts will be constant.
- Amortized time complexity for a set of operations is the average time complexity per operation (not in expectation, but as measured after the operations have happened).
He also asked me about red-black trees. I could say that red-black trees were a self-balancing binary tree with the property that the length of any path root-to-leaf in the tree was within a factor of two of any other path, and that I wouldn't be able to write a red-black tree off the top of my head.
New interviewer, believing that I would be interested in reapplying to triplebyte, did give me feedback on what concrete actions I should take in order to do so. Specifically, he said I should focus on studying red-black trees (OK, fair enough, I guess) and hash tables. I thought I had pretty good coverage, purely within the interview, of hash tables. Is that wrong?
Your hash table knowledge is perfectly fine. So is your red-black tree knowledge. Roughly zero percent of programmers implement those during the course of their job, so knowing their characteristics is more than enough.
There's probably no way to know what they were looking for, short of asking them directly. But you have at least two options: (a) roll your eyes at anyone who tells you that you need more knowledge about red-black trees or hash tables to be an excellent engineer, or (b) realize it's a game, and play it with a passion.
Both routes are perfectly valid, and personally I prefer route (a). But if you're deciding to do route (b), you could study as much as you can on the subjects, quiz yourself on various trivia related to red-black trees, look up related interview questions, etc.
This is a bit of a tangent, but from a motivation standpoint, I've found it's optimal to think of interviews as a lottery ticket with 10% chance of winning regardless of your ability, rather than an as an obstacle that can be failed due to lack of ability. There's no reason to be discouraged when someone rejects you in a world where people reject engineers for reasons that are essentially random.
You were asked to drive a formal proof? Why exactly? How often is that asked at an actual tech interview?
(a|b)*a(a|b)^n
He asked me to prove that 2^n states were required while he waited. I didn't know the proof and wasn't comfortable trying to produce it while being watched. There's no upper limit to how long producing a proof might take.
Out of curiosity what book is that DFA problem from?
That proof isn't a problem in the book; it's mentioned, with a hint for those inclined to derive it themselves, in the running text.
First Round: Online quiz: pretty easy, very intuitive questions. Different languages, but concepts are language-agnostic.
Second Round: Onsite (I live close by). One OOP coding question, Lightning round of very basic college CS concepts, and debugging session.
First Round Matches: The company tries to match you with 10 companies; I got 5. All the companies were either YC or top-vc funded. I received responses from 3 companies; the startups move very quickly -- onsite within a week. However, I've interviewed with 2 of the 3. Did not get an offer from either (qualification and mismatch in culture-fit).
PROS: - The team moves VERY fast when it comes to companies (really tough to think of PTO excuses) - Responsive - each candidate is assigned to a single talent manager - After the initial matches did not fit my liking or yield any results, my talent manager immediately started looking for new matches.
CONS: - Initial matches were not what I was looking for/interested in. (hopefully being fixed).
So far, the experience has been positive. Personally, the lack of stress of finding places to submit to is a huge plus.
One effect of using TripleByte is that we've been able to focus our interviews more on engineering design and seeing how someone thinks. Since we've found TripleByte's interviews for programming skills and knowledge to be quite good, we get to spend more of our time and the candidates' time on areas other than coding questions.
Unlike most people in this thread, I thought their interview process was much more relevant and tangible to real engineering applications than most companies I've interviewed at. I was not presented with any gotchas or arcane algorithms questions.
Finally, the big draw to them as an engineer is that it significantly cuts down on the amount of time you spend on the phone with other companies before going on site. I'm still holding a job (hence the anonymous account), so looking for other opportunities was somewhat prohibitive to me.
[1] http://kellysutton.com/2016/10/20/visualizing-a-job-search-o...
Thanks!
Nice UX informing it so upfront anyway.