Harj Taggar Is Building a New Technical Hiring Pipeline with TripleByte
techcrunch.com
techcrunch.com
Our plan is to continually experiment with new ways to measure technical ability, that aren't as adversarial as most standard practices today. We'd love to hear any ideas for more experiments we could run.
At http://InterviewKickstart.com, for example, we're starting to find some interesting correlations between personal confidence-level of an engineer coming in, and their employability. Yet another correlation is between academic achievements and employability. Small data, but interesting still.
Confidence helps you build a rapport, which is what interviewing is.
Academic achievement, however meaningless that GPA number is, is still a proxy to you working hard for a goal (for the most part). Job search is a goal.
Have you any plans to extend beyond the Technical assessment to help quantify "a good fit with the mission and team"? I'm working with a couple of companies doing interesting things in that behavioural space - and would be fascinated to see outcomes in that field from your data-driven approach. Good luck to you Guillaume and Ammon.
As much as possible, persist the interview results per-candidate and make them reusable so people don't have to re-do the same work over and over and over again. One of the worst parts of interviewing is answering the same basic CS 101 question in every interview because everybody reads the same "how to interview a programmer" post written 15 years ago. Yes, 22 year old interviewer, I know what a compiler is, what an open source license is, what linux is, and how to write a linked list. Also, I've written code for NASA, but you have no basis to evaluate that against your limited knowledge and life experience, so keep asking things you learned two years ago.
I looked into doing something like this a while ago. The long term goal is obviously to become a candidate marketplace where you hold all the quality cards and charge a fee to companies who want a pre-vetted, guaranteed ("our programmers are warrantied against hip dysplasia!") employee pool.
The end result is essentially credentialing/certification where companies only want to hire people "Certified by X." Then, those people can also job hop easier because they are "certified" and don't have to repeat the same interview process everywhere again. Mild speed bumps in credentialing end up being: re-certification (sure I know all this now, but I may not remember any of it 6 months later), bleeding-edge certification (do you even react-goober-swift?), and maintenance certification ("I know this 20 year old technology nobody else knows!").
The reason special people will run away screaming: we don't want to be treated as interchangeable cogs. We are unicorn pony snowflakes. We want our $30k/week contracts and don't need to be evaluated by another programmer's static scripts pseudo-determining what they think is a fair evaluation of our bespoke, artisanal knowledge and abilities.
It seems to me that hiring teams of programmers that are known to work well together and have proven track records is a smart idea. The reason is that hiring individual programmers, who may or may not get along with one another, and whose individual hire decisions may or may not cover the breadth needed or may result in redundancies - is an inefficient and strange practice.
What if you needed an IT department and could hire groups of programmers who have, collectively, proven track records of managing IT well? Need a website done? Yeah, you could hire a designer, a backend server woman, a database guy, a CMS guru - and hope they all get along well and work together.
Or we could imagine hiring a team of people who manage themselves to get the result you're asking for.
Dunno. Zany idea.
First, doesn't every single programmer based ATS do this currently? Second, as soon as this quantitative information becomes symmetrical (it always does, Google "google interview prep"), how do you prevent coders from gaming ("hacking") the system?
There is only one reason to join a startup as an employee: you really really really believe in the mission and you reallyx3+ believe that the team can execute. Anything else and you aren't doing the team or yourself any favours.
So, what are you really applying for?
So you can have a "worked at a YC startup" on your resume:
Just go work for google or facebook or amazon: you'll actually learn a tonne more about engineering/product dev at any of those places and they are a much better brandname for the rest of your career. Think you wont get into those companies? Spend a month preparing for the rigamarole that is a technical interview and you'll be fine.
So you can get investor contact:
Just apply to YC/techstars/500startups. Or email investors directly. Any of those options work better. You aren't getting any meaningful investor contact by being an employee.
So you can learn how a startup works to start your own:
Anyone who tells you this is deluded or conning you. There ain't nothing to it but to do it. The only way to learn how to do your startup is to do it. Nothing else comes close. Anything else is an excuse.
I'd say this is fairly reasonable.
> So, what are you really applying for?
For the opportunity to get in the funnel of startups using this system, so you can evaluate which ones have missions and teams you believe in and find one to work at. Presuming more than a few use this system, even if they don't use it exclusively, that's potentially a more efficient approach than research startups individually, finding ones you would want to work at, and then navigating their own individual hiring systems (and, to the extent you might miss some good startups that aren't using this system, there's no reason you can't also do that as well as the shotgun system.)
In and of itself, that's not a bad list to have. BUT deciding on which one you want to work at for the next 3 years should be more of a "I want to work there" not "let's see which ones will have me". So if you really want to work at a YC startup then go over to http://yclist.com/. Do some due diligence on who excites you ( not "which is the most exciting one that will have me") and apply to that one. Trust me. In the long run doing so will save you a lot more time.
Joining an early stage startup isn't a job, it's a lifestyle.
It absolutely is a job. The choice to pursue that kind of job is a lifestyle choice, sure, but no less so than the choice to prefer a job with a large corporation or similar institution (its a different lifestyle choice, of course, but its no more or less of one.)
My point, which I guess made poorly, is a startup will consume your life so don't just work for "a startup" driven by "startups are cool". Rather, do your due diligence.
...what if you really need the money (and never finished college, so no bigcorps are interested)?
Not every "bigcorp" is so short-sighted that they reject non-traditional candidates.
Are you applying through the blackhole that is an online resume upload? Don't do that.
Find someone in your network who works at one of these places and get them to forward your resume.
Oh yeah, if you really need the money, maybe an early stage startup isn't the best idea :P
I don't really have a network, though; I grew up on a farm and taught myself to program, and everything I've worked on so far has been either a remote contract (which usually ends up falling apart with refusal to pay for services rendered), or a startup (which usually ends up with my salary slashed to 1/8th industry standard under a "we're working for equity, right?" declaration.)
Right now, my wildest dream is a programming job that simply allows me to work eight hours a day and pays for a 1br apartment and food. At this rate, I'm almost considering leaving the industry entirely and working in retail or something.
Unfortunately, most of the obvious advice is true. Since you don't have the "normal" credentials of college/past employement/referrals, you will need to put some time in building an alternate set.
The good news is our field is an awesome one for credentialing yourself. If you don't feel like you have the stamina/drive for a open source projects etcs, how about competing on topcoder?
Good luck.
People who haven't been to college aren't likely to have those kinds of connections. If they do, they're not going to be desperate for work.
> Oh yeah, if you really need the money, maybe an early stage startup isn't the best idea :P
If you really need the money you'll work for the next paycheck no matter who's paying it, even if it's just for the next month or two.
It's really not that hard to meet and develop a relationship with someone who works at FB/Apple/Google/Etc. Especially if you live in the Bay area and go to meetups/network a little.
> I grew up on a farm
Not everyone lives in the bay area. I'm sure there are plenty of places where it really is "that hard", and it sounds like derefr might live in one of them.
The whole world is not the Bay Area. Most of us do not have it so easy as to be able to walk into random meetups and leave with three potential contacts at decent tech companies.
I made a lot of contacts by organizing local meetups, travelling to different conferences in other parts of Europe and fostering relationships online.
I now work as a software engineer at Facebook, prior to that I worked at Apple. Effective networking takes a lot of work and can be achieved even if you are living in a remote area.
The obvious good solution would be to algorithmically recommend startups based on a quick survey. The better solution is to have somebody whose job it is to know all about every YC startup, their teams, culture, and needs spend 5 minutes with you and give a few good recommendations.
Anyway, my response was geared towards how this seems to promote working at "a startup". If you are trying to work at "a startup" you have already (probably) failed because you are getting pulled in by the romantic idea of what a startup is. Don't work at "a startup". Find a company with a mission and team you like and work there.
FWIW, I do think your heart is in the right place: you aren't trying to capitalize of the romantic idea of a startup to hire a bunch of young and naive people :)
If you have any feedback or thoughts, we'd love to hear them.
Do most YC startups use python ?
I'm particularly interested in this as a startup filter. There are a lot of startups out there, many that I don't know about. I'm wondering if Triplebyte could help me find something I'm interested in by filtering out startups that I'm unlikely to be interested in. If they can do that, I'll be a huge fan, and if they can't (because I'm too picky), then it's not too time-consuming or stressful a process to have tried it.
That's exactly what we're going to do. There's an increasing number of tools to help investors find startups to invest in and the same should exist for applicants, since that's essentially what they're doing with their time.
But if you're in a screen share and wathicng someone code and watching them make mistakes, type poorly, or sit there thinking with the cursor just blinking isn't this sort of an interpretive performance art that you're trying to then judge objectively?
How do you avoid being biased to hiring only people who type fast? (There seems to be this obsession with lines of code per minute to the detriment of effective functionality per week in our industry.)
Right now, engineers search through many companies to find a job, and companies search through many engineers to fill a position. The traditional way this is done is an interview gauntlet, whose primary purpose is to verify technical ability. Since that's something that should only be done once per engineer, TripleByte seems to be providing that intermediary service. Then a much shorter set of interviews can be used to check fit, thus saving each candidate and company an incredible amount of time.
hired.com is another company trying to do something like this, albeit using a slightly different methodology.
edit: I accidentally a word.
> "But this is a horizontal technical HR layer that spreads across many companies, instead of being contained inside one enormous corporation."
Perhaps being pessimistic, but I have no wish for a the future where I go from a company that is using TripleByte to another one that is also doing the same, and being told "We know your TripleByte score on your last job was 315, we won't give you $X unless you raise that to 400 in your first year". I also don't understand how this won't create a collusion of sorts amongst companies who all use TripleByte (since TripleByte knows exactly how much they pay you).
[Please feel free to correct me if I have missed out on some fundamental different in the way the company works. I have enormous respect for the founder (Harj) personally, but this seems like a bad idea for the majority of the workforce. ]
Filtering candidates by pedigree is one of the biggest mistakes companies make, particularly in Silicon Valley, but filtering by coding ability (as measured primarily by online tests) is just as naive.
There are tons of people who can code themselves out of a maze but struggle to ship code that solves real problems and creates real value.
A lot of startups would do better with mediocre engineers who can see the big picture than superb engineers who can't see beyond their monitor.
From https://triplebyte.com/manifesto:
> Companies should not have to make recruiting a core competency.
This is incredibly flawed, especially for startups. Recruiting is a two-way street. If an early-stage startup can't effectively sell itself to candidates, chances are it won't be able to sell itself to customers, partners, etc.
Hey Harj,
Do you think that by charging a flat % of the first year salary that TripleByte might be adding downward pressure on the potential salary for an engineer?
It benefits start-ups because they don't have a large cash outlay at once and aligns the incentives of the firm to find great candidates that will stick. It also creates a more predictable revenue stream in the form of monthly recurring revenue.
If a jobseeker won't see that kind of value from a recruiter, they can always take it on themselves.
EDIT People without visas are free to register now. They just can't book phone calls. We'll let everyone who registers know when they can book a call.
What they're really doing is creating a cargo cult tool that gives the illusion of being "objective" and using "data", so that they can sell the serivce based on this appearance, and later get bought out by LinkedIn or something.
Perhaps they're doing a startup themselves because they had problems finding jobs due to hiring practices such as these.
When is technical ability ever the sole criteria? Granted it would be incredibly useful to know up front if a candidate is technically strong enough but you still need to establish if a candidate would work well with your team. Technical ability is worthless if the candidate has a horrible attitude.
That being said, the idea of accumulating significant data over a significant period of time and actually defining what makes a successful engineer is intriguing.
http://blog.samaltman.com/how-to-hire
I hacked out a quick page (that'd inform you it'll log everything, but gave you the problem to solve only after you accepted), that litterally asked you to solve fizzbuzz in any language you like. But it logged timestamped keystrokes to the server and if the tab lost focus.
Of course I wrote a small player that would replay the logs so you could watch the candidate come up with the solution and watch them writing it :)
I just had too many candidates googling questions in a phone interview, so we changed our process.
They'd take the online fizzbuzz test, and then we'll invite them for an onsite interview if they pass, in my opinion it worked surprisingly well.
Too bad I can't try out how these guys are doing it.
The page before they started basically told them everything we'd do. My line of thought was that anyone being able to bypass it is worth interviewing anyways.
I explicitiy told them that we'd log EVERYTHING and asked them to directly write the browser in the JS IDE (well, syntax highlight but no autocomplete) that'll come up, and that they shouldn't switch tabs or windows.
I still think this is fair, for litterally fizzbuzz, nothing similar or anything, just fizzbuzz.
I didn't care about (small) syntax/logic errors or anything (because you'd never run the code), but you wouldn't believe the amount of people unaware of the modulo operator.
It's really just fizzbuzz, not an approximation of real life world, we had on site interviews for that (Switzerland is quite small, not sure how that'd apply to the US).
While that is also human to do, and company cultures obviously develop flavors over time, when it comes to technical hiring, we sorely need an effort to first prove and then to reduce biases, based on data. This is a great effort.
At http://InterviewKickstart.com, we're trying to do this indirectly in a small way. The better you are at solving problems, the less the biases affect you.
You're an engineer. You can create millions of dollars of worth for a company.
Do yourself a favor; learn to market yourself. Learn to create opportunities for yourself.
Want to work at a YC company? Write them directly.
Want to actually make money today, instead of hoping for some grand exit five years down the line? Demonstrate your value (in money), show that you can code, pitch your services to prospective companies and charge a premium.
Please stop joining recruiting companies where you're shoved down a "pipeline" or somewhere where you're considered a "resource" or some outfit that has the name "staffing" in their company name.
You're better than that.
Secondly, this is about helping YC companies hire. Doing the hard, complex work of filtering out for quality candidates, rather than spending weeks of your time interviewing instead of actually improving their product.
Thirdly, not all engineers are battle-hardened 10x SV rockstar ninjas. Some people have programming experience but haven't gotten into the SV echo chamber yet, some people are new to the industry, some people are from other countries, etc. Sure, you ideally want to build yourself up and make a name for yourself, contribute to projects, create some value, but when you're starting out that's basically a load of bullshit as far as useful advice goes, unless you're suggesting people just don't take a job until they've spent a few years self-promoting.
Writing a direct email to a YC Founder vs relying on someone else is far from a "rock-em sock-em neworking people-person". That's basic self-advocacy.
This company takes 25% of your first years salary. Regardless of whether the engineer or the company pays for it, it's less money on the table for you.
YC Combinator companies are well known. If you want to work for one, write them directly. Ask for what you want. Advocate for yourself.
You've probably worked very hard to get where you're at, whether it's your schooling or a long, hard fought career. Don't let someone else represent your accomplishments or your compensation needs.
Similarly, I am not an excellent chef. I can feed myself, but when I am entertaining others we eat out.
This isn't about being lazy; this is about an inability or very poor ability to do something, and outsourcing that ability to another human. If you don't want to be at the party, that's fine, but don't rain on our party night.
I understand all the criticisms, but it seems pretty clear to me why recruiting firms continue to stick around and add real value, despite charging a frustratingly high premium.
My resume has 27 years of showing other employers that I can code. I worked hard to build that resume.
The problem I have these days, being 51 years old and quite public about the fact that I am mentally ill, is that no one believes that I can code.
...and accomplishments, leadership, responsibility (assuming no grossly fabricated lies).
> That's part of why we do blind phone interviews.
Phone calls still reveal a lot. Gender, age, native language, .... Only 100% text based communication is (reasonably) age+genderless.
We've been much more respectful of people (which is good for us) since we stopped looking at resumes.
It also conveys communication skills. What good is a programmer who can write code but can't write a document describing their own work history?
The world isn't programmers. What if you want a non-programmer role? Leads, organizers, directors? Do CEO candidates sit down and write a binary search tree? (of course not, being installed as a CEO means you are one of a privileged class who are given roles, not one who "tries out" for things.)
CEOs and other execs aren't screened with resumes. For better or worse, it's all about their networking skills, perception of previous history, etc.
There's a lot of this kind of thing you can do before even a phone interview, without consulting my employment references.
Everything in my resume is The Gospel Truth, but I am completely convinced that no one believes it. I really am.
That's the entire point in most cases. Don't want to hire a nerd when you need a bro.
Also: relevance! If you're hiring for a Linux-heavy job but the applicant has 20 years of Windows NT experience and they want a Senior Staff Linux Development role, you can either throw their resume away or talk them down to (an insulting) junior position.
In another context, say you're Google. You have 80,000 in-bound interested people wanting to interview with you per year. How, without up front resumes or other relevant information, do you deal? You can't talk to all of them.
That is, their HR people found my website on the web. They didn't find my cover letter and resume in their own applicant tracking system.
That's not the case everywhere; I was hired at Apple in the mid-nineties by applying directly to the company, then sometime later my manager-to-be dug up my resume in their system.
But as far as I am able to tell, resumes submitted to google wind up in the bit bucket.
Google's initial phone screen always consist of just a few very basic computer science questions.
Quite commonly phone screens with other companies consist of "How many years of java experience do you have? How many years of linux experience do you have? How many years of apache experience do you have?"
Oddly, there is never any concern for what one accomplished - nor failed to accomplish - during those years of experience.
I echo your sentiments that I have no idea what goes on in and around their resume system.
Currently, employers expect small amounts of lying about low-impact things - when a resume says "I made this", we all know it really means "I was on a team that designed this, and the company later outsourced its construction". We accept this because it's a social norm.
But say "I worked here" and everyone believes you.
Flatly illegal - yes, but widespread here in These United States.
When I get my business actually making money, I'm going to hire a bunch of homeless people. I really am.
I'm curious about your approach to making sure the candidate experience is really top-notch since it's still your platform, and how deep you plan on going - even good startups may put forth exploding offers, lowball candidates, forget candidates due to high-pressure sales tactics, etc. This doesn't leave a great taste in engineers' mouths.
Pure top-of-funnel filtering is important, but seems oversaturated. The candidate experience and matchmaking process seem like the real differentiators. What do you feel is the best way to address these?