The Only Interview Question That Matters
inc.com
inc.com
I don't see why it's a good question. Presume you get an good answer to the question and every followup question. Now tell me this: how much more do you know about how capable the candidate is of doing the job you're hiring for? And, were these questions the fastest, most accurate way of learning that?
I submit: no. "What were the biggest challenges you faced"? How do I assess the answer to that question? How do I compare it with answers from other candidates? How do I track, over time, which challenges correlate to strong performance in the actual role we're hiring for? "Explain your manager's style and whether you liked it"? Come on.
Don't virtually all of these questions ask me to read tea leaves, to get a sense of whether I "like" or "respect" the candidate? That's not the same thing as learning whether the candidate will do a good job.
Oh, and good luck hiring anyone right out of college this way.
whereas others cap at, "well, I did manage to build the world's first and largest bitcoin currency exchange and reach volumes of about a million a day before we had to shut down."
when you find out they did it by themselves, that is a huge difference. like two orders of magnitude FTE's.
it's like, if some guy spearheaded and came up with the whole idea behind the new Mac Pro, in an era of stagnant desktops, the whole mechanics and central idea (with the heat core). This question lets you know this.
a million examples, this is really just one.
The big issue is, what does this style of interviewing do for you with a real-world candidate selection challenge --- say, 3 developers, all similar on paper?
There is no reason the best candidate necessarily has the best story. The story of your "best accomplishment" has as much to do with random chance as it does with capability. This question demands that an interview somehow de-confound the luck element and the skill element.
And to what end? Even if you got a clean reading on this question, somehow equalized between the candidates, I ask again: do you really know a lot more about how the candidate will actually perform on the job? Because that is almost the only question that matters in candidate selection.
"Gee whiz, I'd say my biggest flaw is that I'm TOO passionate about technology!"
Maybe I coded an iphone app. Meanwhile you coded an iphone app, and fixed bugs in a huge legacy codebase, and did hiring and code reviews and mentored new employees, and contributed to open source projects, and introduced your company to source control and automated testing, and masterminded a trivial code change that made the company a million dollars a year, and you spend your free time running a programming club at a local orphanage.
If we're competing for a job and we're asked about our single biggest achievement, we'll both say iphone app. An employer who wants to accurately gauge who is the better employee should probably dig deeper.
Say I saved a million dollars a year by adjusting a configuration setting, no coding required. That might go over well with a business type, but if the interviewer is all about the programming, it doesn't demonstrate technical mastery. There will be no scope for talking about the technical challenges, the languages and databases I used, what I learned from the experience, or how much I enjoyed it and grew as a person and a professional.
Say I introduced source control and automated testing at a medium sized company. Again, the technical work is limited - you can install Jenkins with apt-get, the real challenges are political and educational, changing 'official' workflows and 'supported' software and convincing people to sink time into using the tools for long enough that they can see the benefits which are not immediately apparent. If I'm being interviewed by a young guy at a startup they might think they have no politics so my skills will be useless - or worse that I'll introduce politics. Or it just won't seem like much of an achievement, after all the six of us at this startup switched from bzr to git in an afternoon.
Needless to say, if the original job description says "wanted: someone to teach us source control and automated testing" that would be a different matter :)
(Of course, in a real interview I'd have a good answer for what you wanted to hear. I'm just saying that an honest answer would be useless.)
It also biases you against people with many small accomplishments as opposed to one grand one. But people who can handle 20 different small tasks on a schedule are also valuable. Your organization probably needs more of them than me (I'm the one big thing sort of person).
When you think about it, the fact that we almost universally hire candidates based on their narratives sounds fucking insane.
I've managed a team and there are those who will find a grand solution to a thorny problem or find a solution to something you didn't think of as a problem and there are others who will plow on to fix countless bugs and make the system better for it. Both are needed, and others as well along the spectrum.
I believe that knowing your strengths and playing to them would be a way to get the best for both sides. Unless of course the interviewee is looking to change course completely.
Job interviews are hostile. In fact, they're one of the most hostile experiences most of us have to deal with. I watch candidates in face-to-face interviews very carefully now, because this is a problem I am studying, and when you stop and consider candidates not as gamepieces but instead as human beings, you'll be amazed at what you notice. For instance, a majority of the candidates I've sat down with in interviews have, subtly but noticeably, been physically shaking.
Job interviews are like a game of Jeopardy! where one of the players is being filmed and broadcast on live television and the other player is playing from their couch and has been given an answer book, and the poor televised shmoe is judged on the same curve as the person on the couch.
Second, you're assuming that the way a candidate interacts with your team in an interview is an accurate representation of how they'll gel with the team down the road. Some of the shyest people I've interviewed have gone on to be the ones I actually end up talking to all day. Part of that is because interviews are hostile and make people shy, and part of it is that different people take different amounts of time to open up to strangers.
Finally, you're describing the "gut check" interview process used by almost every engineering team since the 1990s. We have evidence for how well that process works. Answer: not well.
As an aside: the idea of hiring candidates based on how well team members "get along" with them is repellant to me, as is "culture fit", since they're basically ways of making sure you have a team full of people who listen to roughly the same music, drink roughly the same beer, play roughly the same X360 games, and like roughly the same technology stacks. But that's an emotional argument; the real argument is that it doesn't work.
I think the problem is that everyone on HN thinks things are binary, when practically every scenario in this universe is not. You can't just eschew potential team fit because you want a rigorous hiring process that depends only on candidate qualification. The opposite holds true as well. Here are the two things you need for an awesome team:
1. A team that works well with each other.
2. Individuals on the team can do their job.
>>> the real argument is that it doesn't work.
It's not an argument, it's a statement. It would become argument if it would be supported with something that is not of equal weight with the contrary claim "no, it actually works quite well". Everybody has their anecdotes, do you have more than that?
Is your argument here that conventional engineering job interviews...
(the sort where candidates interview with a series of engineers from the team all of whom ask different sets of questions and seek to gauge from their answers previous accomplishments, working style, technical competence, and team compatibility)
... are generally an effective way for teams to screen candidates?
Or, ask an engineer how many times over the course of their career they've worked with "worthless" teammates, or people who should never have been hired. I've worked across a decent slice of the industry these past 19 years, and from what I can tell, this story is universal.
Another strong piece of evidence on my side is software quality, or the pervasive lack thereof.
Another piece of evidence is fizzbuzz.
Another piece of evidence is the (harmful) trend of demanding that candidates work on a contract-to-permanent basis as part of the recruiting process; what is this other than an attempt at work-sample testing that also happens to exploit candidates and repel the best of them?
I agree with you that close to 100% of working engineers are hired this way. But that doesn't prove anything; it's just as likely, perhaps more likely given the evidence, that what it shows is that development teams can ship marketable software despite terrible inefficiencies.
If you think that conventional hiring does a good job of selecting effective and competent programmers to the median ISV (and note here I'm stipulating ISVs, the best-case scenario for your argument, not line-of-business software teams at F500s), fine. I don't feel like I know a lot of smart engineering managers who are comfortable with the predictive power of their interview systems.
If I as someone interviewing for a QA role ask them a question such as 'Tell me about the QA work you are most proud of.' And they spend the next 20 minutes not talking about anything QA related it's a red flag. And an important data point to me when doing a retro with my peers. Perhaps that recent scenario is a reflection of the folks my company is able to attract in this area.
Conversely, if I am interviewing with you and I ask you, what's the most interesting thing your team has done and how did you help them accomplish it.' And you go into great detail about a deep area in your field that's part of the product your team works on its useful and important information when I need to make a decision. However if you are only interested and talking about me, my technical (or lack of - I know very little about security) understanding and you brush off conversational attempts to gain information that will help me judge the book by its cover it's usefully info when I need to make a decision.
I am not sure why your tone was so hostile? I did not mean to imply anything by my question. I am merely looking for other ways that people have found works for them.
What you need going into this interview is a solid understanding of your corporate culture -- what behaviors you value. Maybe you work for a bank, and the most important things for you are integrity and conservatism. Or, maybe you work for an early stage start-up, and calculated risk-taking and team spirit are most important.
In either case, as you walk through these questions, you ask about the how's and why's of the person's behavior. Doing so is the best way to figure out if the person is going to act in ways that you would want and expect them to. Again, the significance of the biggest thing itself is small, it's how and why they achieved it, gleaned through the 20-odd follow-up questions, that will get you to the answer.
You mention college hiring -- yes, that's a different beast. A lot of companies have success with paid internships the summer before graduation. You can also ask some of these behavioral questions, but unfortunately our educational system doesn't include enough collaborative, project-based work to get a good baseline. Nonetheless, a college student who has done significant summer project work, or who has worked on an extracurricular project, should have some data to extract.
Job interviews suck, and I feel like people who have been hiring in this industry for a long time recognize at some level that they're essentially random functions, made workable only by dev teams willingness to put up with employee churn and extreme inefficiency.
I don't agree that this question is a silver bullet, but I think it's valuable to see what sort of achievment the candidate considers significant, to ask follow-up questions about how they handled problems along the way to that achievment, etc.
The question helps reduce the power assymetry of the interview, by letting the candidate choose something they're interested in talking about and presumably familiar with. I find that people seem more relaxed when they get into the flow of describing something they're interested in, which lets me explore various aspects of their working style with as little stress to them as possible.
Job interviews do suck, but the alternatives I've seen proposed suck at least as bad, though in different ways.
Work sample testing has been extremely powerful for us. In particular: it has enabled us to screen candidates for aptitude instead of experience, and the result has been a series of hires with no prior experience in our (very specialized) field who have completely blown us the fuck away, like the former line-of-biz insurance company .NET guy who, a month or so in, looked at Rails for the first time and found a serious security hole (which now has a CVE), or the other line-of-biz .NET guy who now runs our crypto review board and co-wrote our crypto challenges, or the math major we hired out of college who ripped through the MSP430 CTF in a couple days (and also has a Rails CVE).
We picked up a guy who had done a couple months of Android development after college, and then moved to what I think (from his resume) was a devops job --- I never asked him in the whole course of our hiring process what that job was about, so I don't know. Six months later, he implemented a CRI elliptic curve DSA exploit that involved BKZ lattice reduction and an FFT filter step. The lattice reduction steps on that attack took something like SIX HOURS to run, and when they finished, you still had to get the FFT step working to figure out if you got the BKZ step right.† Months later and I'm blown away just thinking about how he could have gotten that working. In case you're wondering, he kicks ass on the more boring parts of our work too.
Some other things to know about that guy:
* No prior crypto experience
* No prior software security experience
* Less than 2.5 years dev experience across both jobs
* From flyover country
* Physically uncomfortable during the interview (something noted by interviewers on his feedback)
How compelling do you think his "greatest achievement so far" story would have been? We weren't dumb enough to ask, so we'll have to guess.
Before we did work-sample testing, overcommunication about hiring process, demoted the importance of phone screens, and standardized face-to-face testing, we never would have hired these people. Any process that filters outs Seans, Alexes, Peters, and Ryans over things like "passion" or "what is your greatest achievement" is a stupid fucking process. But there it is, enshrined as the default process for our whole industry. And here's a thread full of people arguing on its behalf.
Yes, I think someone is going to make a lot of money correcting that market inefficiency.
† If you're wondering, number of other people on our team who could have helped him debug lattice reduction C++ code: zero.
I agree that work sample tests can be quite valuable.
I think it is a fantastic question to ask someone out of college. It shows those people who are passionate about something and went beyond the minimal requirements for their degree to achieve it. A passionate student could answer this question in a number of ways; he could point out an open source project that he published on Github that has users, some research that he did with a professor that got published, or even a class project that got him interested in the current position he is applying to. I mean, John Resig would talk about the projects he did as an undergraduate that lead to jQuery[1].
If you dig into the details of a candidate's answers to this question, you can find skills that they might have forgotten to put on their resume. Such as the fact that they know git or Matlab. You can ask them whether they used design patterns in their project and use that as a launching point for a short quiz on design patterns.
You do lose a bit of control of the interview process when you ask an open-ended question like this. A candidate can go off on a tangent about something irrelevant to the job. That is a risk. But it could turn out that you learn more about the candidate from his answers to this question than the puzzle problems that you might have planned to give him.
"Passion" doesn't matter.
What matters in a job is effectiveness and competence. Effectiveness gets things done. Competence ensures that what's getting done isn't going to backfire and create more work down the line.
In our common usage, "passion" describes the coder who stays until 9:00PM on a Friday hacking on things. But that property doesn't tell a team anything about whether that coder is the most effective use of headcount dollars. Passionate coders stay late working on compilers for obscure languages, or rewriting ugly but working code into trendy new languages and frameworks. Even if they're building important capabilities for a team's core product, they're doing it in a way that is basically impossible to schedule, on timelines that are impossible to reason about, because "passion" is idiosyncratic.
Passion (about what? quality? new technology? learning? building things? these are all different things) doesn't hurt anything. It's not a bad thing. It might even be a good thing. But it is not a crucial attribute of a new hire on most teams. If you need passion, consider whether you're hiring cofounders, not teammates.
This is to say nothing of my total lack of faith in the ability of engineers to assess "passion". For any given engineer, candidates that happen to do things in their spare time that the interviewing engineer happens to enjoy or appreciate are "passionate". Meanwhile, a supremely competent professional who, given a spec, can reliably estimate the time to execute that spec and then actually nail that schedule, that person is a 9-5er, because they don't have strong opinions about Node.js.
•Shudder•.
Almost every idea that engineers have about hiring is broken.
Even if they're building important capabilities for a team's core product, they're doing it in a way that is basically impossible to schedule, on timelines that are impossible to reason about, because "passion" is idiosyncratic.
Yeah, passion is a bit of an x-factor, isn't it? By itself, it is meaningless. You may get something unexpectedly great or unexpectedly bad. But, using myself as an example again, I was able to get a project at work done well ahead of schedule because I had already been working on something related in my spare time.
Passion (about what? quality? new technology? learning? building things? these are all different things) doesn't hurt anything. It's not a bad thing. It might even be a good thing. But it is not a crucial attribute of a new hire on most teams. If you need passion, consider whether you're hiring cofounders, not teammates.
Inasmuch as someone is proud of a project they did and can talk enthusiastically about it, and that project is related to work currently going on at the company they are applying to, I think a passionate candidate can be better long term than another candidate that is initially more competent and more effective. If experience is the main reason that candidate A is more competent and effective than candidate B, then candidate B may eventually surpass candidate A as a better employee simply because passion drives candidate B to keep honing his skills in his free time. I think this is especially true for college graduates or other junior candidates who may not have anything else to go by but their projects.
These skills can come with experience, but can also be personal traits. An employee that doesn't code the week-end but focuses on his job while on site and takes pride in being professional and reliable is a solid asset for any company. In the long term he might be a corner stone of your team, balancing the ones that are half dead because they didn't sleep at night to work on the projects they really invest themselves into.
You're right, "passion" isn't one of the things to hire for. At best, in my experience, it's highly correlated with the breadth of experience that fuels "creativity" and "effectiveness" - at least, in the sense that it causes people to be persistent against hard and unclear problems. It's fair to ask whether this should be required for teammates or just co-founders -- but even in my most enterprise-y gigs, I've rarely seen a team that could succeed without people with passion helping drive the work.
Maturity is also critical. And passion and maturity are not at all mutually exclusive.
If there are misconceptions about passion floating around the memespace, then you are certainly right to challenge them, but let's not throw the baby out with the bathwater.
I think it's because building great products isn't just a matter of execution. It's also, to some extent, an art.
I have trouble imagining someone being effective and competent with absolutely no passion. I will grant, though, that there are competent people who are more passionate, and those who are less so. I think a successful organization could have a blend of people across the passion scale, but I don't think one could build a successful product business without at least a few people who are passionate about what they're doing. Maybe in other kinds of businesses you could; I don't know.
Of course it gets trickier when the more competent one seems the less passionate. Then it's a question of how much to weight the two dimensions (along with whatever others one is considering). I have no magic formula here; I think it depends on the circumstances.
There seems to be a subtext here that work sample tests give no indication of passion. I don't think that's true; when I've given people programming exercises, I think the craftsmanship of the code (is it well-modularized, well-documented, nicely indented, etc.) is indicative of their passion. Competence, to me, is more about whether the code works.
So yes, I'm using craftsmanship as a proxy for passion, so I suppose next you'll ask, why not just go for craftsmanship? The point we seem to be coming to is that passion is difficult or impossible to measure directly, so why not stick to qualities that are more measurable?
I don't think everything important is easy to measure.
As bad as management's actions were for me, the absolute worst thing was watching the change in this guy. Before, he would constantly seek out every way to improve the customer experience and go above and beyond. He turned into a guy who literally said "I don't give a shit."
Almost every person I've ever worked and every job with has been passionate about doing their best, unless actively stopped from doing it. The one or two who didn't could not have been filtered out by any kind of interview question because they knew how to bluff their way through the process. But you can tell after working with them for a few months that they just try to do the minimum necessary to not get fired.
An attempt to recruit for passion is a bad sign. You can't recruit for good morale, either.[1] Either your company is a nice place to work or it isn't. Either your company will bring out the passion that almost all developers have or it will destroy it.
[1] Yes, there are assholes who destroy morale, just like there are total clock-punchers. Sometimes you can tell by having a normal human conversation, and a normal human conversation should of course be part of any interview.
If you are are trying to find the liars, assholes, frauds, and professional slackers, you need to realize that
1) they have you vastly outmatched in this contest unless maybe you are bringing in psychologists trained in this,
2) there is a social cost from treating your prospective candidates as if they are one of these things.
Thus, the only way to be effective and competent is to have a passion for what you do. Otherwise, if you limit yourself to 9 to 5, you'll be treated as such. And it works like that in many industries, and in those industries, those with a passion will routinely do better, whereas those that don't, will stick to the 9 to 5 places.
Almost every idea anyone has about interviewing is wrong.
Interviews are a terrible way to recruit. It's amazing to me that STEM companies with money and scientists still use interviewing in recruitment processes and haven't found something else.
There is the "on the job interview" where a candidate is invited to do the actual job for a day or half day. For obvious reasons this method tends to be used for minimum wage positions, or for places where you're recruiting people who you know might interview poorly. But it seems to be a great way to recruit people.
That actually tests a set of skills, which puts it well ahead of various interview questions which test nothing of relevance. But it's also a limited set of skills, some of which a recent graduate might reasonably lack but might be reasonably be expected to obtain quickly (if the right kind of smart, committed, a quick study etc).
The problem is I don't know really know how to test for anything but current ability to write reasonable code for a not-too-large problem. The more experience I get interviewing, the more decisions feel arbitrary to me. And I'm pretty skeptical when other people claim to have it figured out.
Though this question can be a testimony to see whether the candidate is articulate at describing things. People who have trouble to describe problems to another person is usually not a good co-worker to have (no matter how competent he or she is). You will just waste so much time going to put his or her words into complete picture. or the new co-worker will work on his own problem the whole day and no team work.
I deeply respect you for this and many other posts. This was a really good point. I hope i can run into you at blackhat or RSA this year.
What I like to ask is what project or achievement people are most proud of. That forces them to use their own subjective judgment rather than attempt to cast about for some sort of objective definition of "significant".
There are many reasons why I love this question.
First off, it puts things squarely into the realm of the real instead of the abstract. It can tell you a lot about whether or not you're dealing with someone who gets things done or just someone who "does work".
From there it's a quick jumping off point into the specific skills and competencies of the candidate, viewed through the lens of productive work. It's a way to see what skills people have in sort of a sideways way, because you see what skills they actually use, rather than the skills they tell you they have.
Perhaps most importantly, it tells you about a person's priorities and values. It tells you the sort of things they think are important. Did they do something that actually delivered value to anyone? Did they do something that required overcoming obstacles or did they just cruise along and build something "cool"? Or something that was technologically sophisticated but useless in practical terms? Do they value titles and money over real accomplishments where something of lasting importance or value is built and delivered to users?
It also gives an opportunity to explore a lot of issues that are hard to suss out otherwise. For example, does the candidate have initiative or do they just follow along? Do they strive to improve the way things work or just tolerate the status quo? That can be evidenced if the project they describe was something they took on of their own accord or was handed to them, whether it was an internal improvement project, and whether they had to fight against the establishment to get things done.
So there you go. Values, skillset, competencies, initiative, and ability to execute. You won't necessarily get all of that out of a candidate every time you ask such a question, but there's no such thing as a perfect interviewing technique.
For someone straight out of college or without much experience of course you have to modify how you interview them, but that's always going to be true, and sometimes this same question can still be valuable, just not in exactly the same way.
P.S. Also, just having a good set of questions or a good process to go through is not enough to make someone a good interviewer. Interviewing is hard, easily one of the hardest things in the profession, and only a small subset of folks who do it are actually good at it. I don't know exactly how to make a bad interviewer into a good one, that's a subject that could fill volumes and volumes. But I think this particular kind of question is, on par, much more helpful than not and a step in the right direction.
How do you decide which past accomplishment is worthy of a 30 minute conversation? Usually what comes to mind when asked this question are simply the most recent things I've worked on. If you've had the misfortune of doing crappy work for the last few months then it makes it a bit harder.
I've seen lots of these kinds of posts, trying to "solve" interviewing. It's a topic close to me right now as I'm in the middle of the interview cycle myself. I just don't think it can be solved. There are too many variables and too many non-scientific aspects to it. Most of the time interviews are decided in that first impression with your interviewer - it feels like a primal instinct people have about other people.
One thing I can say though, is that you aren't going to find the perfect candidate with just one question.
I'm in job-hunt mode so I guess I'm going to sit down and prepare an answer for this one, too.
But, like you, what does that say about the next 15 years of my career? I'm a waaaaay better developer than I was at the end of my first job.
It is important we understand the words that we left out of the original article.
The point of the question isn't to get an impressive project. While we, the interviewee would love to say, guidance computer on Voyager or the Ray Tracer used in Toy Story; the point of the question is get to know the candidate and how they operate.
What have I worked on since: ACDZip, ACDSee, Tracktion, EAW Resolution, SPIN Review. ACDSee was a brief success, but the rest - meh.
Any given project will be calibrated somewhere between the extremes of "just ship it" and "take 5 years to make a perfect 1.0". A developer who, based on experience, is flexible enough to calibrate themselves to match the project -- who can adapt to reasonable balances of "do it faster" vs. "do it better" -- is valuable.
This magic-bullet interview question had me depressed and made me feel like a failure. I too have nearly 15 years of professional experience. My last projects that I can show off were 4 and 8 years ago.
I don't feel like a failure anymore. I got a reminder that success depends on blind luck a lot and is rather overrated. Hard work, improvement and patience are still essential.
P.S. I have used both Miranda and ACDSee in the past. They reached my highest criteria for software, that few applications reach – were small, fast, reliable and very useful. I wish FigBug best of luck.
"Tell me about a project in which you demonstrate the skills and character attributes that I am looking for without me telling you what those are."
A good interviewer will ask questions like this only having given you enough context to choose the most appropriate example.
A bad interviewer will not, instead believing that the less information they give you the more revealing your answers will be. All they achieve is to introduce more chance into the interview process by leading you to pick less relevant examples.
I would probably use some of the stuff I did on back at my first job working in the mathematical modeling section of a world ranked RnD organization.
If it was big data I would use my experience back in the 80's doing map reduce for British telecom - and that proves i can ship :-)
If this is the best question you can ask people, we got really lucky, because this is the essential question on all YC's application forms for events (e.g. Startup School) and has been one of the core questions on the YC application since the beginning.
It's part of a bigger problem seen here in Europe (at least for those who don't move away.) Online work profiles like Linkedin are an obsession here -- and these sites are all oriented around highlighting school and businesses associations, instead of around the list of projects one's worked on. Useless for hiring.
This illustrates the two very different approaches to the problem of getting hired:
A) If you convince school to certify you -> you will convince a company to hire you.
B) If you achieve relevant work -> a company will hire you to perform more work.
I suspect there's a certain self-serving aspect in that on the part of schools and universities. If students had the view that their education was going to be a foot in the door; maybe listed as a line at the base of their CV, and the time at uni listed as another job. - If students were expected to ask of the stuff they were doing right then, 'What is this helping me achieve that I'll be able to put on my CV?'... the answer would frequently be, 'Bugger all.'
It's not in schools of universities interests to give you good advice if that advice contradicts a view of the world that supports the importance of their services. And there are precious few other sources of advice that are likely to know what they're talking about.
Is this deceptively informative? I hate to cite Malcolm Gladwell but I do think Blink shows us that first impressions and fleeting impressions can be dangerous and wrong. What do we think "real passion" is? Does it really show on the face, or do we only think we know? Can being flustered or off-kilter skew the impressions we give?
Interviewing is just a screening process, and of course fluster increases the risk of a false negative, but in that situation I'm much more concerned with false positives, and in that respect: No, watching also the body language for that kind of question isn't deceptively informative; it's much more informative than reading the same answer from a written text.
Looks like it's about both. It had been a while since I'd read it. The downside of snap judgements is obvious, so I remembered the more interesting part, where trained experts' snap judgements are actually good.
This is a subset of the fact that the employment paradigm, with its multiple perverse incentives for all players, requires us to be liars. If we're very lucky, we just have to know how to be a liar.
What if you are interviewing a kid from south-side Chicago or kid from third world who against all odds went to college. Is smart and motivated and could be a great asset to your company but never had the exposure to great ideas.
You can argue that she can just say 'Going to college was my most significant achievement' but are you in a position to rationally consider that as a 'significant achievement ' ?
In a age if great socio economic inequality very few people at the top of the society get to have ' significant accomplishment', it is more of an indicator of you socio economic status than your actual capabilities.
The main point, which is key to hiring, is that you need to ensure that people are answering questions based on actual experiences, not hypotheticals.
When you ask someone "how WOULD you handle xyz", they feel free to answer with ideal behavior. When you ask "how did you handle the situation you just told me about," they tell you what they actually did.
You should not be judging the significance of the person's achievement, in most cases -- you should be probing on WHY they think it was significant, what made it significant, how they worked with others to realize the achievement, where they stumbled, etc.
I don't look at academic success too hard when I hire, but if someone really worked their tail off to beat obstacles and get that 4.0, I'll find those stories to be a great indicator.
You can say, my greatest accomplishment this year was maintaining a large rails app, or you can say, I lead a team of 3 developers to architect and deliver a custom commerce app that helped a startup quadruple their revenues over the next quarter.
Both are the same story, one of them is far more interesting. I would expect your hypothetical character to explain why going to college was his most significant achievement in an interesting way (not "I went to college" but "I overcame all $these_obstacles to achieve my dream of a college degree"), and then I don't see why a company wouldn't be excited by his accomplishment.
Answered well, and if you listen to the answer properly, this question will tell you how someone overcomes obstacles and handles adversity.
This question belongs on a country club application not on tech job application. Tech industry is one of last few places in america where one can hope to turn things around in life. Become great by getting exposure to new ideas and new memes.
'what is your most significant accomplishment to date' is much more telling.
pg is not hiring for a role, tho. he is trying to assess determination for a particularly difficult lifestyle choice - starting a company
How classist of you. There are plenty of people from working class backgrounds who achieved great things. Just because you started in poverty doesn't mean that you've never achieved something worth talking about in response to an interview question like this.
Honestly, I think disadvantaged people are in some ways at an advantage here--we have so many more limitations, that we have to become quite creative in our attempts to get around them. The more advantaged people can take a more convergent path because there are fewer obstacles in their way necessitating an annoying (but ultimately educational) detour.
Only in the specific (and almost certainly relatively narrow) context of the time and place in which the achievement occurred. The question is, at best, a vague and inaccurate heuristic measure of the thing you note. The answer to it, and follow-on questions, may have little or no bearing on the position the candidate is interviewing for.
That is exactly the sort of achievement that impresses us, actually. For example, Qasar Younis started life in a house with dirt floors in a village in Pakistan. His family moved to the US, to Detroit, when he was 7. He appears to have worked his ass off from the moment they landed. We funded his startup, Talkbin, in Winter 2011, mainly because we were so impressed with him. Talkbin was acquired by Google soon after Demo Day, and he is now a part time partner at YC.
(Why do so many people assume that after 9 years of picking founders, we still have huge blind spots that are obvious to them but not to us?)
I am merely suggesting( and hoping) that this not something that shouldn't be emulated by everyone.
Can we play G.H Hardy once in while and discover Ramanujams from social and economic ghettos?
This is not an example of someone who is from the third world or from a ghetto. Immigrant parents have strong work ethic which they instill in their kids ( I belong to the club, so I know).
I am talking about girls from southside Chicago whose idea of 'achievement' is not getting pregnant by 15.
http://www.usnews.com/education/best-high-schools/michigan/d...
Anyone who doesn't think excelling in that environment isn't a highly significant achievement is sadly mistaken.
Is there a story to it - like somebody asking this exact question, passing on such girl, and then regretting the decision?
I have always found the YC application strange in that sense. So much emphasis is given to the question of the "most impressive achievement", but in order to convey any achievement with context, one needs to be quite verbose with their replies. Given that an application only has few seconds to impress, conveying anything is a challenge. Has YC given much thought to an essay type application?
I am proud of where I am today. I grew up in Mumbai, India (not quite a village). We weren't rich. I still have scars on my right foot from an abscess caused from a piece of glass that had pierced my foot while playing soccer bare feet on concrete. We couldn't afford shoes. Luckily my father's business started picking up around the time I entered engineering school. The plan was for me to graduate with a bachelor's in structural engineering and then we could work together to grow the business further. At 20, I earned my bachelor's in structural engineering. Unfortunately my father had died in an accident at work when I was in my penultimate year of engineering. I managed to get enough resources together to be able to attend grad school in the US. I met my wife there. BTW, she has an amazing story too. We now have a startup (https://magtag.me) that received angel investment recently. We feel like outsiders in the tech scene but our spirits are high and when we look back at our lives we feel nothing but pride.
My life is an ongoing achievement. Would you have time to hear me out?
>> Given that an application only has few seconds to impress, conveying anything is a challenge.
Your next paragraph took me a few seconds to read, and impresses a helluva lot. I would not worry about the form of the application process.
>>> but are you in a position to rationally consider that as a 'significant achievement ' ?
That sounds pretty bad. You're basically saying "it was easy to me to have this thought right after reading about it, but for you such thought is probably out of the question". Why would you assume such thing?
The "significant accomplishment" that the interviewer is looking for is one that the interviewee had to work relentlessly towards achieving. Not one that the interviewee "[got] to have" as if it were placed under his christmas tree with a red bow on top.
How does the interviewer know that the interviewee REALLY struggled with the issue? By having the interviewee go into great detail. The author of the article suggested discussing the accomplishment for 15-20 minutes, and he outlined over twenty questions that the interviewer should ask with respect to the accomplishment.
There's a short clip where Elon Musk outlines a similar approach to interviewing applicants:
"...I ask them to tell me about the problems they worked on and how they solved them. If someone was really the person that solved it, they'll be able to answer multiple levels, they'll be able to go down to the brass-tax. And if they weren't, they'll get stuck and you can say 'Oh this person was not really the person who solved it'. Because anyone who really struggled hard with a problem never forgets it."[1]
So, in this sense, I don't believe that the quality of the interviewee's answer hinges on "socio economic status". In fact -in my opinion- the less you have handed to you, the BETTER equipped you are to answer this question in a satisfactory manor.
[1]: http://www.businessinsider.com/elon-musk-job-interview-rule-...
The best part about it is that it cuts so deeply. It gets to whether or not people are capable of getting things done. And it gets down to what a person thinks is important. It's amazing how common it is for devs to not care about the right things, and how easy it is for others not to notice such misaligned priorities.
Hiring people via interviews is hard, because evaluating people with limited time and context is hard. There's no such thing as a question that will tell you everything.
If you're really trying to get an accurate picture of a candidate, conduct an accurate simulation. Get away from interviews as the only means of evaluating someone, and use something that more closely approximates the actual conditions under which they'll be working.
Maybe that's a probationary period where you pay them at contractor rates, or maybe you pay them to make a bugfix to an open-source project that you depend on, or whatever. Asking questions can tell you a lot about how a person thinks and works, but it's never a substitute for the real thing.
I like your idea about contributing to some open source project (paid), though even in that case, you will likely run into qualified people who are unwilling (or legally unable) to do this and as an interviewer, you need to be careful not to exclude those individuals.
I used to think "what a big hassle" about those companies that would make you work a contract basis at first. But after several months of thought, I realized, "hey, if I were to suddenly lose my job, it would dramatically reduce my life-stress if I knew there were a few companies out there that the unemployed could excel at apply for, and that would provide me some income while I was looking for jobs."
I agree that long-term contract-to-hire is a reasonable way to go for may employers and employees, particularly folks that already are already contractors, but the premise of "you've got a couple weeks to prove yourself" is a non-starter. I'm not going to quit my day job and risk my family's financial security for a company that is unwilling to take a similar risk on me. In an ideal world, it is the best way to evaluate a candidate. However, if it were practical, it would have already become an industry practice. The fact that it seems to be rare indicates to me that it does not work very well.
As a Systems Engineer, a practice that I would like to see more of (and may try to implement at the company I work for) is a brief troubleshooting exercise with a shared tmux session. Set up a server with some problem and ask the candidate to identify the issue and try to resolve it.
If you ask this, you're not going to get much actual data. You won't see me write code, you won't see me react to something totally new.
I might ask this during a screening interview, but it's a soft question that should lead into more concrete technical assessments.
So: Meh. No silver bullet.
Then, work-sample. Although in some cases (screening lots of people), a simple work-sample just as a pre-filter might make sense, particularly if you can administer it remotely/auto-grade/etc.
The hiring process has twin goals: not pissing off candidates (hired OR unhired), and hiring the best people you can. Being time-efficient for candidates as well as your own staff helps a lot. Painlessly getting people in or out of the pool before a 6h interview process = win.
Having a repeatable, quantitative process also seems very important, although the cases where I had to hire >3 people with a team of >10 all came with someone else's HR process so I couldn't implement too much of this.
Surely everyone has worked with that person who is confident and charismatic, who inspires confidence in everyone who hasn't (for example) read their code, but who produces negative work every day on the job. When that person eventually leaves (perhaps after spending months or years being shuffled around to the places they can do the least harm by their peers), they will have no problem finding a lucrative job elsewhere, because they are easy and fun to talk to, and can go on for hours about the very important projects they were on.
That wouldn't be a deal breaker for me, but for someone who supposedly has a bit of experience it's a red flag.
I've worked on many great projects that have ended up going nowhere, sometimes because the market simply didn't materialize, sometimes because management pulled the plug just before shipping, sometimes because the whole company went bankrupt because funding dried up because our lead investor was on a plane that some terrorists blew into a building.
These "how to hire the best people!" articles are worse than "Cosmo" giving dating advice to teenage girls.
Your comment sounds like a perfectly fine answer to me. I personally would ask you which project was the most interesting to work on? Or which were you most disappointed by when it didn't ship? And why.
At least, that's what I'm looking for when I ask it.
"merely" doing solid and great programming on "average" and typical applications
I'm not saying that this doesn't work. I've been through my fair share of interviews on both sides of the table, and focusing an interview on various dimensions of a single project is a far, far better strategy than the typical haphazard array of semi-related questions on people skills and technical ability. Really, I think it's not even so important that you focus on the "that you're most proud of" part. This tack would work just as well, I suspect, with the interviewees second or third most notable accomplishment. What's important is that it is a project of sufficient size that you can ask the follow-ups, and in doing so construct a consistent picture of the candidate as an employee.
If you really want one question to get the most information from someone, my goto has always been:
"What is your current favorite language/framework/technology stack? What do you hate about it?"
Ok, so technically that's two (or one two-part question), but I've found that this question will very quickly reveal how able someone is to think critically and to assess all facets of the decisions that they make.
Employer: "How low can I pay you, and how many hours will you work?"
Candidate: "I can't compete with the H1B slaves."
Employer: "Oh, okay. Then let me give you a tricky whiteboard question and reject you based on that, so you don't sue me."
There are people with good potential, who will never be able to answer this question, because they were never given a chance. I was unemployed for 3 years, and I used to work in a completely different industry. The job was much like assembly line, and I couldn't even change my computer's time&date without admin's approval. Opportunities for achievements were extremely limited.
So, maaaybe this is a good question, for the _employer_. But considering the fact employers are complaining about lack of qualified workforce at the same time as LOTS of people are unemployed, I think not! Today, if you ask this question, you're limiting yourself more than you think. It's similar to HR managers who ask for X years of experience in Y. Spending 4 years in your 3rd grade is not an achievement, and I seriously doubt the interviewer would be able to tell the difference.
Everyone likes to churn out a list of triple digit publications, but 99% of those will have no discernible impact on the world.
Interviews go both ways.
These articles are always aimed at the candidate. Where are the articles about "The most important question the interviewer has to be able to answer" ?
e.g. How are (you/the guy who's going to manage me) going to enable me to do great things here ?
What's the most (important project/major change), that was proposed by someone in this kind of position, that has gone into production ?
If I do this job well, where do you expect me to be in 5 years time ?
"Do you want to work here?"
.. followed up by:
"Why do you want to work here?"
If the answers to these two questions don't jive with what you need/want as an employer, then there are no further questions to ask.
The first question is just a staging question. You are asking the candidate to think about a situation that you are proud of, one you will be happy talking about, and one that you hopefully remember in detail.
The real meat is in all of the follow-ups -- what's called "behavioral interviewing". You're asking people to tell you about how they behaved in a real situation. For example, when there was a huge disagreement on the project, how did you solve it? How did you motive the team through a low point? Tell me about big decisions that you made that worked out, and ones that didn't.
Research has shown time and time again that good behavioral interviewing is the best way to find successful employees. If it's a technical position, you clearly also need to discern their technical skills (code on the board, or submit a github resume, or the like), but these are the questions that will tell you if this is someone who will be an asset to the team in non-technical ways.
The question is "What single project or task would you consider your most significant accomplishment in your career to date?"
Big whoop!
I generally like this sort of approach as people will "fold" eventually if pressed hard enough, i.e. they have to admit that they don't know something or they'll try to act like they do. And the goal of an interview is not just reach to point where people "fold" but the path travelled before reaching that point.
The best interview question, IMO, is still the one that Sergey Brin used to ask: "Describe to me something that I already don't know".
I mean, I get it, I use this question regularly as an interviewer, but if you have me on the other side of the table, you're giving me a lot of room to tell you exactly the story you want to hear... Especially if you pull this question out of the bag late in the process, when I you've already given me enough insight in what would impress you.
Like most interview tactics, it's very context sensitive. There is no silver bullet for interviews, and certainly not only one question that matters.
Agree that this post has a terrible title and making a hire/no hire on one question is insane, but it's a good question.
That is why I fundamentally disagree with such a question being important for hiring. If you want to test how someone handles pressure, that can be manifested in other ways. If you want someone to discuss their accomplishments, a simple request for the person to describe some of his/her accomplishments would be less flawed (i.e. "Tell me a little about yourself and what you have done") - it would also be less confrontational.
The fact is this question weeds out people who truly havent' done anything difficult or unable to communicate their achievements properly. If you have an impressive set of skill / strength, then it should be easy enough for you to pick one.
In my mind the question forces the candidate to show the interviewer his / her:
1) Communication skills (i.e, is this person going to tell me all their problems, or just the one major problem that he/she needs help with)
2) What they view as an accomplishment or challenge
3) How they handle challenges
4) Their ambition (i.e, did they take a leadership position in their accomplishment or did they sit back and let everyone lead the project)
If I were asked this type of question, it would be a huge red flag to me, and potentially a deal breaker due to signaling that the interviewer has not put enough thought into his/her questions. I would be able to answer the question just fine, and it would not reflect poorly on me, but to me, it reflects poorly on the interview process since you are highly unlikely to have a wholesome picture of the candidate when more useful approaches to get more knowledge are pretty clear, and is fraught with the same flaws you see in many typical processes today that are not meant to find optimal candidates.
How should I answer? Just pick out a recent one I guess.
Does anyone else have this problem?
Show us the stats. Do your own Project Oxygen if you don't have the stats.
What constitutes a "significant accomplishment" anyway?
* building an awesome web site ? * raising a child ? * graduating college ? * overcoming cancer ? * learning to play the piano?
I do agree you will get to know somebody very quickly if you ask these kinds of question. I am 30 and I will review them in my time.
Hmm. I guess if you don't count the the next #19 questions?