The most useful "question" is any generic task.
It can be as simple as
-- please put this spreadsheet in alphabetical order
-- I have misplaced your resume, could you send me another copy?
-- That sounds like an interesting article, would you send me a copy later on
Even the most trivial task will really illuminate the difference between candidates. Compared to a task, merely talking is entirely useless. Some very polished candidates will simply ignore the request and do nothing. Some will do one of two or three. Some very quiet candidates who are not "impressive" will do all the requests, thoroughly and with obvious care.
How can someone handle the request to forward an article with care? You have to see it to understand. They may remind you of the context, say a word about the article, and add another one for more context for example.
I have worked with some people would would promise things and the immediately forget that they did so. This was extremely irritating. There are also people who never bother with conversation context when sending emails.
Asking for a trivial thing which needs to be done after interview shows the response to both of those questions. Will candidate forget this completely? Will they send an email with no subject or body and just an attachment alone?
This one makes sense to me since it roughly gauges the non-verbal aspects of interpersonal communication. If the interviewee does send a copy, it's a signal that he's dependable in some sense but perhaps more importantly if he goes a bit beyond and sends a few other articles on the topic, it shows passion for the topic and a desire to communicate that knowledge to others.
Really, listening to a request and following up is just one of those key things. If I had them come in and do a longer interview which I had prepared for, I still find it useful!
I wonder what they think of FizzBuzz tests!
To add to this idea. I know someone who used to be very deeply embedded in an organization that has historically skirted the line of being a cult. Apparently one of the key tenets of this group is the importance of integrity, which they partly define as doing what you say you'll do. (From what I can tell, this is important for the organization, because then, they can set up environments where they can get you to agree to things then know that you'll follow through even if you come to your senses later.) What my friend noticed, is that people who moved on from this organization tended to be pretty successful in their careers, and they commonly attribute the ease of their success being that they simply do what they say they're going to do, and not in a woo-woo self-actualization sort of way. They simply do the types of simple tasks that others don't bother with, or forget about, like forward an article to someone who they said they would.
I agree with your observation.
I will add that one simple reason people fail to do the task is because they try to remember it in their heads and then procrastinate. That is a deadly combo.
A simple to-do list can solve this issue.
However, a good manager can be the to do list. You can use an app to assign tasks. That is partly why Trello boards and the like work so well. It simply gets people to focus on things and not just have a bunch of tasks in memory.
I must also add that a Trello board can force the manager to put down what they expect to get done in writing.
It's an interview - who on earth is going to fail this particular item? Do you mean during the interview? Or after, like the 'interesting article'?
I have worked with people who would sincerely reply that they already submitted the resume, and others that might take it as a personal insult that their resume was 'lost'.
This filters out those types of people, as well as those that aren't very interested in getting hired for the position.
I think their assumption may be that someone printed it out for me, but didn't forward the email. Or that it was accidentally deleted.
I don't pretend I've never seen the resume, and I do ask questions about the resume.
I struggle with candidate responses because it's quite possible to get bad answers from great candidates due to nervousness, monetary confusion, accidental use of company-specific jargon that can be misrepresented etc. And interviews are abnormal, so you're selecting for people who do well in interviews hoping it's correlated with the skill you actually need. Yet I don't know of a good alternative to almost anything beyond a truly unskilled job.
But these examples are great because they normalize out most of the realtime "high stakes" stuff. Sure, they can stress about precisely the right phrasing to put above whatever attachment they might be sending, but that's still an easier problem.
My favorite example of "interview perspective imbalance" comes from the Allied occupation of Japan: when the first allied troops arrived (after the Japanese had signaled surrender) they were lead to a brief reception. Apparently the Japanese representatives had worried that the orange juice they set out was at the right temperature while the American troopers were worried that they might be assassinated. Quite an imbalance of expectation!
When hiring for an intern/junior position, I will often ask for a sample of a candidate's school projects. My rationale for this request is to see an example of their code. In practice, some candidates never follow up on my request; some send me a big hairy zip file with no context; and some obviously take the time to clean up their code and add comments before sending it to me. I've even had one or two create a GitHub account and push their code to a public repo to share with me, after we've gotten to talk about their knowledge of git.
Thank you for the insight that while trying to gauge technical skill I ended up evaluating thoroughness and care ;)
In terms of an actual question, the only useful ones are ones that come from this chat, because it's where we have a back-and-forth conversation about tech decisions, what issues they had, what they had to overcome, and what decisions that particular dev made. My goal is to get a reaction, negative or positive, and dig into why that situation happened, what they did to overcome it, and what they'd do with hindsight.
Not only does it make an interview more interesting for all involved, it tells you more about the candidate than a quiz would. You learn what they do under pressure, what pisses them off, and what experience they have for the kind of work you do, because you've been using your experience against theirs in conversation. Sometimes, to make it fair, I tell them (without dropping client names) issues I've had recently, and I see if any of the solutions were raised by the team, or are ultimately the solution we went for.
I've had enough success with this approach that (when I did interviews) this was my approach:
1. A really basic take home test, probably in a language they don't use, just to see if they can write any code. Literally something that takes an hour to do. A favourite of mine was to get them to write an application in a given language to hit an endpoint, with a given code and their email address. When done correctly, an email is sent to our HR team, who then set up an interview. Based on our logs, you'd be surprised at how many candidates couldn't do this - by which I mean had clearly worked on the test, submitted something, and had failed to send a PUT request with the correct data.
2. A relaxed interview, like above, in neutral ground like a coffee shop. I use the walk from office to coffee shop to handle company questios, and the standard interview stuff,
Even if you specifically told me that I must actually hit your server, I'd still test on my own server, and then possibly forget to change the endpoint before submitting (no way to unit test that). Shit happens, and you lose someone on a technicality.
Also, what if your server stops working? Now you're wasting people's time.
Don't rely on magic; it eventually bites you in the ass.
If you can't connect to the server, or it returns a fatal HTTP error, a reasonable candidate would reach out via other means (assuming they aren't deliberately hidden or unavailable).
We were also keen to stress that this should take far less than an hour of their time. We didn't want to see dependency injection, metaprogramming, or anything overboard that showcased a developers knowledge - this is purely just to see if they can deliver code that matches a requirement in the time provided.
In the case where we had downtime (once in 18 months), I just forwarded their email to HR and we spent some extra time going over code in the sit-down interview.
Shows understanding, shows willingness to reach out and solve problems, shows ability to delegate responsibility.
> I'd have assumed...
I would expect a good candidate/colleague to ask instead of assuming things.
> And then forget to... > Shit happens...
If someone can't manage to complete a simple assignment without forgetting half the steps, I don't want them pushing code to production.
And a service with one http endpoint having reasonable uptime seems very far from magic.
Literally, the only point of the exercise is to get that email address. It says that much in the description of the task, alongside raising a GitHub issue if the server is down. It was hosted on Azure, and had our uptime tool checking it anyway. If it was down, there'd be a badge on the repo homepage saying it was down. It happened once, and I invited the person to the interview regardless regardless.
I say this much when we send over the technical test. It's like the weigh-in for a boxing match. If they want to showboat, it's on them, but all I care about is whether they can follow a simple instruction with minimal code in the time we said it'd take. I couldn't give a shit what fancy tricks you do in your code, because it's code you've written without time constraints in your own time. The whole reason for chatting about past experience is to see what you do in a REAL development situation.
Perhaps that is relaxing to you and most people but not everyone. Coffee shops can be very distracting because you hear other people talking and walking by and some even play music. Perhaps you want to screen out such candidates anyway so it doesn't matter much. But you are also screening out a lot of skilled developers who don't function well in distracting environments.
Sorry to be harsh but if you are unable to have a conversation in a coffee shop its unlikely you will make a good developer, regardless of how well you can code.
You don't have to be a loud-mouthed extrovert or "heart-and-soul-of-the-party" type but social skills are necessary for most dev jobs.
Get them talking about what things they get emotionally invested in. You learn a lot about someone this way.
The trick to good interviewing is putting the candidate at ease. When someone's future is on the line, they behave very differently, and preform worse than they would in a regular work environment.
Asking questions like "what have you done well" and "what mistakes have you made" will favor narcissists, because they will embellish, whereas those more on the neurotic side will take more blame, and dislike taking credit.
So put them at ease. You'll get far more real answers that way.
Yep. I ask something similar and explicitly tell the candidate I'm NOT necessarily after "tell me a time you delivered strong business impact", "tell me a time you added a new skill to your resume", or the other stereotypical things that make a good "interview story".
I've been asked this question in interviews on more than one occasion. I find it gives me a good opportunity to show off some knowledge, and with a good interviewer, it's a very good conversation starter for something that can end up being quite deep diving.
"Tell me more about what you did." Can be asked off almost any line of conversation. I'm seeking two outcomes:
1. Candidate's ability to quickly articulate a complex process, organization, problem, etc. to someone who has absolutely no understanding of it (i.e., me).
2. An understanding of their specific role and impact, as well as how the team worked together. You often have to coach people through the latter, as many candidates will default to the royal-we.
The other one I'm playing with is, "Let's say you start working here, but six months later, you leave for another opportunity. Why would you leave?"
I'm still figuring out the delivery on that one, but you get an interesting insight into what matters to the candidate.
The only time that I have recommended someone be hired who was not able to answer this question was when they were under an NDA and were not able to talk about the work they had done. That candidate aced the practical side of the interview that I gave ("Here's a scenario at a high level using the skills you claim to have, what would you do and why?") and is the best worker I have ever recommended to hire.
Why would I leave a firm in six months? Most likely because my boss is an asshole. Am I daft enough to get into that with an interviewer, who may be that boss? No.
The overhead of starting again the job search from scratch, explaining why I want to quit after 6 months (for real this time) is much bigger.
Likewise, just because they would interpret that answer as a black mark, doesn't make him an ass. Maybe the reason I run into asses all the time, is because I am one. Even if they don't interpret it as a black mark, it's a pretty daft way to start a professional relationship.
There's too many unknowns, so the correct course of action is to make up some non-confrontational lie.
That would be such a huge red flag for a interviewee. Does this team have such severe issues with frequent turnover that the interviewer feels the need to ask such a question? Is there some sort of dysfunction in the organization that the interviewer isn't telling me about?
Not to mention, as another poster pointed out, there's no honest answer to that question that sounds good.
A corollary to your first outcome is an understanding the candidate's situational awareness. How well the candidate grasps a complex process, organization, problem, etc. Even if they struggle to articulate the complexity well, it's possible to probe their response a bit to grasp if it's an articulation issue or an awareness issue.
Note that there are no right or wrong answers for how they respond, only aligning the the person with the role. If the role is subject to a lot of volatility, complexity, or outside influence, a high degree of situational awareness is critical. For roles that are more insulated, it's perfectly fine if they don't pick up on that stuff.
Same for other aspects of the role. If the workload is fairly monotonous or is a backfill roll with relatively well defined and stable needs, identifying someone that is content and has a preference for that is critical. There's a whole range from repetitive, mind-numbing roles to roles that are dynamic in the day-to-day sense but narrowly scoped around a specific need/focus to roles that are currently ill-defined but will stabilize in the near term to roles that's will be a never ending stream of entropy thrown at you but scoped to a specific dimension to roles that will be a never ending stream of entropy from multiple dimensions.
The bulk of my interviews tend to be exploratory conversations that jump off at this question. Matching the candidate to both the work environment in general and the role in particular, it pre-empts a lot of potential points of friction that would lead to early attrition. And for candidates that get hired, gives me a solid foundation to start with in understanding how to support and manage that individual.
That said, I don't recommend that interviewing style to everyone. If your own situational awareness isn't well developed, it's hard to gauge someone else's from this line of questioning. It can also lead to misreading a candidate's response, as well as making it hard to organically evolve the conversation to dive deeper. Which may or may not be needed for the role you're hiring for, but is something to be aware of.
A rule I have is that everyone is good in something and bad in something else. So, if by the end of the interview I haven't found a topic where the candidate excels and a topic where sucks, the interview hasn't been effective.
"Tell me about a result or achievement you're very proud of. How did it come to be a goal for you, and what did you do to accomplish it?"
(Crucial followup after they answer) "Now tell me about a mistake you made along the way and how you overcame it."
Talking about real work provides insight into motivations and enthusiasms, and gives you permission to dive into details as they talk. It also helps show whether they have a healthy attitude about mistakes. This question can also provide some insight into team dynamics since very few big things are accomplished solo these days.
From the applicant side:
"Let's say I get hired and I do an amazing job. You're going to write my 6-month or annual review--what would you write?"
Like Amazon's trick of writing the press release first, this helps hiring managers get specific about their expectations for the hire. It's amazing how often companies hire folks with only a vague sense of what success should look like.
I find this tells me a lot about the person who's interviewing me and the current climate they're operating in. Asking follow-up questions helps a lot here too. It's a good way to find out if the interviewer (frequently the hiring manager) is struggling with an unmotivated team, overly-onerous bureaucracy, lack of funding, staff turnover, outdated software, etc.
For each position in the candidate's resume (or the last 3 if the resume is too long) I ask "what did you like about the job? what did you dislike? What made you happy to go to work and what made you drag your feet? It can be anything: boss, coworkers, how work was organized, office politics or lack thereof, visibility/interest of the job, commute time, offices... There are no wrong answers". Then I let them talk.
When it's done right, I find this is a great way to understand what motivates them and to make sure their intrinsic motivation is aligned with the position they're applying for and the company context. For instance, for a developer I'll be interested in someone who likes fixing problems; for a project manager, being result-oriented is usually a good thing. Someone who likes having a clear set of steps to follow is probably not a good match for my management style. Another thing that I can check is how they coped with a situation they disliked. Did they try to address it, work around it, or did they give up/avoid it?
I was first exposed to this approach as a candidate. I remember spending 90 minutes talking about all my previous positions, reflecting about what I'd liked and disliked for each of them, and coming out of there knowing more about myself than I did going in.
NOT a test as in "find the 10 hidden mistakes", but an invitation to talk about ways to improve the code. I make it clear to the candidate that it's not about the syntax, or any algorithm, but about quality.
It's really good to see what kinds of things they care about (e.g. do they look into low-level performance, or more into things like variable naming and method length), how deep their knowledge of the language is (e.g. do they recognize non-idiomatic code, and suggest more modern approaches), if they are more of a high-level thinker or more detail-oriented etc. And of course, it nicely weeds out the (surprisingly high) number of people who cannot really program, without relying on any memorization of keywords. Plus, it's pretty realistic - nobody ever writes code on a whiteboard, but doing code reviews of code somebody else wrote is something that happens all the time.
Most candidates find that task quite fun - and with really good candidates I quite frequently learn something new myself.
I was planning to do a blog post about why and how I do that review - let me know if that would be of interest to you.
I can imagine that another good thing about it is that it takes enough focus and is a familiar enough task that even if I’m in nervous-applicant mode, it will kick me over into acting like I’m already working there.
"What is something you believe about product management that differs from conventional wisdom (or what the majority of other product managers believe?)"
I thought it was a good question, because it gives the chance for the candidate to talk about how they think, and it gives them a license to speak a little more honestly than they ordinarily would say in a normal interview. It shows you how much thought the person has given to their profession and how they approach their job.
If you were asking this during hiring, I think it should be done with care on your end, because I could see it easily being misinterpreted by the wrong team/interviewer.
For example, if someone came in and said something like "Compared to others, I think A/B testing is a waste of time", it would behoove you to dive in a little bit more to understand why they think that. It shouldn't be disqualifying if you're a team that does a lot of A/B testing.
1. I tell them the story about a massive screw up in my career (its funny) about how I thought I was getting fired about being frank about it... etc. I then ask "do you have a story about a major screw up in your career. The answer should say a lot. Dealing with crisis, dealing with mistakes, can you laugh about traumatic things (a great coping mechanism as a programer). I'm looking for insight into you in a professional setting. There have been occasions that this one question takes up the BULK of my allotted time and has on occasion gone over.
2. Can you read and talk about a piece of code GIVEN to you. The bulk of the battle for most devs is READING existing code and trying to figure out where to fit fixes in, or how to integrate what they are doing into a larger, standing system. The provided sample should have errors, room for improvement and plenty of ambiguity where the reader has to make assumptions or ask questions.
Asking "Do you have experience with technology X" is useless - everybody knows that "yes" is the expected answer.
Asking specific technical questions is random - they might or might not have come across that particular problem. Also, in practice the answer would simply be looked up, and you don't want to hire for memorization.
Instead, ask things like"How did you find working with <technology X>? Can you compare it to <common alternative>, or <similar tech also listed on their resume>?". Or maybe "what worked really well with <technology X>? What pitfalls did you fall into? Anything for which you would, or would not, recommed it?"
This kind of info is hard to fake, gives you good insight into their thought process and priorities, or just gets people talking about their projects more easily than a generic "tell me about your work on project y".
Expected answer contains: Overall process, roles distribution (who does what), project management style, issues triage, etc...
My follow-up is usually something like, "Explain that job title/position to me. What does that person do? What skills do they have?"
I'm curious about hearing from you about the things that excite you in some way. It helps set the tone for the rest of our conversation, and helps me to gauge your level of engagement with the questions and answers after that.
In a 100% honest world, you wouldn't even need to interview for technical positions; you could look at the person's resume and tell instantly whether or not they are capable of doing the job. But people lie a lot and over-embellish. So best questions to ask are about the items listed on their resume, because you find out who's telling the truth and who's not.
I even had one interview where that question basically lead to the interviewer explaining just how bad it was, and how unlikely it was that I actually wanted the job (and I had no reason to believe that was a trick or bluff). I certainly left feeling like I dodged a bullet.
Behind the question-- I'm probing for curiosity and intellection. Also, what's important and top of mind to them.
It usually reveals character on how a person deals with tough situations.