How We Designed Our Interview Process
blog.readme.io
blog.readme.io
This should always freaking be the case. You, the interviewer, are there to decide if you want to work with this person in the future. You're not going to learn that by making an adversary. You're not going to learn that by making it super easy either. If you're worth your salt (as an interviewer, and you should be given that you're now part of the hiring process), you should be able to go through an open ended problem (it's what I do). This open ended problem should have increasingly difficult sub-tasks and of course be related to what you intend the person your interviewing to do. One of my worst experiences interviewing was with a company that clearly didn't give a rats rear end (at one location). Interviewers were constantly late, didn't know what I'd applied for, or even at times my name. Same company, different location...totally different story. Your most valuable resource as a company is your people, treat them as such from start to finish. Needless to say, I chose where I'm at because of the people. I'm glad to see others sharing this attitude as well.
This looks great from what's printed on the page.
The question that jumps out at me from reading the interview prep screenshot is: How often does a candidate fail the "work on your own project" part simply because the people watching them have no interest in the problem space or worse, no idea what's going on? I just don't see how that type of interview could be scored equitably across a range of candidates.
Is that considered a failure on culture grounds? I'm highly suspect of basically all culture related hiring standards.
I've been in almost all positions of a hiring team: phone screen, white boarding, Q&A, etc.; and in the end I want to hear what this person can bring to the company and see if they're a good fit or vice versa.
I feel like so many places put emphasis on abstract knowledge and technical ability and leave out so many other traits that are good. Also, in Techlandia, I feel there is too much of a male machismo that permeates through and there are many people that treat the interview as a chance to demonstrate ability to the interviewee and/or trick them - to the point of intimidation.
Take a SWE job for example, what is the most effective, efficient, and fair approach to evaluate a candidate?
Can anyone name an approach that is, in general, better than the main-stream technical interview. Factors to consider: 1. Technical competence 2. Culture fit 3. Interviewer/interviewee/scheduling etc time cost 4. Logistic cost
I don't disagree, but personally I would rather work with a competent (or brilliant) asshole than an incompetent nice person.
I (freelancer) once had a software-making client that employed me for a month do contribute a special component to their big project. When I arrived the big boss himself received me and showed me around - and when it came to showing me where and with whom I actually worked he apologized profusely before he even started.
Apparently I had to work with the one (and only) guy who sat alone in his office. But he was the guy who had written the specs that I had to write the code for, and he had room since all offices were designed for at least 2 people. The apologies were because that was the most difficult person in the entire company to work with, by a margin, but he was too good so they just kept him in his cell...
Well, I wasn't daunted at all actually. Turned out that yes, he was difficult socially. For example, there was a woman who came in to ask him something (it was rare anyone talked to him while I was there), and in an audibly annoyed tone he pointed out that he had already sent her that exact information 5 days earlier in document XYZ on page N, headline "3.2 foobar" (yes, to that detail, without looking anything up).
He also was really, rally good. The specs he had written, several hundred pages in three documents, were near perfect. I didn't have to ask a single question at any time, he had thought of every detail.
I myself also had no trouble talking to him, I had fun with the other people that I visited in their offices (who repeatedly told me pitied me for having to share an office with that guy), but also with my room mate. Because of the surrounding highly unusual circumstances this still is one of my most memorable (and completely positive) freelancer jobs.
It sounds like that guy had a high IQ and low EQ, which is highly efficient for work scenarios, but it usually sucks for any kind of social interaction.
I also like the do-a-real-thing step. We do something similar at Pivotal. Once you pass the basic tech screen, we invite you to work with us for a day.
We pair program, so this is easy to do: take the candidate, drop them (usually) into a real project with a real engineer working on a real problem. Then they work together on the real problem. That's it.
Assuming nothing goes drastically wrong: One project in the morning, then lunch, then a different project in the afternoon.
If we agree that you know what you're doing and that we want to work with you, you're hired. Every engineer at Pivotal has passed the test of "do you want to work with this person?"
In the quest to work out "can this person do the job?", I personally continue to be amazed at the gymnastics companies will engage in to avoid just ... trying folk on the job.
We had a period where we had many candidates and not enough recruiters (hiring good recruiters is actually really hard). Some of the recruiters would get stressed out and quit. Some would, as you say, simply drop candidates on the floor without telling them. We also had a lot of trouble with our previous candidate management software. We switched to Greenhouse 2 months ago, it seems (touch wood) to be doing much better.
If you like, you can contact me by email and I'll follow up for you. This offer extends to anyone else.
In dividing the day, we always try to pick one project where hopefully they will feel somewhat at home. Good at Ruby? Here's a Rails app. Java person? I hope you like Spring Boot!
Then for the other project it's usually something unfamiliar. Our goal is not to trip you up. It's not to play gotcha.
It's that dealing with unfamiliar languages, toolchains, ecosystems, coding styles, testing frameworks etc is a big part of life for pivots, especially Labs pivots.
This might sound like we only want generalists. This isn't quite true. We respect deep expertise -- deep knowledge of some stack or subject. But we also like comfort with the unknown. These are distinctive things. One is about knowledge held, the other about holding on in the face of the unknown.
You do however bring up the problem of projects with high rampups. The answer is that these are the hardest to schedule and assess. In these cases we focus on picking stories out of the backlog which are as self-contained and low-context as possible.
In very (very) rare cases, it will be impossible to find such a story or project able to take candidates. In those cases we have a collection of "canned" projects. Since it's not a real thing, we only use these as a desperate, no-alternatives, final, last resort.
The idea of letting someone actually work on a problem is far better than generic algorithm or data structure questions. Two things though...
> As soon as we schedule the interview, applicants receive a link taking them to this welcome page.
Love the page but the content is all static. This should be in an email. If it's in an email, especially in invite form, I have access to the content at all times on my phone and can even accept an invite. Putting it in the browser means I have to find the email and open the browser (alternatively bookmark or keep the site open). But the content is static so it should be in an email.
In my opinion anyway.
> After that is the main part of the technical interview: we ask interviewees to work on their own project, using whatever resources they deem necessary (and yes, that includes Google). We want resourceful developers who can problem solve, not people with an encyclopedic memory. We use this as an opportunity to learn what our candidates are passionate about. This helps us understand whether ReadMe is a place where they can pursue their long-term goals.
Hmm. So many projects I want to work on, especially for an hour or two session, may not necessarily reflect my long term goals but be more oriented towards completing something that either myself or I perceive my interviewers being interested in. Yeah I get the whole "work on what you want" thing but in such a short timeframe to show off what I can do is absolutely not indicative of my long term goals.
I see a lot of these interview discussions talking about having candidates bang out real features in a couple of hours and I just don't understand how that's possible.
Also, not to pick, but the owl character that might seem like a good idea to younger candidates, that might respect it for the good design and soft psychological impact, might scare off the mature employees that are there to get the job done and are there to engineer solutions.
Despite these things, I think it looks professional, and the fact is that if you're a startup, you're most likely looking for a younger candidate, because they are cheaper, can be more motivated, and are definitely easier to mold. And yes, if you discriminate, that is illegal in the U.S., at least.
However, at some point, you need to hire more experienced candidates, and this process will need to change to reflect that. I'm a more mature person myself, and would not seek to fill a development team with juniors. I've seen what that can do. It's fine to have interns and juniors, but that shouldn't be the primary talent pool, or you'll end up with serious design problems and lots of poor code. But that can happen with more experienced candidates also- that is why you want good, more experienced candidates to create an environment that fosters growth of incredibly smart juniors.
It looks fine as-is to me, and within my domain of expertise I'm about as senior a person as is temporally possible to be (I'm pretty sure I know everyone who's more senior than I am).
But maybe it's just seeing how this goes out of its way to fix the problems in hiring processes that I've been seeing, and ranting about, for quite a while now.
I'm just speaking as a representative of the mid-40's crowd who laments the highlighted startups focusing on sheen over substance, because the sheen gets the VC's money and the juniors hired, when what is needed is a solid business plan and good platform/solution.
> But maybe it's just seeing how this goes out of its way to fix the problems in hiring processes that I've been seeing, and ranting about, for quite a while now.
If your rant is that companies should clearly reflect who they are and who they are looking for in their advertisement, if they can do so without sabotaging their chances at hiring better people that can help fix things, then yes, I agree.
If you think that every company needs to introduce handholding and cartoon characters into its HR/recruiting process, then I'd only agree with that depending on the type of person you are trying to recruit, and would caution that might not be appropriate for someone my age.
Yet somehow I'm consistently rated very highly by my peers, to the point where I have significant influence on the technical direction of projects I work on.
The reason I don't have hobby projects or dev social media accounts is this: by the time I get home, I have a spouse and two children who I have a life with outside of programming. I frankly don't have time to work on hobby programming outside of work to the level of sophistication that rises to the sort I'd feel comfortable showing in an interview. And picking some random toy project that does would quickly reveal how little I care about it.
Edit for clarity: that said, I think this level of transparency is fantastic. I have nothing negative to say about it.
The original question asked whether this was slanted towards younger developers. Although I don't interview people this way, I would expect that good senior developers could pull an interesting kata out of their back pocket as long as they had enough warning to set up their development environment ahead of time. Hell, I'm quite sure I could fill up a couple of hours with FizzBuzz variations and still come away giving a good impression.
Basically, when interviewing for junior developers, I want to see if they can code and if they can take direction. So it's cool to write any stupid code and then help them explore it. I'm trying to find out if they can listen to criticism and if they can utilise that criticism to improve their code.
For senior developers I want a lot more. I want some evidence that they have spent their time thinking about software. I want to see their opinions, but more than that, I want them to be able to show why their opinions are worth listening to. When a senior dev comes on my team, they are not there just to write code. They are there to inspire others to greater heights.
If you quickly revealed to me how little you care about some random toy project, I would try to help you see the point. It's not about the toy project. That project is only a medium for communication. If you still couldn't see the point, then I would probably conclude that you were not right for our team. That might be my loss, but as long as my filter selects enough good people it's not really a big problem for me.
I am led to believe, by the people on this site and others, that unless you are AmaGoogBookSoft or a Valley darling good developers are so hard to come by that they can't afford to miss good ones.
There are good programmers everywhere. Grabbing a couple (ideally including a senior dev) of really excellent people will go a long way improving your team. But you need to do more than advertise and interview to get these people. You need to go out into your community, participate, and create a reputation. You need to work on your internal processes and be interesting enough for an excellent dev to be interested in you. My advice for a small up and coming team is to spend the cash to hire a well known senior consultant with a good track record. From there you can get your culture set up and you can make contacts with other good devs that the consultant knows. After that it is really just hard work to hone everything that you do so that good people think that it will be a worthwhile career move to spend time with you.
If your management strategy is to throw money at talent and hope that it gels, I think you will have a lot of trouble. Once you have talent, it's important that they are set up to succeed. This is more than just turning them loose on a project. IMHO, good teams have many dimensions to them. You don't necessarily need (or want) to have a superstar in every position. It can be good, but it requires a corresponding level of proficiency from management to get them all working together well.
If you are worried about the one that got away in your interview process, then I think it is a sign that you have very much bigger problems that you need to deal with. If you tell me that this is common in the Valley, it will not be something that surprises me ;-)
I wish more companies were more honest about things like this. Of course, I've been on the interviewer side plenty, and have many times wished for more honesty in a candidate.
If people don't have stuff they can show, we provide them a few small things they can do. The sort of problems which really should only take 20-30 minutes. We really are just looking at "can this person write code at all", so it doesn't take much.
I think someone having a personal project is a strong indicator that they're passionate about the trade though which is a great thing.
I know lots of people have a laptop already, but I'm sure I'm not the only person who doesn't (I just can't work on one; I get frustrated). Or is that part of the selection process; they don't want the kind of person who doesn't own a laptop, or doesn't bring it with them to interviews.
Provide a desktop for the candidate to use. Make sure it's got vim, emacs, gedit, and whatever the cool kids use now (sublime?). Make sure it's got a recent set of tools for the technology at hand. GCC and build tools or the equivalent standard. Job done.
Obviously making it a mandatory function of the interview isn't a good idea, but you can learn a lot from someone by seeing how they've setup their tooling.
This is how I used my laptop for years (I still use big monitors, but I don't usually use an external keyboard now, although I'm thinking getting another cherry mx blue keyboard). When I'm working, I use the external monitors as the primary and the laptop screen for slack/documentation (since its not at nice eye-height). It works very well.
I even know people who leave their laptops closed when they work, using purely external monitor and external keyboard/mouse.
A laptop is just a portable desktop.
But for programming, I'm much more comfortable in my environment using my tools and configuration (for example, I'm a colemak typist, vim is my preferred editor, etc). I mean, I think a machine should be provided in case the candidate doesn't bring one (or doesn't want to; or like you, prefers to use desktops), but allowing the candidate the option of using their own helps set them up for success as they can use their own familiar environment, while forcing people to use something unfamiliar may be setting them up for failure.
We tell candidates that we'll provide a laptop for them if they don't have one but also tell them to feel free to bring in their own.
Also to turn your statement around a bit, perhaps one could claim that your need to have a big monitor (or two) is your own security blanket, and that you should be able to work without one. In my office some people have oodles of monitors. Some people just use their laptops. Some are a bit of both. Everyone has their preferences.
The day a job interviewer tells me to bring in my desktop and monitor selection with me, I'll be the first to complain.
It was "just a personal note". My point is that telling people to bring their own laptop in for a job interview is odd enough to object to.
They require[1]: 'Your GitHub is full of everything from embarrassing hacks to impressive Open Source contributions'
I understand some reasons behind seeing Github profile of candidate but requiring it is too much. But this is still better then company i saw the other day which was requiring Github project with X000 stargazers.
I think we have to choose between bad and worse.
[1] https://readme.io/careers/#job-full-stack-nodejs-developer
Though I agree that a lot of skilled people don't have the will/feel the need to build a public profile of their work.
They're not.
They definitely are - it's part of human psychology, and it happens to those of us even aware of the fact.
We start making judgements quickly. It's the way we work.
It takes a lot of self-awareness, and usually methodology to overcome it.
I seriously think there's an underlying pissing contest that has permeated the software development culture. If other engineering disciplines can be hired by talking intelligently and answering questions related to their field, why can't software engineers?
Other engineers have professional registration and thus some form of regulation. In some countries you're not allowed to cal yourself an engineer unless you have some particular qualification and the registration.
For me, job interviews were always incredibly nerve wracking. It feels like you're being judged, but of course you are being judged- the whole point of the interview is to judge your suitability for a position. It's also nerve wracking because I was always interviewing for something I wanted really badly, that was a step up from whatever I was doing at the time. So, I was always interviewing from a position of weakness, and it made me feel even more self-conscious.
At this point in my career I've probably been the interviewer >= 20x as often as I've been the interviewee. Some were phone screens, the rest were in person, and about half the time I was one of two interviewers (some companies do one-on-one interviews, some do two-on-one, and supposedly some do many-on-one interviews but I've never experienced that (but it sounds terrible)).
In all of that, I don't think I've ever been malicious or tried to show off or make someone feel bad, and I don't think I've ever witnessed or even heard of a colleague doing it either. Maybe I'm just lucky, and got all my jobs at companies that had a less horrible interview process, but I wonder.
Job interviews at companies I really want to work for have been intimidating because it is intimidating. I wanted to move up in the world, take the next step, and so it always felt like the people interviewing me were implicitly people I wanted to emulate. That's so awkward- it feels like approaching pseudorandom strangers and asking, "I want to be like you, am I cool enough for you to give me a job here so I can be like you?" You're being measured against a yardstick, and it feels like the yardstick is the person interviewing you, because they've already got a job at the company you want to work at.
So I would have to fight against being intimidated by the people who were interviewing me, but what I didn't realize was that although I felt intimidated, they (probably) weren't being intimidating. I was just projecting what I was feeling onto them, they were just being their normal selves and sucking at interviewing because everybody sucks at interviewing.
There's no reason for anyone to be a jerk to an interviewee, the interviewers want to have successful interviews just as much as the candidate. Nobody wants to drive away a potentially good hire just to act superior for a few minutes in front of a job seeker that they will never see again for the rest of their life.
So I would like to think that all of the horror stories about terrible interviewers come from interviewees projecting their (totally natural) stress onto other people, but who knows. People can be irrational jerks too unfortunately. :( I'm also curious if the bad interviewer behavior was exclusively in one-on-one interviews. Those seem more likely to go off the rails, since there's no other interviewer to counteract someone who is an especially bad interviewer to begin with.
I can hardly speak to this in general, but Google always schedules one of your interviews with two interviewers, one of which is supposed to be training the other to do a good interview.
I felt that those interviews were much nastier than the one-on-ones (2 out of 2 times).
I'm convinced that the companies think that this is a good idea, and I'm convinced that this is wrong.
It's easy to push something. It's a lot harder to build something good. Our goal is to get away from static docs, and show each developer exactly what they need.
Their stuff does look pretty though.
In the end the price point put me off enough for me to go in another direction. Yes, it would save time and that time would absolutely be worth $59 or $199 here and now. Maybe even that much every month for three months. At some point though I'm going to run the books and see that I've paid $2k for something I could have built myself in a day. Hopefully when that happens the company is at a point where that is a complete no-brainer "I'm glad I made that call", but there's no guarantee of that. Most companies fail and all that jazz.
We're bootstrapped so the equation may be different if we raised money. I assume that's a significant part of their customer base.
While ReadMe is built for hosting code/API documentation, we've had no problems using it for just about everything else. And since the UI is great everyone from my operations lead to my executive assistant are able to jump in right away and contribute towards living documentation.
Down the road, I plan to host our first customer support documentation on ReadMe. The both techies and non-techies can contribute towards keeping it up to date and a lot of the features that devs want with API documentation translate naturally to product support docs, such as the QA / Knowledge Base section.