How I Interview
rkoutnik.com
rkoutnik.com
I have 25 years of software development under my belt, I published a book, spoke at various conferences in Europe, have a Github profile (admittedly not very lively), worked for well-established enterprises and have great references. Why shall I still prove myself?
If I was a senior lawyer or an architect joining a new studio, I strongly doubt I would have to go through an interview to test if I can use the drawing table or if I know the body of laws of my countries. It is just ridiculous.
As I work as a freelance, my approach will be to offer 2 weeks of work on site for free. After the 2nd week, if they still think I'm good enough for the company, I will charge the 2 weeks, otherwise goodbye and no questions asked.
Those 2 weeks should determine whether it works out on other levels (e.g. are both parties comfortable with working each other) and if you do work in that time, you better charge for it.
So, to answer your question, I will apply this approach from 2016.
Because we see candidates claiming that level of experience that cannot construct a simple for loop.
Unless they are references from people who those involved with hiring personally know.
These folks with many "senior" level positions on their resume can't manage a task our last hire did in about 6 minutes.
It is usually a pretty organic process. If we get into it and question their coding ability, we might ask a few basic questions, like how do you declare an array or vector in C++. Do you use std::algorithm? Then possibly into C++14 territory. It depends on the position. On the other hand, if they breeze through some basic coding questions, it quickly turns into a discussion and feeling the candidate out in terms of fit. But if we get a senior SE come in, and we talk about previous projects and we get "I fixed bugs, reduced memory usage, and saved $X million dollars" that is a big red flag.
Sounds like your expectations are part of the problem.
Why do you expect someone 15 years out of university, working in the now enormously wide software space to have your pet questions memorized? C++ is well known for being too big and needing to be subset(-ed). And, why not realize that such a person could become an expert in those questions in less than an hour of googling and reading?
The bigger problem is that companies are not willing to invest even four hours of training into employees!
We typically need to hire quickly, and having to wait 2 weeks - and coach someone on all the different components, create accounts for them, etc. - is not "free" for us.
If you're that good you should blaze through the technical questions in half an hour, and give us all confidence we're not wasting each others' time.
Then again you're a freelancer to so it's a bit different to being hired as a contractor/permie. But either way, we want the hiring gap filled, not as something we have to revisit after 2 weeks before we decide whether we have to go back out to market.
Put it this way. You come in for an interview, refuse the tech questions and we're left thinking "this guy looks good on paper, so why doesn't he want to explain what a design pattern is?". Straight after you a guy comes in who answers the technical questions OK. Who do you think we'd hire?
It doesn't make sense anyway. In 25 years, the sheer amount of cowboys I have seen literally bringing down projects out of incompetence and lack of humility is staggering. And these guys were hired out of technical riddles.
As a Good Person, you should get taken on by clients who don't have this requirement while others who do lose you.
I've only done one of those tests in the last couple of years because I wanted the (not insignificant) pay rise and it gave me an opportunity to learn something. It's all about negotiating power (they had more).
But yeah, in general I don't have to bother with them. OTOH I don't see them as entirely negative because hopefully it should mean you'll be working with competent people (provided the tasks are small demonstrations and not full applications which smacks of incompetence).
I think that two weeks are a good time-frame to evaluate a mutual "love" and would also help better understand the soft-skill and how well the candidate fits in the company.
As a consultant I have tried a variation on your approach. I've offered to work for a client for 2-3 weeks. At the end of which THEY get to decide how much I'm worth. I reserve the right not to carry on working for them but for those 2-3 weeks I will accept whatever they pay including nothing.
In over a decade not one client has accepted this deal.
It's always the clients who claim to be 'innovative', 'game changing', 'disruptive' etc. that are the most risk averse. They seem completely blind to the cognitive dissonance.
Now, the GP says he will work first two weeks without pay? Simple to understand and accept.
1. People can't just take two weeks off to go code for another company.
2. With so many other candidates out there, a company is typically not going to bother with an "eccentric" who refuses to interview. (not saying you are eccentric, just that's how you'll appear next to the other 100 traditional "interview me I need a job!" applicants)
I've seen #2 play out in real life. Company interviewed a guy who refused to do any code on a whiteboard, even simple fizzbuzzy stuff, pointing to his multi-decade resume full of big name companies and high-profile projects as proof that he's good. The company's response was basically "OK, there's the door".
I for one commend "whiteboard refusers."
We're not architects, and nor do we have a certification system. There needs to be SOME way to probe a candidate's abilities, and it's not easy.
I'd rather work with the person who would approach that by taking 3 hours to plan, 3 hours to code, and 2 hours to bug fix, rather than the "Ready, Fire, Aim" person who took 1 hour to code, 4 hours to fix what he did, 8 more hours to find his design was totally wrong and re-do major portions of it, 8 more fixing the bugs in that version........
Unless you plan to filter out the people who just jumped into the code, in which the exercise is just a trick question.
The point here is not "Write as many lines of code as you can". I want the candidate to plan - it's a crucial part of the job. FizzBuzz, sorts and the like don't require much planning. Minesweeper does.
Then ask for that, and assess the plan, otherwise you're expecting Carnac the Magnificent: http://raganwald.com/2015/05/08/carnac-the-magnificent.html
I did plan...but if you were watching me you would not have seen me planning.
I decided right away that my display was going to be a table with one table cell per Minesweeper cell, with the table cell's ID of the form "X_Y" where X and Y are the coordinates of the corresponding Minesweeper cell. I did the rest of my planning while getting that HTML written (first in vim, and then when that was going too slow even with macro assistance by writing a little Perl script to generate the HTML).
I was not able to do it in an hour. At the end of one hour, I had everything for a rudimentary Minesweeper coded, but had a bug in the code that reveals the cells around a clicked empty cell, and it took another 20 minutes to find and fix that.
Here is what I came up with: https://github.com/tzs/interview_minesweeper
The initial commit was made at the one hour point, so that would have been what I would have turned in during an interview.
A couple of years ago I applied for a certain company that this year turned out to be a unicorn.
The automated reply said to build a Minesweeper. So I did. It took a several hours to get into a shape that is playable. Then it took several days to polish it into app store quality.
You can play here: http://www.ronilan.com/bugsweeper/
Download here: http://www.appstore.com/bugsweepergame
And copy as much as you want from here: http://www.ronilan.com/bugsweeper/js/bugsweeper.js
If you are wondering how it went from there, they were "evasive". Then I got a softball "center a div" interview with one of their front-ends. Then I got a polite no thanks from the recruiter.
And that's all I have to say about it.
I won't even respond to these "companies" anymore. It's a waste of time.
I love to learn new things and actually really want to learn about the mistakes I made. Giving me a "no thank you" from a noreply after all that time is like a big middle finger in the face.
As I got deeper into IT-recruiting, I realised that candidate filtering at the top of the funnel is fundamentally broken.
The market for software engineers is different from the market for - let's say - actors. Especially for senior developers there is way more demand than there is supply.
This has the very important implication that as a company you can not annoy candidates too much. Asking for tedious homework, too many coding challenges or references might be already overkill. Good people do not really need to work for you; they can also work for Google Zürich (biggest engineering office outside the US), or other cool tech-companies / ETH-spinoffs.
If you look for a tech-job in the most liveable city in the world, check out my story "8 reasons why I moved to Switzerland to work in IT" on https://medium.com/@iwaninzurich/eight-reasons-why-i-moved-t... or send me a mail to the mail-address in my HN-profile.
With my comments I raise awareness for my side-income but primarily I try adding value to HN discussions.
The "undersupply of engineers" meme again! This is a simplification. Supply and demand are measured at a given price. There may be way more demand than supply at one price (salary), but supply and demand is balanced at a more appropriate salary.
First was the choice of language. They offered to take it in JS, Ruby, Python, or Java. I'm a .NET developer whose competent with JS, but I really could be better at it. I asked to take it in C# but they refused and said it's not about specific language knowledge. So I did in in JS. I turned out to be almost completely about JS specifics and quirks. Because I've been learning JS recently, I've had the fortune of having to avoid the bad parts. When asked to recite them I struggled. This part is obviously my fault but if I had been allowed to perform in C#, I would have done a hell of a lot better.
Second was the environment. I have very limited exposure to programming on a Macbook and I spent the first few minutes struggling to translate all of my shortcuts. During a timed test it was pretty disheartening to not be able to type as fast as I could. I haven't used Sublime before but the default theme was near unreadable and I was allowed to switch it after a few minutes. The third was the test framework. I really wasn't familiar with how Jasmine worked in the browser, and it showed. Again, my fault. But taking 5 minutes to familiarize me with their setup would have made things go much, much better for me.
Before I get jumped on for blaming them for my poor performance, this was for a test automation role. I explicitly said that my JS/Jasmine/etc. skills were not that great and they still brought me on site.
Honestly, I would have preferred the whiteboard. Each test took up nearly the entire period, leaving very short time for talks about what I really care about - culture. I have a Github with a bunch of good projects. I've been employed as a developer for a while now. I can obviously program well. Drop the bullshit and lets talk.
Even asking them to change it to Dvorak for me (since I wasn't familiar with the menus) I could tell they were judging me. Once I bit the the bullet and learned the mac interface suddenly people were assuming I was super hardcore because I use Dvorak.... Stereotypes cut both ways I guess.
So.. "This isnt about a specific language, but you must use the ones we tell you!"
At one place - they gave me a very old computer and their actual code base and told me to fix a bug. No other information provided (that was actually a fun exercise).
Different places do it differently - but all I can say is that none of these are enjoyable to the interviewee, and I think in most cases they are not enjoyable to the interviewers either.
But the absolute worst thing is dealing with recruiters - starting with asking candidates to fill out forms so long that would put gov bureaucrats to shame. They also have a list of tech questions given to them to "screen" candidates and you can guess how that goes ("No not Linux, have you worked with LAMP?", "I am not talking about Javascript, I'm talking about jQuery, could you rate yourself on jQuery?")
I'm generally a fan of getting a candidate to tell me about aspects of their previous work that relate to the skills the job requires. I wrote this at my last job on the topic: http://www.huffingtonpost.com/alanjohnson/were-hiring-engine...
Depriving of recipe books is perhaps a better analogy? I think depriving of saucepans would be like hiding the keyboard!
That actually works pretty well, even doctors are expected to look things up if they get stuck. Why should programmers be any different? SO is extremely useful for funny edge cases in software that someone else spent an afternoon trying to figure out.
Also, the only profession (I know of) where twenty years experience is considered a detriment. Perhaps fashion modeling is another? ;)
In the interview we ask things like:
* what is OO and how is it different from procedural programming? (high level conceptual) * why do you want to leave your current position? * what appeals to you about this job? what do you think you might like least? * what is the purpose of testing? * tell us about the programming languages you've used. what's your favorite? what do you like most about it? least?
I'd rather have a more conversational interview after already having established their ability to code.
People keep saying that, but I'd MUCH, MUCH rather do a single couple of hour "homework exercise" that I can submit than do a full day (or more!) of "implement whatever pet algorithm/data structure I think is really swell" for like 3-6 different people in series, which is how a horrifyingly large amount of companies still tend to do things.
Give me a task, let me do it, have your tech people discuss my solution amongst themselves. If they like my solution bring me in to discuss it (to verify I didn't "cheat") and to determine culture fit. Seems like a lot less wasted time for everyone with far few of the stressors (hard time pressure, etc) that make most tech interviews so amazingly terrible.
The best part is that a lot of companies make you do both : ) I've had that happen to me.
FWIW, when I attend an interview, as either party, I want to have a discussion, a bit of back-and-forth, and crucially to determine if it's likely we'd be able to get on, day to day. Coding competency is not the only important thing. And anyway, it can easily be established via a bit of white-boarding, conversation, and preparatory reviews of any existing work they have online. If you are in doubt, then maybe pair on a fun little algorithm,... something that's not solved, and not dull. Something original would be best.
Being given a pre-packaged task to complete feels very procedural and bland...
This part is important:
> Having a consistent test also allows me to compare candidates easily and without bias.
Also, I think most people would care more about the content of the exercise, rather than whether it is prepackaged or not.
No, it isn't. The person who performs best on the test is often not the person who will perform the best on the job.
Tests like these should always be pass/fail.
This is how I interview - We have a bitbucket set up with a project very similar to our current site. We simply ask everyone who applies to get the code from Bitbucket and install it locally. And then fix a simple bug. Most developers who are well versed in the basics are able to install the code in 2-4 hours. People new to something specific are stuck for a day. It helps us weed out people who are a) Not comfortable reading code b) Not hungry enough to ask questions or google a little bit if they do not understand c) Not able to debug and fix simple issues.
So far all developers who have lasted that test have been good technically. Its another challenge to keep them motivated over long periods of time.
As a candidate, you'd bring less pre-packaged tasks and more questions and things to look out for. When I'm on the other side of the interviewing table, I evaluate the company as much as they evaluate me. Things like "What does it take to succeed at this company?" and "If I'm not meeting expectations, how will I know?" are critical to see if you want to work there.
Anyway, stay tuned!
Where I am right now, I'm pretty happy just turning down the interview if I don't oblige.
For example, I immediately recalled I solved the problem of mine placement with a 1 liner like:
var cells = ('*'.repeat(mineCount) + ' '.repeat(size - mineCount)).split("").sort(function() { return math.rand() - .5; });
It evenly distributes over 0-1. Removing .5, just distributes it evenly between -0.5 and +0.5. Sort basically says >0, the first option should come first and if <0, the second option should come first.
I believe this still maintains a random distribution and is quite a tidy solution. Thoughts?
As the list is sorted, elements are inserted from the back by finding the first element that the new element is smaller than. The last element to be inserted is the one at the end of the unsorted array, and, with a random comparator, it has a 50% chance of being "larger" than the biggest of the rest of the elements, and thus remaining at the end of the list.
Microsoft used this technique to "randomize" a list of browsers back in 2010, but, when viewing the page in IE, it ended up putting IE in the last position (out of a set of five browsers) 50% of the time: http://www.robweir.com/blog/2010/02/microsoft-random-browser...
At interviews where I've later been given an offer, it's usually a relatively small algorithmic design challenge, where the specifics don't matter, just the pseudocode.
In interviews I've given involving whiteboard programming, I've always emphasized that it doesn't have to be valid or even semi-valid code for any language, just a semi-rigorous pseudocode expression of an idea.
I think, "if done correctly", whiteboard coding is a good filter for those who can think algorithmically, who can catch the bugs that aren't typos, but incorrect assumptions.
That said, that sort of mindset may not be what you need for your role. Need to consider that.
---
Tangentially, one of the better technical interview experiences I've had involved chatting for a while, the interviewer describing an actual problem they'd had (relevant to the role and the company's product), and discussing my approach to the problem. (and passing or failing the interview wasn't dependent on using the exact same approach...)
I'm sure you're not, but my experience is completely opposite of yours. I'm still relatively early on in my career (I've had 3 software jobs, but have grown up coding since I was 12), but every job I scored was one that didn't use a whiteboard/paper problem during an interview.
I got a lot better at whiteboards with practice; but early on, every whiteboard problem I received resulted in me blanking on even the most basic concepts. I would have no idea how to solve problems on a whiteboard that I explicitly solved in a codebase the week prior. Then I'd step out of the interview and the solution would just hit me.
It's like state-dependent learning, in a way. As I got tested more on whiteboard coding, I figured out that whole problem-solving process better, and I do pretty well on them now. Still, I continue complain about those problems. They feel like a completely foreign, perpendicular challenge compared to having a discussion, answering questions, or writing code to solve the same problem. It's not a skill I've ever used outside of interviews, nor is it a situation that has ever been simulated at any workplace I've been at.
There's nothing wrong with evaluating a candidate's whiteboard presentation skills. If the goal is specifically to evaluate technical or algorithmic knowledge, it's worth taking into account that testing for that knowledge on a whiteboard can introduce a side-effect that bottlenecks the candidate's ability to present their answer.
Personally, if you know you're going to be interviewing, start practicing for whiteboarding. There are books and online resources available to help you get at least comfortable with it. The resources also help drive home that whiteboarding should be about process, not about perfect solutions.
Algorithms have generally taken years to perfect by PhDs, why should you be expected to perfect an algorithm you haven't memorised in 30 minutes?
I actually have no idea how.
I'm happily employed at the moment (knock on wood) but when that day comes where I have to go back out an interview, I don't plan on wasting any time doing these kinds of interviews where they make you spend hours doing some kind of programming challenge. To me it's an indicator that the company doesn't have anybody knowledgeable to actually pick out the best programmers.
You're wrong: http://www.ioatwork.com/selection-methods-almost-a-century-o...
Would you prefer Google's take on it? http://www.wired.com/2015/04/hire-like-google/
> So most "sample work" for an interview will actually be nothing like that but a short rushed version that is supposed to emulate it in some way.
Yes, that's the idea! And it works.
"but for many jobs there are too many variables involved day‑to‑day to allow the construction of a representative work sample. All our technical hires, whether in engineering or product management, go through a work sample test of sorts, where they are asked to solve engineering problems during the interview."
Plus a number of other things according to the article.
Sounds like they are agreeing with what I said (at least to some degree), but still asking for something anyway.
Yes, combining methods works even better! It's far from saying that the best way is "to just talk to them".
I have told potential employees as much after they asked for a sample of my work (which I had made available for that purpose), and they after that, they still wanted a three hour exercise done before I had even spoken to anyone.
In fact I am about to start a new job in the new year, I had a few interviews, and a couple of jobs that interested me sufficiently. One involved a three hour test, the other asked for a sample for my work. Guess which one I took? Guess which one the interviewer said they were struggling to find candidates?
Half of what developers do is master arcane rules from obscure texts. Is this not, in fact, a work sample?
I managed to get something fairly workable, albeit ugly code-wise, banged out in 1hr. If you reviewed my google searches, you would see that:
- I didn't know how to init a 2d array in javascript
- I didn't know that the things on the canvas couldn't be accessed directly for click handlers
- I cheated on the solution for finding neighbors because I was running out of time
*edit formatting
1) I want to see passion for solving problems in the space. There are endless problems to solve in the industry, but I want to know they'll be excited to work on my team's problems.
2) What are they doing here? Was this a numbers game (interview at a ton of different places in hopes of getting one good offer)? Were they recruited or did they actively seek this position out?
I've seen candidates where in one interview I have to throw multiple problems at them, because they think about it for 30 seconds and then come up with a correct solution and quickly write it out on the board. I'll usually run out of questions for these candidates.
Ultimately, white-board coding works well for some, and is a complete disaster for others. Yet I don't believe it means white-board interviews should be thrown away in an effort to be "fair." I just think as engineers, if we really want to land a position at a Google, Facebook, or Uber, etc, is it too much to ask that we study up on our algorithms?
I just played with JSBin and it doesn't have autocomplete out of the box.
Edit: whoa I just re-read the OP and found this
> Most [candidates] tend to use the laptop & JSBin provided
That's super hypocritical, OP (see your paragraph headed "Comfortable situation")! If you "provide" them with an environment of course most are going to use what you provide since that is what you are suggesting. So now you're filtering for people for whom that laptop model and the JSBin tool are familiar and comfortable.
Or in the case of those that reject your environment and choose their own familiar and comfortable laptop and editor, you are filtering for those with the self-confidence to reject the suggestion of the interviewer, which doesn't really help with your stated goal.
It's a shame because I like some of your other points :/
The key is to get them to explain things until you understand the essence at a foundational level - hopefully you understand what combination of the notions of statistical independence / power / dimensionality / algorithmic complexity has birthed novel system or algorithm they built.
The process selects for people who have a) already built something substantial, and b) have the communication skills to explain it.
I agree that this seems like a good way to establish someone's technical competence. I do wonder though from the interviewee's side what the optimal time to spend on one person is. It's kind of like dating right? (Well what I imagine dating would be like if I lived in a big city) You have concurrent first impressions with maybe second and third follow ups before you seal the deal. This may be a bit like asking for a relationship on the first date.
Demonstrating ability in this manner works for more junior people who maybe don't have a github or other personal projects, but I'd consider discussing and reviewing those more valuable.
I stopped making such assumptions since I started interviewing.
Warming up question: "How would you check that two strings are the anagram of each other ?"
"What's an anagram ?"
Really? I barely know anyone who works for several years at any place. I'm looking at whether I can tolerate it for six months. That's only so I can take a job search break for a few months before hitting the grind again. I'd love to work at a place that's so great that I'd be there for several years but that's hard to imagine. Most companies I've interviewed with don't exist in 3 years, let alone 'several' (7+).
The startup scene is not representative of the job market for the majority of people.
On that note, I'm surprised that startups would hire frequent-short-stint employees. The startup I work at has a great proportion of people who have stuck around since the early days; yet I have worked (and interviewed) at large companies where everyone bails within a couple of years max. I don't know what the West Coast "scene" is like, so my company probably doesn't resemble SV startups, but the pay is great, the people are smart and passionate, and the businesses is hitting all of its goals, so I'm not complaining.