Hacker News Reader

Comment by StevePerkins | Hacker News Reader

I do almost all of the technical phone screens for my company. The process starts with a "where-do-you-see-yourself-in-5-years" personality screen, with an H.R. drone or outside recruiter. It moves on me, and then on to a brief "homework" coding exercise that we ask people to write and submit. If we like their code sample, then they come in for the panel of "whiteboard-exercise" people who conduct the face to face round. At my stage, I seldom bother with certification style questions about the programming language, or with highly abstract thought exercises, because other people will do those things in the face to face stage.

Typically, I ask questions that give me an idea about the candidate's depth of experience and awareness. For example, "I see that you've spent X years using Subversion for source control. What are your opinions on trunk-first development vs. branch-first development?" I ask questions that speak to practical experience and design ability, without getting into too much depth. For example, "Pretend that you're using an OO language to build an application for <insert purpose here>. Just off the top of your head, what are some classes that you would expect to see in your class diagram?". Etc.

I find that this level of questioning is a much better screening tool than trick questions about programming language quirks or the minutiae of frameworks, or cliche puzzles about how many golf balls fit on an airplane. However, I groan and roll my eyes when I hear people challenge the need for technical interviews at all. Yes, they are necessary.

Having performed a thousand interviews by now, I am awestruck by how poor the software development talent pool is. I am aware that the Bay Area is overrepresented in HN's readership, and that crowd tends to take for granted the talent level found in the technical equivalent of Mecca. However, I assure you that the rest of the planet is dominated by sleepy line-of-business developers... who have all the passion beaten out of them in the first 5-10 years, and spend the rest of their career just phoning it in and not growing.

I sometimes ask candidates what the letters "MVC" stand for. The successful response rate is around 50/50. I ask candidates to briefly explain the advantages of the Model View Controller pattern, and only 10-20% can field the question. We bring in Java and C# candidates who have been working with their respective language for 10 or 15 years, and they get COMPLETELY EXPOSED during the face to face round when asked a series of basic certification exam style questions. Nothing tricky, just core fundamentals.

People who post here are not the norm. The "norm" is atrocious. So yes, unfortunately we all must endure technical interviews... to filter out people who have enough confidence or personality to excel in the other interview segments, yet are utterly useless.

Let's put it this way: the more experience I've had with interviewing, the more selective I've been in my own job searching, and the more aggressive I've been in my salary negotiations. If you are really good, and are located outside of San Francisco, then you are worth your weight in gold and should value yourself accordingly. You wield tremendous leverage once you make a strong showing in a technical interview.

View on HN

> "I see that you've spent X years using Subversion for source control. What are your opinions on trunk-first development vs. branch-first development?"

Actually, this demonstrates a problem right here. It's not clear to me what you mean by these terms. A quick Google search ("trunk-first vs branch-first svn") suggests that you're not using common terminology.

After thinking about it for a minute or two, what I _think_ you're asking is "should all development happen on a common branch, or should developers create separate branches for individual features/fixes, merging back into the mainline when finished". And, indeed, I'd be happy to have a conversation with you about this.

But if it takes me a couple of minutes to figure this out while just sitting here at home, in the pressure of an interview, I'm probably going to fumble, or say "I don't understand what you mean", which will make me wonder if I've blown the entire interview. This despite the fact that I've been programming for 20 years and have used a number of version control systems (CVS, ClearCase, P4, Subversion, and most recently Git).

In an actual interview, I would only pull out that specific example for a candidate who's worked with CVS and Subversion almost exclusively. The actual question is also a bit more verbose:

Have you worked for a company that strives to do as much development as possible in 'trunk', creating a release branch near deployment time for production bugfixes? Have you worked for a company that preferred to branch at the outset of new development, merging back to 'trunk' periodically? What did you find to be the strengths and weaknesses of each approach?

Their answer lets me read between the lines and gleen much more information about their work history. Have they worked in settings where multiple work streams were in development simultaneously? Do they have substantial experience in collaborating without trampling on shared resources? If so, then they usually mention the different pain points in merging. I don't consider there to be a "right" or "wrong" answer to this question, and not having a clever answer certainly doesn't disqualify someone from being a strong programmer. But it does help to level-set, and identify candidates who think like team leads or might be well suited for responsibly beyond raw coding.

However, I don't want too much text between parentheses that are injected into a long sentence. So you get "trunk-first vs. branch-first". :)

Exactly why is it that you would making a screening decision based on version control practices? Of the many things you'll need to ramp candidates up on, this seems like one of the very easiest. Not only that, but because most firms do release management a little differently from each other, there's some VC ramp-up effort you'll need even with people who have mastered your VC tool.

I feel like dev interviews are full of questions like this, things that require some expertise and experience to answer, but do virtually nothing to predict on-the-job performance.

When you ask an interview question, you are pricing candidates. That's obvious when you think about it: you're screening, and so your questions alter the supply of candidates that will hit the bottom of your screening processes. Fewer selectees -> poorer employer BATNA -> higher prices.

Do you really want to price candidates based on how they use version control tools? How much are you willing to pay extra for people who have a lot of experience with different VC methodologies? Are you sure the weight of your questions about VC match up with the (hopefully minimal) premium you're hoping to pay for VC expertise?

I'm not sure that you fully read the parent comment. I could care less about their Subversion expertise. My company doesn't even use Subversion anywhere.

However, if you list "10 years experience with <Technology X>" on your resume, then you should absolutely be prepared to discuss your experience with <Technology X> at a high level. Not anal minutia or contrived trick questions, but certainly you should be able to respond to an open-ended question about the basics with enough context to show that you weren't lying to pad your resume.

More importantly, as I explained in the parent comment, I am interested to see if their response reflects experience with multiple teams working on parallel over overlapping efforts within the same codebase simultaneously. Everyone says that they have experience like that, but if you poke a bit deeper you find that half the time it's exaggerated. They may have worked in a context with multiple teams, but affecting the same area of the application or system. If you have had legit experience of this kind, or at least show a high level of insight in talking through the issues that can arise, then you might be considered for a team lead role sooner than you otherwise might have been. As I said earlier, it's not a "right or wrong" question that can disqualify you from being a capable programmer... it's a "level setting" question that helps gauge which level of responsibility you might start out with.

There is no less productive form of interview than the "resume validation interview". Resumes are practically useless in the best case. Here, you seem to propose paying a premium for candidates who can properly estimate their facility with VCS systems and then cogently discuss that estimation in an interview. That can't possibly be a skill relevant to building software on your team!

Personally I think it's an excellent question. He's probing into a candidate's understanding of why you use version control systems and why different approaches might be used.

If you have a significant amount of experience then hopefully you have seen enough different situations to be aware of the advantages and disadvantages of each approach.

It's really a question to figure out whether or not you can reason about high level concepts.

Stipulate that it is. Why is that a good idea for an interview?

personally, I think the SVN question is a pretty good example of "can you talk intelligently about any aspect of any complex system you've ever worked with?". If a candidate can explain one, I'll recommend to hire them. The resume provides a good list of possible subjects to discuss, otherwise I just cast about randomly, try to chase threads in the conversation until I find something with some depth.

This.

I completely agree with this - asking about VCS is a cross-cutting question that nearly all candidates have experience with unlike, for example, graphics, desktop, or cloud experiences. So, requesting that the candidate explain their VCS workflow is a way to generate a practical and relevant data point to use when evaluating multiple candidates.

For example, the 1-in-a-million candidate that has used git with a CI/CD configuration (note: I'm also outside SV) versus the candidate who uses TFS ("git? You mean Github? Yes).

I've carefully read every comment on this subthread, and here's what I think:

This question is a trifecta of ineffective candidate screening tactics:

(a) It's a technical screening question, one a strong candidate could get wrong, based on a technical aptitude that is trivial to teach on the job and thus rarely worth paying a premium for.

(b) It's a subjective technical question, for which reasonable engineers can have differing opinions, which means it's an outlet for interviewer subconscious bias. Did the interviewer just eat lunch? Candidates will do better on this question if they have.

(c) It's a tea-leaf-reading question about engineering/team management: it's superficially and overtly about technology, but subtextually about a bunch of other things. Let's hope the candidate realizes that.

A typical interview lasts about 60 minutes. Let's say it takes 10 minutes to pursue this particular line of inquiry. That's 16% of your interview you're spending with a question that greatly rewards people who are good at talking about technology. Worse, if you ask that question early to a quiet but excellent candidate, you can psych them out, which means you pay for that version control question in every other question you ask.

It is totally reasonable to assess soft skills and team compatibility. But you have to design an interview that does it. You can't improv it based on candidate resumes.

Avoid questions like this.

This thread started with someone saying they're pretty good at interviews. It turns out that they try to assess soft skills with technology questions based on the luck- of- the- draw of candidate resumes. Candidates with effective resumes will have an easy time passing these interviews. From the comments on this thread, that obviously sounds reasonable to some people.

I submit: those people are not competing for talent. They may think they are, but the real contenders in this market won't be OK with letting good candidates slip past because their job-hunting skills aren't finely tuned. In fact: they'll do the opposite: those candidates are steals in this market.

The original commenter acknowledged that when he said that his experience was that the market was full of poor candidates. If that's the case, you especially can't afford to filter out effective devs because they fail to impress you when they explain how they use version control, or when their resume overstates their facility with version control.

Thanks for being open about your perspective on the interviewing process, and for taking the time to follow up on these threads.

Do you believe that technical phone pre-screens are ineffective in general? From what I've read Matasano doesn't pre-screen candidates, but provides complementary study materials instead. Is that because the subject matter is specialized? Would you approach hiring web dev roles differently?

From your remarks, it sounds like you would reject the practice of asking open-ended interview questions (e.g. describe your workflow, describe a typical day, describe a recent project) due to interference from the interviewer's bias. What, if any, value do you place on open-ended questions?

In general, my feeling is that the Matasano process (which I currently manage) works outstandingly well where there isn't a flood of qualified candidates. If you have a glut of folks who are ready to start working, you can get away with a terrible process.

We do free-form technical interviews, but only to try to detect candidates who really aren't ready for the work-sample challenges. Our in-person interviews are standardized and try to evaluate consulting/architecture skills that are hard (impossible?) to measure without people. These involve open-ended intermediary questions, but the final answers are structured.

The bottom line is this: you cannot compare candidates using free-form interviews. You must compare candidates who are going to be doing similar work. Thus, free-form interviews have no evaluative value.

Do you ask to see the resumes of candidates?

"Can you discuss what you've done?" seems like a skill likely to be highly relevant to team software building.

It's highly relevant to getting a job on a software team, but only because most teams are assembled via interviews, and verbally relating stories from your past in a face-to-face meeting is the most effective interviewing technique.

And it's relevant to a job like sales. (That's unsurprising because a job interview is actually a sales meeting.) And many management jobs do have a sales aspect -- you have to justify budgets, sell work inside the company, et cetera.

But if explaining our work to outsiders were a particularly important or routine skill for programmers, we wouldn't be so bad at it. And, on average, we are bad at it. Because our actual on-the-job communication, which we practice all the time, is largely written and asynchronous, taking place on media like Slack or Github. It relies on plenty of job-specific shared knowledge, domain experience, and jargon, and it all happens in the shadow of a job-specific shared codebase that is supposed to speak for itself -- the whole point of software is to build something that works by itself -- but is also perpetually unfinished.

There are social skills that are important to have on a software team, but it's difficult to judge them in an interview. Interviews are staged events. Challenging a candidate to defend a resume in an interview is like asking them to do improv comedy, and selects for many of the same factors: Verbal gracefulness, comfort in the spotlight, the ability to seamlessly change the subject, and the amount of time spent in rehearsal. Good candidates rehearse their resumes. We get to write them ourselves, after all, and with practice we learn to design them with hooks that lead into our best material.

I would say the whole point of software is to build something that people can use. If someone builds a module, but can't tell me how to use it, it's not of much use to me.

Oh, but surely they'll write documentation? Programmers, by and large, seem to suck at that too, which is why we have tech writers. But somebody still needs to explain to the tech writer what's happening! And only a few companies seem to carry tech writers for internal only products.

To put it simply, if I ask somebody how their code works and they say "Go away for an hour while I write documents" I think I'd rather not work with them. Or worse, they ask me for help debugging but can't tell me what they're trying to do. No thank you.

Your questions are good and relevant ones, I agree. Let me rephrase them a little:

"Hello, coworker! Did you enjoy the cake we both got to eat the other day in the company cafeteria?"

"By the way, I'd like to ask you a question, and don't worry: This isn't an interview or anything, so if you can't answer me right away, or if your answer lacks grace, it's not as if you'll lose your job."

"Anyway, coworker: I found this code, which you wrote while working for my company, under the direction of my company's management, and which solves a problem that my company actually has, and which builds upon my team's platforms, languages, and coding standards, and which might even link directly to my code, and which both of us have had a moment to read and think about and which is right in front of us on this monitor. How does this code work?"

"Also, can you help me debug this code I have here? It builds atop the code I showed you last week, and is written in the same language that we all use, and attempts to solve a problem you've seen before – which is not a coincidence, because you were the person who asked me to solve the problem."

These questions are incredibly relevant to our work, but interviews can't cover them. Candidates are not our coworkers and they share none of our context. Instead, interviews are, at best, an exercise in prediction. In practice, they are often an exercise in magical thinking.

During the workday, people aren't being constantly judged. They don't implement functions on whiteboards without unit tests, solve brainteasers out loud during stand-up meetings, or implement quicksort from memory. They do have to explain code to coworkers, but not to people who don't understand the problem space, the language, the background, or the constraints. These rarely-exercised feats of skill are valuable -- sales is valuable -- and our gut feeling is that such feats are somehow related to relevant job skills. But gut feelings are often wrong. And not every job is in sales.

>Here, you seem to propose paying a premium for candidates who can properly estimate their facility with VCS systems and then cogently discuss that estimation in an interview. That can't possibly be a skill relevant to building software on your team!

I think that's an unfair interpretation of what StevePerkins is saying.

I don't get the impression from him that VCS knowledge in particular will make-or-break a candidate. He's saying that IF the candidate put it on his resume, then the candidate himself is the one who opened that door for discussion.

Out of the infinite list of programming topics to discuss, what are some options to narrow it down? Well, the candidate (through his own volition) put <Topic X> on his resume... so... let's talk about Topic X! It doesn't matter what Topic X happens to be (whether it's "svn", "parallel algorithms", "Ruby", "TDD", "cloud scaling", whatever). What matters is that the candidate is the one who thought it important enough to highlight it. From there, it's reasonable to think it's something the candidate is already comfortable with discussing in depth. If not, he shouldn't put it on the resume.

Your response makes it seem like StevePerkins is playing Alex Trebek with Jeopardy random topics and asking "gotcha" questions. It's not random -- the source is the candidate's resume. It certainly seems fair and reasonable to discuss any topics the candidate put on his resume. Imo, it's also fair to augment with questions that are not represented on his resume (but that's a different discussion from StevePerkin's example.)

That's not what I think is going on here. He's not asking "Tell me the commands that I have to type to do X, Y and Z in GIT, SVN and Mercurial", he's asking a higher level question about how one can utilise VCS (and DCVS) tools to work in different ways.

Depending on the experience of the candidates he's trying to acquire/interview, this may be appropriate to indicate that they have either experienced different ways of developing at different companies, or have an interest in software development practices wider than just "The way I've done it is the way I was told."

I'd consider it an 'indicator' question, indicating that the candidate has either experience or interest in software delivery and the way that different organisations work/operate. It's probably not a make-or-break question, but if a candidate had a good number of years under his belt and hasn't at least heard of some kinds of different practices then they might be the "do what I'm told but no more" coder that that organisation wants to avoid.

There is a subtle but crucial difference between "candidates with experience working on teams delivering enterprise software" and "candidates with aptitude for working on teams delivering enterprise software". Worse, many in the former set aren't in the latter.

You might want to build an interview process that selects candidates who belong to the latter set. I'm a little baffled why you'd want to select from the former set in preference to the latter.

I got a bit baffled by that too, but came to the same conclusion as you did after a minute or two. In an actual interview, I suppose I would just start by clarifying what is meant by those terms before going on with a discussion about it?

> Actually, this demonstrates a problem right here. It's not clear to me what you mean by these terms.

I'm not trying to get in an ego stroking contest here, I'm just providing anecdata.

I worked with SVN for ~4 years and git for ~4 more. What StevePerkins was asking was perfectly clear to me after about three seconds of thinking. Of course, in an interview, I would be sure to parrot back my understanding of the question. :)

FWIW, you and I understood his question to mean the same thing.

I feel the same way, with approximately the same number of years of experience with each. And I definitely have strong opinions on the answer to the question :)

I don't see the issue to be honest; it's an interview, not a written exam. You can think aloud, the interviewer can guide you if you're confused by the terminology and you can ask clarification questions.

Unless a software developer is tasked with deciding which version control system to use or for some odd reason has to do a deep dive into the philosophy/design of subversion, why on earth would they bother to know this? You might as well ask if they prefer Cherry MX Blue or Brown. I don't know SVN but if it's anything like Git, there are 100 ways to use it, exactly 5 of which are useful to 90% of developers on any given day.

I think you could just ask for clarification. I suppose the whole point is to have a conversation.

Of course this again biases against people who suck at interviewing (too nervous or whatever).

A quick Google search ("trunk-first vs branch-first svn") suggests that you're not using common terminology.

It's not a terminology question, it's a concept question. A good candidate should be able to recognize abstract concepts regardless of the words used to describe them. Even in an interview setting.

Why? What makes being able to persuasively discuss this particular concept an attribute of a good candidate? And, when you identify the answer to that question, can you then answer: is this question the best means I have of assessing that attribute?

I agree with you that this type of technical interview is a great way to fail to hire ideal candidates. I've experienced this myself. I was commenting only on the specific complaint about "terminology," and that one should be able to see through unfamiliar terminology in an interview, not that such an interview is the best way to hire excellent developers.

Although I believe everything you say, I'd like to provide a counterpoint.

I think I'm a pretty good programmer and I've got a reasonable body of work on GitHub to back it up, but I've often failed technical interviews because I go to pieces under the pressure and my brain just stops working. I've been a dev for > 15 years but if anything I've got worse at interviewing over time. I only apply for positions that I genuinely think I'd be good at, but the technical interview gives the impression that I'm a clueless idiot. From my (admittedly selfish) point of view, the approach proposed in the article seems much better than the process you describe here.

You should spend time improving your live performance skills. It comes in handy for more than just job interviews.

I think you're correct, but I also think it's a real shame that live performance skills are important to excel at a career that has nothing to do with live performance.

I wonder if there are any careers where you only have to get good at the thing you're doing rather than a bunch of meta-things.

Live performance skills are most absolutely unnecessary to excel in development; this comment thread is discussing how certain people can improve their interviewing performance through the practice of being spontaneous and present. A very helpful trait that some people have naturally, and that others do not.

I think it's great when the practice of some discipline or craft has beneficial effects in other areas of our our lives.

Personally, I think (or at least hope!) that I'm pretty good at differentiating between nervousness vs. simply not having an answer. With the MVC example that I gave earlier, sometimes people fumble along and provide a lot of information even though they can't pull it together in any coherent way. Other times, they flat-out shrug and tell me that they don't know. Protip: I recommend saying something and at least making an attempt at the former.

I like to ask people about side projects, and what is the latest interesting thing that they've been learning in their own time. However, my issue with the "Your-GitHub-Is-Your-Resume" nonsense is that it falls apart past the age of 30. Before I got married and had kids, I used to spend almost every waking moment of personal time (and about half of my employer's time, to be honest) working on areas of personal interest. I wrote a handful of minor open source frameworks that are too obsolete to be worth mentioning, a set of Java bindings to the wxWidget library, authored a book, and served as technical reviewer for a stack of other books.

Then I got married, and became a father.

Now, quite frankly, I don't do that anymore. I just can't. It isn't a matter of passion, it's a matter of physics. My GitHub account includes some small personal apps that I tinker with from time to time (e.g. a diet and exercise tracking application), but I would be MORTIFIED if someone thought that was a representative sample for job application purposes. It's a fairly trivial web app, that I could just as well have written 10 years ago.

In fact, the type of work that you do later in your career really doesn't lend itself to showcasing in a small GitHub portfolio. You work on Node.js websites, with whichever client-side data binding framework is popular this week? Awesome, GitHub would be perfect for showcasing an example of that. But your most recent projects include integrating a dozen microservices with an AMQP broker and Apache Camel, or using Spark to crawl a mass of data in a Cassandra cluster? Large scale development just doesn't lend itself as well to representation in a personal GitHub portfolio.

So side projects may be a useful screening tool with junior level candidates for web developer positions, but I think it's a naive suggestion for more senior level candidates in more complex domains.

The notion of having an on-site coding exercise is better in my opinion, but that's not a panacea either. Anything worthwhile would probably take a matter of hours, turning your on-site interview into an all day affair for which the candidate would have to take a full day of PTO from their current job. There is NO WAY that I would submit to something like that unless I was already very far into the recruiting process with a particular company, and very much interested in working there already. You're not going to reach that point unless you already have technical screening tools in an earlier stage of the process, so there you are right back at the original problem.

I do agree with the article, as well as many of the comments here, that "whiteboard exercises" are a fairly pointless tool (e.g. "write a recursive function to traverse this tree", or "walk me through your process for guessing how many golf balls could fit in an airplane"). Even though my company does a bit of that too, I grumble about it and refuse to incorporate any of that into my stage of the process. However, almost all of the alternatives that I've heard suggested seem to be proposed by very young junior-level devs, who lack perspective on what their career path and mindset are going to look like 10 years down the line.

Thank you. There should more interviewers like you and i hope to be facing someone alike the next time i am on the applying side of the table.

I felt bad about my nervousness in interview situations too, so I started taking improv classes. Now interviews don't give me any performance anxiety at all; once you've performed in front of a large audience with no idea what you're going to say or do, it makes interviews a cakewalk.

And remember, even though the primary skill you're interviewing for is coding, they are also testing you for communications ability. Being able to clearly and concisely explain what you're doing is at least as important as being able to get things done.

I've been doing improv for 10+ years and had never realised this, but it may well be a large part of the reason why I don't get nervous in interviews. Thanks for helping me make the connection!

Plus, as one other commenter points out elsewhere in this thread, once you've been on the other side of the table for a while, you get a much better idea of where your own strengths lie (and what you're up against, which can instil a lot more confidence than you might think!).

Try Toastmasters. Definitely something out of most people's comfort/familiarity, but works wonders for improving speaking talents.

Hey I used to be in the same boat as you but recently I seem to have gotten over it and get lots of offers =D

If you're interested, the way I got over it was by just accepting the fact the interview process is not perfect and in the end I have no idea what these people are looking for. That being the case, I just do my best and enjoy the fact I get to work on algorithmic questions usually much more interesting than what I would get to see while working. I guess I try to just have fun!

It really seems like a lot of people freak out because they are afraid of rejection. I don't want to get stereotypical about how nerds are antisocial or whatever, but I definitely found that once I stopped caring whether I was going to get hired or not, I started doing and feeling a lot better. I actually started enjoying interviews because I get to see interesting questions, see what other people are working on, and because I am junior, learn about more technologies and why they decided to use them.

With respect, I think you might just be making (what we should now call) the "Fizzbuzz fallacy": "99% of applicants can't answer basic questions, therefore the talent pool is poor".

Really, it's more like "99% of the people still searching, who don't have a documented contribution history, and don't have a network that lets them find good jobs in a heartbeat, who must resort to this to find a job[1], can't answer basic questions."

Most of the talent pool is working and has to be pried away; they won't show up through HR channels.

[1] Not a diss, by the way; I count myself in that set, though am still not "on the market", and still don't fail at fizzbuzz style questions.

If you are outside any big tech hub, most jobs and candidates will be filled by recruiters or normal job posts. I've been developing professionally for about 15 years, changed jobs a lot of times, and never was through a referral/network. Most of the people I know are the same. And yes, also having done interviews and sorted through thousands of resumes, there is so much out of touch folks out there that I wouldn't hire anyone nowadays without at least 1-2 small tests to validate their knowledge.

Yes. Living in the midwest, I've done software development for nearly 30 years. I've had more than half a dozen employers and I have never been given any kind of "technical test" in an interview, such as writing code on the spot, diagramming an algorithm, or solving a puzzle. I don't know where this happens, but it's not everywhere.

I saw it in london once. I failed the test so badly that the interviewer refused to believe that I had written the demo project which got me a first interview. He said it was one of the best he had ever seen. But then said I couldn't have written it because I was failing the interview so badly. People who dont get that nervous just dont understand how debilitating being really nervous is. Also the white board code challenge was a total waste of time.

(I'm very late to this discussion, so I may try this question another time).

But I've been thinking about the marginal value of each additional technical question. In other words, how much additional relevant information do you get about a candidate increase with each increasing level of difficulty?

For instance, suppose you ask fizzbuzz and the candidate has an easy time of it. Then you ask about building/searching a binary tree, which the developer manages, but only after fumbling round a bit. Then you get into finding cycles in linked lists or graphs, and the developer takes a crack at it, but would need to look it up. Or maybe the developer gets it, and the interviewer ask about finding all permutations of a string...

How much more do you learn by going from fizzbuzz to binary trees. How much more do you get by asking about cycles in linked lists? And so on…

Just for the record, I can't stand technical interviews, and I dislike them so much that I'm considerably less inclined to apply for and interview for new jobs because I feel that I've studied for my data structures and algorithms midterm one time too many. I just don't want to re-load merge sort into short term memory again.

However, in spite of all this, I'm still somewhat sympathetic to interviewers[1]. The truth is, if you really don't have a lot of direct evidence about a developer, you truly are at risk of hiring someone who can't code.

[1] My problem with silicon valley hiring practices isn't that they have a process that leads to a high false negative rate. They should do what they feel is best for their company. My problem is that they do this while complaining about a critical shortage of developers so severe that it endangers the entire tech economy.

As a self-taught person beginning to write software who is not in SF, your description of someone who cannot explain the advantages of the MVC pattern makes me incredibly nervous. I don't really know the MVC pattern in detail, or what its advantages are. I do know about Big O complexity, data structures, loops, registers, as well as logic gates, and the basic structure of computer processors. I can code small programs in Python and Java, and have written some basic Flask apps.

Am I totally unfit for developing software?

If not, What is the correct answer that will prevent me from getting COMPLETELY EXPOSED (in your words) if you ask me this MVC question in an interview?

As a complete beginner? No, I wouldn't call you totally unfit for developing software. I'd call you entry-level, but that's totally okay (as long as your applying for entry-level jobs and have realistic salary expectations). I didn't specify this in my comment above, but I've never really had to interview for entry-level req's. The positions that I described involved candidates with years of experience... so the fact that THEY don't have a clue about the basic design of virtually every web framework for the past two decades (i.e. MVC) is a glaring red flag.

To be honest, I'm not entirely sure that the MVC example would be a good question to ask to a younger candidate. If you had been around during the early days of web development (i.e. CGI scripts based on Perl or raw PHP, or Java where all of your logic is crammed into a JSP page), then you could say all sorts of things about the shift toward MVC being the norm. However, if you came along after that shift, and every web framework you've ever seen has been some variation on MVC principles, then it's a bit like asking a fish to describe water.

Anyway, if you're just getting started in the field, and haven't had much interview experience yet, then try not to let it get to you. Interviewing involves a TON of rejection, and that never really stops. If interviews ever start feeling easy, then it means you're selling yourself short... and you should be interviewing for more senior positions with higher salaries.

To get started though, I would suggest simply doing a Google search on "programmer interview questions", or something more specific to your background (e.g. "java interview questions"). This is exactly what the majority of your lazy interviewers are doing to come up with their questions anyway. As you encounter questions that you can't answer, then look them up and read about it (e.g. http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93cont...).

>If not, What is the correct answer that will prevent me from getting COMPLETELY EXPOSED (in your words) if you ask me this MVC question in an interview?

Every interviewer has their own pet question that the candidate is COMPLETELY UNQUALIFIED if they don't know.

My short answer to "What is MVC?" is that it's a popular web development trend, but without much substance to back it up. Every MVC project that I've been supporting/maintaining was a nightmare, taking 3x-5x longer to get stuff done than what I consider reasonable.

MVC = Model-View-Controller. It's usually for web development. You structure your code in a way that separates the data logic (Model), the user presentation logic (View), and the glue that holds the bits together (Controller).

There are many popular MVC frameworks that let you churn out a lot of mediocre websites quickly (angular.js, Ruby on Rails, Zend, Django(Python), and many others). The MVC framework does a lot of routine coding for you, but it comes at a price. The code can become a nightmare to maintain, especially if you use the MVC framework incorrectly. If you do something outside what the MVC framework provides, it can become a handicap rather than a help. The MVC framework usually demands you write your whole application in its preferred style, such as spreading out your code in various directories with forced naming conventions. MVC frameworks tend to be poorly documented compared to the underlying language (For example, compare the zend documentation to php.net).

BTW, there isn't just one MVC pattern.

Yeah, I know, angular.js is MVVM and not MVC. It still sucked. The boss chose angular because he wanted to pad his resume with angular.js experience, and not because it was the right tool. That startup is still stumbling along, but wasn't as successful as it should have been.

(In angular, when you edit data in the view, the model is immediately updated, which is why it's MVVM and not MVC.)

Funny thing... MVC is a pattern for GUIs (Windows and Mac desktop applications). It was simulated for the web, and none of those that you mention with the possible exception of Angular allow a real MVC pattern in a web application.

The point of it is two-fold: separation of concerns, and DRY. But the reason it so ugly on the web when it produced elegant code for GUIs is the reason that web controller schemes are not MVC.

> Funny thing... MVC is a pattern for GUIs (Windows and Mac desktop applications).

Errm, it's a pattern often used for UI (CLI and GUI) programming that was first commercially introduced in Smalltalk-76 (in 1976). [This predates Windows and Mac by many years.]

MVC can be done -with varying degrees of difficulty- in any computer language. You don't even need a web development framework to implement it! ;-)

Just because it can be done, doesn't mean it can be done well.

iOS/OS X seems to do real MVC, with lightweight reusable view objects managed by ready-made controller classes. The obvious benefits are minimal memory and streamlined data flow. You only ever make/use the view objects you really need, you can recycle them to save memory, and - in OS X especially - controller objects include useful ready-made convenience methods.

Web frameworks often seem to do something that looks like MVC if you squint and don't think too hard. But when you're forced to use separate languages for logic (js), markup (HTML), and view design (CSS), and handle separate client/server environments, and there's inevitable overlap between all of the above (jquery etc) because stuff doesn't "just work", and you probably have yet another layer as a DB driver, it gets very complicated very quickly - to no great benefit.

The web has become a snarly ball of warty epicycles. That's why native is so popular - you get one common language for logic, views, and data, with a clean-ish interface to a remote server if you need one.

When everything works together like that, you can think seriously about MVC.

When the core concerns are all over the place already, it's a mess before you even start.

Which is not to say you can't work with it and build cool stuff - more that you can't just airdrop in a design pattern from a different tradition without thinking really, really hard about what, why, how, and what happens a year from now.

Actually, it never worked in Windows GUIs either. [1] It's just a broken concept.

Separation of concerns and DRY are both good. MVC doesn't succeed at either, except in idealized corner cases.

[1] A lot of widgets in Windows, for instance text fields, encapsulate the view (it's where the text is displayed), the model (which keeps the text in the control), and the controller (the widget handles the UI for entering text, catches and handles the mouse, etc.).

As an engineer with ten years of software dev experience who interviews quite a lot of candidates for my current team, I'd say don't overthink this.

BigO, data structures and having a good idea how computers work, what matters for performance both from theoretical and practical perspective, matters infinitely more to us that knowing how to play object-oriented BS bingo or subversion usage patterns.

If you know what MVC stands for, you're know enough about OO patterns to me.

Not going to argue that we got it right way(tm), just wanted to give you another perspective.

The MVC question is probably for candidates who have some experience in developing software. Not knowing it does not mean you are unfit for developing software. But, even if you have a little experience, you should probably know about MVC, even if you have never used it. This shows that you have familiarity with current technology. Not being familiar with current technology is definitely bad. If you are applying for a beginner position, not knowing about MVC should not be a big minus. You will only be tested on algorithms and data structures. If you don't know the answer, it is better to inform the interviewer of that and inform them of the topics you know about.

Once you get beyond small programs, you run into software architecture questions, which touch issues of maintainability and complexity. One of the techniques we have to manage this complexity is to break the program into components, smaller pieces that couple to each other loosely. This limits the impact of a change, since a change in one component is less likely to require changes in other components. At least that's the hope.

There's lots of ways to decompose a program, each with its own strengths and weaknesses. MVC is a particular decomposition that is useful for interactive applications. The components are named Model, View, and Controller. An example of an advantage of this decomposition is that you can apply a different user interface (e.g. a command line one) by exchanging the View, without needing to change the Model.

Other answers here have focused on MVC as it applies to web development. But MVC is from the 70s, predating the web by many years! If you look beyond the trend, you see ideas rooted in an effort to architect software well, and see MVC as just one design, one particular decomposition. The key idea is not MVC itself, but the principles behind it. Being able to place ideas into larger contexts shows mastery, in my view.

As a total curiosity, what's the biggest from-scratch codebase you've ever made? In my experience so far, things like MVC don't always make a whole lot of sense until:

1) You've got a code base large enough that they matter; 2) You wrote it initially without using something like MVC; 3) And then it grew into a ridiculous monstrosity, which you then cleaned up by breaking it into models, views, and controllers.

At that point, if it turns out well, you start to "know it in your heart" instead of just being a nebulous concept that you read in a blog/textbook/whatever somewhere.

>>I sometimes ask candidates what the letters "MVC" stand for. The successful response rate is around 50/50. I ask candidates to briefly explain the advantages of the Model View Controller pattern,

It's good you don't ask for a definition. Even the inventor of MVC thinks it's rarely been applied as intended [1] (see forward by Trygve Reenskaug).

[1] http://downloads.nakedobjects.net/resources/Pawson%20thesis....

This. I recently got asked a definition question. Oddly enough, one I researched prior: Polymorphism and Inheritance.

Short of quoting a single book's definition of them, word-by-word, you'll find hundreds of different definitions of those two terms online.

So, definitions are bad, rather ask for an "understanding" or "explanation" of the interviewee.

I'm surprised you would say that there "hundreds of different definitions" of polymorphism. Here's one of the top-ranked results for the term: http://stackoverflow.com/questions/1031273/what-is-polymorph...

If you look at the answers, aside from a few that are really poorly phrased, they all fundamentally say the same thing.

If I were interviewing someone and asking them about polymorphism, I would expect a definition which shows understanding and perhaps an example which explains to me that they really do understand it in concrete terms. (I'd probably give them bonus points for mentioning what the Greek words mean, but only because someone who knows that is likely to be someone I'd enjoy working with.)

Inheritance is certainly a dicier subject. I'm not 100% sure I'd ask about either polymorphism or inheritance in an interview nowadays (though I certainly have in the past), but if I were asking about the latter, I'd probably phrase it in the specific context of a particular language - or perhaps request a comparison of how inheritance differs between two languages. I guess, really, I'd actually want a discussion about inheritance versus composition - but, having asked for just that in the past, it was quite surprising to me (at the time) how many people had literally no idea what the term "composition" meant (despite, I'm sure, using it on a daily basis).

> (I'd probably give them bonus points for mentioning what the Greek words mean, but only because someone who knows that is likely to be someone I'd enjoy working with.)

I know what the Greek words mean, but it would never in a million years occur to me to mention it in an interview; I would have no way of knowing without being told that you would assign positive value to such.

I'm certain the vast majority of interviewers wouldn't care in the slightest if you demonstrated that knowledge; I just happen to have a degree in classics and thus a bias towards such things! (Not that knowing the meaning of two fairly common Greek roots really demonstrates much knowledge of classical languages, but there we are.)

Having performed a thousand interviews by now, I am awestruck by how poor the software development talent pool is.

My experience: exactly the opposite.

Steve works for a relatively unknown company making products for businesses, mostly using Java/C#, whose technical staff seems to be based in Atlanta, which is not a super attractive city for tech workers.

Your experience comes from matasano, a company where people get paid to be hack, a dream job for many techies, with offices in New York, Chicago, and Silicon Valley.

Is it surprising that your experience is exactly the opposite? I would imagine there's a huge self-selection factor at play here with regards to who applies to work for each company.

(None of this is to say Steve's company isn't awesome - I'm sure it is - just that recruiting is going to be much more difficult).

p.s. - I agree with you that tech interviews are terrible - I just don't think anyone has got it figured out yet and what worked or didn't work at you for you at matasano is not guaranteed to work or not work for the rest of the industry.

I would believe that if (a) the kinds of people we're looking for weren't so generally hard to find in the market and (b) we were tending to hire people with some kind of professional track record. Neither were true. If my pipeline of candidates was "people who would otherwise have gone to work for Dropbox", there'd be something more to this argument.

I seldom post lengthy comments on HN, and when doing so I tend to include qualifiers such as, "I know that the Bay Area is overrepresented among HN readers, and the preceding opinion is based on a perspective from elsewhere." ... "I know that 20-somethings are overrepresented among HN readers, and the preceding opinion is based on an older perspective." Etc.

Invariably, one of the top replies boils down to, "Hey! I'm a 20-something in the valley, and I see it completely different!".

Well, yeah.

I'm neither a 20-something nor in the valley. I'm (I think) older than you are, and in Chicago, which is even less of a technical hub than Atlanta.

Sorry, my remark wasn't referring to you exclusively, just noting a frequent trend. Fair enough.

There's another big factor at work.

If you read Thomas's recent post on Matasano's hiring process [1], they were looking to funnel people in. Steve's process seems pretty clearly designed to filter people out.

Compare the idea of sending would-be candidates books to study and the idea of an interview process that COMPLETELY EXPOSES candidates. One is trying to bring people in, the other is trying to drive them out.

Somehow tech hiring has often become an adversarial process. It shouldn't be shocking to us that approach makes it hard to find good people.

[1] http://sockpuppet.org/blog/2015/03/06/the-hiring-post/

I like this summary and plan on stealing it from you.

@tptacek, I'd be interested in your thoughts on how one could adapt your methods from Matasano to, say, a startup doing web dev or to an enterprise software company. I've done some thinking based on some of your writings and the idea that the best way to test someone for suitability to do a job is to have them do the job, and haven't really come up with any good ideas.

1. Take application you've already built.

2. Package it, with all of the assets and utilities needed to get it running with "vagrant up".

3. Carve out some feature/features from the application, and replace them with stub functionality.

4. Deliver the vagrant app and a functional spec to candidates. Have them implement the missing feature.

5. Devise a scoring rubric (unit test coverage, lines of code, algorithms used, safe/unsafe APIs, performance, whatever). Mechanically evaluate candidate submissions.

6. (Optional) Devise a 15-20 minute on-site interview component to verify that the candidate actually did the work. We didn't bother with this, and multiplied the size of our team (NCC is the largest software security firm in North America) and had 100% retention. But it's a big concern for some people.

I'm still not sure how giving employees a book to read and then testing them on it is a good process. If I'm going to spend a couple months of free time learning something, I'd pick something like Python or Android development or HTML5 app development, rather than learning a skill that will only be useful for 1-3 employers.

How can I know if an employer is worth 1-3 months of my free time until I meet my potential future coworkers? Demanding a candidate invest 1-3 months of free time before you meet them seems like an insult to me.

If your job uses a niche skill, why demand candidates be an expert already before you hire them?

I did not have a problem with passing on candidates who weren't not sold on learning our subspecialty.

I do have a problem with passing on candidates who were sold on our subspecialty, had an aptitude for it, but could not pass an interview on it "cold".

To me, that sounds like a convincing argument that your specialty is not worth learning. I'd rather work someplace that wants to invest in their employees, rather than expecting them to already be experts in some obscure techniques.

Are you hiring people who are brilliant, or people who are so desperate that they'll spend a couple months preparing for one interview?

Let's not flee to abstraction. Matasano/NCC does software security. We were willing to invest some time and money to bring people up to speed in software security and exploit development.

Not your cup of tea? Totally fine. Not everyone is interested in doing security.

I remember back when I was a teenager looking at colleges seeing a private tech college with a high acceptance rate and a public state college with a low acceptance rate. The Princeton Review had rated the private tech college as much more selective than the public state college. Teenage me was confused - how was that possible? - the state school had a significantly lower acceptance rate!

The reason is, of course, self-selection bias. On average, the people who applied for the tech college were just stronger students than the people who applied to the state college. This likely is because applicants to the tech college were more likely to be people whose primary goal was to peruse an academic discipline that interested them while the state school had a good number of students applying who wanted to party and attend football games. If you were to look at students who applied to BOTH schools, you would actually see a lower acceptance rate for the tech school than the state school.

I suspect that the same factor is at play here. I suspect that the average quality of a candidate who applies for a software security engineering position is much higher than the average quality of a candidate that applies for a enterprise software development position. The software security engineering position is an esoteric position that is more likely to attract applicants who are enthusiasts or at least very interested in the field. The enterprise software development position is more likely to attract anyone with a tech background looking to phone it in and collect a paycheck.

If this is true, while both Thomas and Steve have a difficult time hiring developers, they have a difficult time for fundamentally different reasons. Thomas is like the tech school - his difficulty is with getting candidates to apply. Steve is like the state school - his problem is with separating qualified candidates from "phone it in and collect a paycheck" candidates.

This is why I am skeptical of Thomas thinking he's got a much better way of hiring figured out. Maybe he does for companies that focus on security - judging by his Linkedin profile, he's only worked for companies focused on security. This is certainly valuable, but he has no idea what it is like for Steve.

By the way, I think many of us have seen this self-selection bias at play even within our own companies. Post a primarily Java job and a primarily Scala job and then try to tell me that the same principles apply for filling both positions.

Why is Atlanta not attractive to tech workers? Georgia Tech has a phenomenal CS program, and we've got a pretty large and diverse startup population.

I'm not sure if what you're saying isn't true, or if I'm just biased and not able to see Atlanta from an outsider's perspective.

Atlanta is a pleasant enough city. I'm from there originally, though I moved to L.A. about 15 years ago.

I left because I wanted to live in a big city, not in suburbia. Atlanta has some of the plusses of a city, but they're relatively limited compared to LA, SF, NY, DC. It's more a collection of suburbs than a city.

If suburban life is what you want, Atlanta is fine for that. Honestly, most of Silicon Valley is suburbs, outside of SF itself. And they aren't as appealing as Atlanta suburbs, with many more trees and a lot more space. If that's what someone wants, Atlanta is a better choice than pretty much anywhere in California.

One of the best things about California, particularly in LA and the Bay Area, is that there are more people living their lives according to their own unique desires than anywhere else I've ever been. Having "odd" choices about your lifestyle or having crazy dreams and aspirations is the norm here, and that's something I want around me. It's a plus for me to be surrounded by that kind of thing. There's some of that in Atlanta, but it's more localized to certain areas. Here it's everywhere, just how things are.

You should come back sometime! The Beltline has really reinvigorated a lot of intown areas. Midtown is booming, as is the Old Fourth Ward and lots of other intown neighborhoods.

But I do get your point about "odd" lifestyles. We're certainly getting weirder here, but I don't think it can yet compare to what you see in NYC or the Bay Area. Still, there's a lot to like here, and cost of living alone is a great argument for Atlanta.

Heh, I kinda let that dig slide, because the rest of the comment was very well thought out and on-point. However, of course I disagreed.

Atlanta is by far the technology hub of the southeastern United States. It has no competition to the west until you reach Houston (and even that is mostly specific to the energy industry). It faces only mild competition from the Research Triangle region of North Carolina, and beyond that it's a technology DMZ all the way to Chicago or the northeast.

It's home to Georgia Tech, perennially one of the top 5 engineering schools in the nation, and pumps out thousands of new programmers per year in addition to the thousands who migrate here. It has a highly diverse business base, not tied to any one predominant industry, and its Buckhead and Midtown districts have one of the most thriving startup scenes that you'll find outside of SV or NY.

Of course, if a Georgia Tech grad is in the uppermost percentiles, then odds are he or she would be enticed by Google, or a west coast startup. I'm certainly not saying that Atlanta is in the same league as San Francisco, or even New York (at least not its finance/quant opportunities). But my earlier point was that these environments are extreme outliers, and virtually no other city is comparable to that either.

Hey now, we're doing some pretty kick-ass things over here in Augusta. We may not be as big as Atlanta (yet), but we're working our butts off. You should come down some time and take a look at what we're doing at http://theclubhou.se

Sorry for not phrasing that better, I definitely would portray the scenario in the same way that you do, but I didn't want to distract from the main point. I probably should have phrased it "not globally attractive" instead of the negatively toned "not super attractive" but even that probably isn't nuanced enough.

I'm not entirely convinced that the CS program is phenomenal at tech. I wish it was, since it's my alma mater (ECE/PHYS 08). I've tried to hire people out of there for ebaynyc and now shopspring.com, but they don't seem nearly as prepared as the northeast college graduates. Tech seems way too old school. They only teach java (I'm pretty sure they don't even offer a functional programming class!), and all of the candidates that I talked to had no experience even building a simple web app.

http://www.paulgraham.com/avg.html

"After a couple years of this I could tell which companies to worry about and which not to. The more of an IT flavor the job descriptions had, the less dangerous the company was. The safest kind were the ones that wanted Oracle experience. You never had to worry about those. You were also safe if they said they wanted C++ or Java developers. If they wanted Perl or Python programmers, that would be a bit frightening-- that's starting to sound like a company where the technical side, at least, is run by real hackers. If I had ever seen a job posting looking for Lisp hackers, I would have been really worried."

For what it's worth, I think these are great interview questions. You can quickly pick up how much experience a candidate has in a particular area by how in-depth their thoughts on the topic are. Asking a candidate to compare two languages, data structures or technologies in general usually yields good results too.

I think what some of the replies to your post are forgetting is there are no right or wrong answers to these questions. The questions are about gauging the experience of the candidate so you can make an appropriate offer if you wanted to hire them. It's probably not a huge deal if you couldn't describe MVC but you gave an in-depth answer to the OOP question for example. It is a problem however if several questions reveal there are major holes in your knowledge and experience though.

It's interesting that so few people can define, or fully explain, MVC.

I wonder how the success rate would change if you defined "model," "view" and "controller," and asked why developers might choose to organize projects in such a way.

My guess is that most working developers would get the concept right away and might even be able to relate it to other named models we've seen--we might just all exist in a web bubble where MVC is trendy because of the levels of abstraction that exist in our common toolsets.

Even Cobol's ancient "divisions" seem to have a similar organizational structure (http://en.wikipedia.org/wiki/COBOL#Features) and so, I'm sure, did a lot of old-school, green screen terminal apps.

Our computers are generally fast enough and our tools good enough for what we're asking them to do that we can often just define data models and user views fairly abstractly--index this, make that an input box rounded corners and indented text--and that we can often avoid delving into the low-level details of how data's stored or pixels laid out on the screen. I'm sure it would be possible to describe an operating system, or a video game engine, or a DBMS in terms of which sections of code model data, handle the meat-and-potatoes "business logic" and communicate with the outside world, but you'd probably need deeper layers of abstraction to actually plan how code would be organized.

We also have enough fast memory that we can basically pass data from one level to another without issue and don't have to deal with abstraction-breaking optimizations like processing data while a disk head or spinning drum is in the right place or before an old video game system's tiny RAM chip is full.

But I think programmers who worked with those kinds of systems could quickly jump into web and app development, and if they couldn't, it's probably not because they don't know what MVC stands for (although it's possible it's a useful proxy).

From the above article:

"Why do we keep doing whiteboard interviews, even though they tell us little about how good the applicant is at actual software development?"

"We bring in Java and C# candidates who have been working with their respective language for 10 or 15 years, and they get COMPLETELY EXPOSED during the face to face round when asked a series of basic certification exam style questions. Nothing tricky, just core fundamentals."

Let me guess, they work for a "big box software factory/consultancy".

They don't program, they just click buttons on Eclipse.

That should be your filter.

What this process seems to be tuned for excluding potential candidates. And if the end result of many many interviews is the idea that the talent pool outside silicon valley is poor, perhaps that reflects the focus of the approach.

I suspect that it is very likely that this process ends up excluding very high-level producers.

Interesting report! Did you weight in this - the bad developers are most likely to interview many times, so they appear often to you? Talents either get advised into a new job or stay where they are, because many shops know they are valuable, I'd think.

> Did you weight in this - the bad developers are most likely to interview many times..

Which implies, almost by definition, that they will become much better at the process over time and eventually become very good at gaming it, therefore making it even less reliable.

i like the style/approach of your questions--a nice optimum that minimizes false positives and false negatives.

and i agree with you that threshold competence in the general applicant pool is so low that some form of technical screening is absolutely essential. My problem is how best to do it. For one thing, it has taken me far too long to recognize that the pool of interview questions and plausible answers are widely available and apparently studied by many applicants prior to technical onsite interviews. Their answers are quickly given and so polished that we bring them on, then fairly soon find that answers to those interview questions is pretty much all they know.

Strongly but respectfully disagree, see my comment which is sitting at the bottom because it ruffled some feathers.

Reply on news.ycombinator.com