Hacker News Reader

Not to say that I disagree, but it's worth noting that your criteria pushes the onus of an interview away from purely objective metrics into territory that overlaps with communication and rapport.

Again, I don't necessarily disagree with that, but I think it's always useful to consider how much of my interview criteria can be reduced to "do I vibe with the candidate" and make sure there's more to my interview than just that (there IS a place for it though. The world isn't an objective metric)

There is no objective way to assess a candidate on an interview just as there is no objective way to specify and measure what a good employee is.

What I try to do is I try to learn specific things about the candidate not because they can be measured easily or objectively but rather because I think they are important.

I am not "vibing" with the candidate (although this is definitely a plus!) I have a checklist of things and I try to make sure, by the end of the interview, that I can run through my checklist and have an answer for each point on it.

"Do we get along" is, in many cases, more important than hiring the best programmer possible. Once a team exists, the team dynamic is much more important than any one individual's abilities. (And, if there is a need to work with someone difficult, then a lot depends on the leader's ability to isolate and direct the difficult person.)

Relying too much on "do we vibe" is how you get an EEOC lawsuit, and rightfully so.

> Can I have productive discussion with the candidate about designing his solution? Can the candidate converse in a clear and precise language?

> Can the candidate recognize and make use of an advice?

I must admit that following those two criteria helped me make the most effective hires.

I used to ask a question that was mildly confusing, but only to the kind of people I was trying to screen out. The people who I was trying to hire would usually get the question right away.

Basically, in one of my questions I had developers use a lesser-known threading API in C#. The people I hired either got the answer right away, or could quickly adapt with my advice and answer the question.

Another interesting anecdote: I used to ask candidates to discuss tradeoffs of the XML APIs built into .Net. The point was to discuss tradeoffs and to demonstrate more than superficial knowledge of serialization. (Most .Net developers have some exposure to its XML APIs at some point.) One candidate quickly hopped into a JSON is better than XML tirade. I ended the interview quickly.

That candidate probably tells a version of this story where they dodged a bullet by avoiding a job where they would spend all their time serialising XML.

Please tell me there was more to that last story? Having someone come in and have a strong opinion on a topic, and have the gumption to state so in an interview, is exactly what I would look for.

Did you poke them for more? Did you ask how JSON compares to XML? If they only answers "Well, I've never used XML, but I was trained on JSON and won't ever use XML, I don't know any of the downsides of XML", then you were correct to pass on them. But if they responded "Because of reasons X, Y, and Z, JSON is better than XML, and all XML interfaces are equivalent in this way, so I can't really answer your question because while I know enough about XML to know that it's bad, I can't tell you exactly the difference between different interfaces, as I've avoided them." That would be an answer I would accept.

> got the answer right away

This is the major critique of asking these interview questions. There's a small subset of them, so candidates train on exactly them, and only know how to do these, and when you hire them they won't be able to actually code anything. Using ML terms, they're 'overfit'. Getting it right away is actually a bad thing, but then you get candidates who act like they are going through the process just to look like they are learning on the spot... it's a tough problem.

Regarding JSON vs XML, I just pointed out that this is an interview question about discussing engineering trade-offs. The purpose of the question was to ensure that the candidate could understanding and evaluate trade-offs without making snap judgements.

The candidate was incapable of doing so.

For the threading question, the reason why I chose an unusual API was for your exact reason: You'll never see it on a list of interview study questions.

>* Can the candidate predict what his code is going to do before he executes it? Repetitively executing code to see if it works may seem innocuous, but really is a huge defect in candidate's process which makes it really difficult for them to write anything non trivial to be reliable.

I'm going to disagree here. Testing your code in small chunks can not only catch small errors before they propagate into something larger, but also give you insight into whether you are going down the wrong path.

If you write a small piece of code and can't tell if it is going to work or not you don't really understand what it does.

We are not talking using external dependencies or frameworks. I am talking simple code that you just wrote in entirety, maybe couple of loops and conditionals.

If you can't predict what that small piece of code does, exactly, you are thoroughly unable to write reliable code, by definition.

Reliable code == code that does exactly what you intend it to do, nothing less, nothing more.

If you can't predict what your code does you are writing buggy code. Experimentation can uncover some of the bugs, but if you start with buggy code the experimentation will never remove all bugs -- only the ones that are easy to detect.

You have a good point overall. However, can you truly reduce the definition of reliable code to something that simple? I think there are a lot more factors that determine if code is reliable or not.

I actually think that this is good definition of reliable code -- product of an individual developer's process.

It does not say whether it was specified well, but I think an application can reliably execute wrong process. There is no contradiction.

I don't want to say that making sure that your code does exactly what you intend it to do is at all simple.

As your application grows, as you start incorporating dependencies, external systems, code that came from others, things that you do not know very well -- writing code that does exactly what you intend it to becomes very difficult or even impossible.

But on the lowest level, when we are talking couple of variables, conditionals, loops, maybe small data structure -- I would expect a developer to be able to write it to do what is exactly intended.

Because if you have that ability and use it for every piece of code that you write, the overall result (even if it does not guarantee reliability) will be better than a person that cannot do it.

In my experience (and it depends a lot on our HR and candidates that are being sent my way) only one in about 10 candidates (for senior Java dev position) can do that.

I think different companies view this differently. I recently did an interview with a social media company (not Facebook, one of the other ones) where I wrote up a solution to the problem that "just worked" in one go, as did my extensions. After the interview, when I asked for feedback, the interviewer told me that I should have been running and testing my code as we went along. However, I've also heard in mentoring sessions from engineers at other companies that the goal in interviews is to write code that works the first time without having to test along the way.

In general, I prefer the approach of writing out the entire solution at one go, and trusting that bugs will be easy to fix at the end if the code is clean enough to understand what is happening at every step. This is in part because there usually isn't enough time in interviews for running tests at every crux moment. However, I think our current virtual environment may make it easier to test for (and to utilize) these types of programming best practices, since instead of writing code on the whiteboard, candidates and companies can use one of the interview interfaces that allows running code as you write it.

> After the interview, when I asked for feedback, the interviewer told me that I should have been running and testing my code as we went along.

Have they specified that you should be testing your code? If they haven't specified then it is wrong to expect some kind of arbitrary outcome.

People write code in many different ways, some do it very chaotically, some have very organized process.

I try to figure out which of these behaviors are fundamental to writing good code and which are not.

For example, I believe a habit of writing readable code IS fundamental to writing good code. If you don't have a habit of writing readable code, on average your code is going to be worse than if you had.

On the other hand, I believe writing unit tests IS NOT fundamental. Unit tests is one possible way of working with the code, but people were writing reliable code before unit testing became popular. Unit testing takes time and effort that can be allocated differently.

And so I try to look (I can't call it measuring because it is subjective) only at the behaviors that I believe are fundamental and dismiss others.

In the end, if you are smart person that can program well, you will be able to learn those other behaviors. But if you are not smart or you lack certain fundamental abilities, it is going to be much more difficult to fix it.

I do an interview in which we do TDD on a problem together. So we are frequently running tests during the interview. However, one of the things that's expected from the candidate is that they will be able to predict what happens when we run tests. Will the tests pass? Will an exception be thrown? How will it fail?

Maybe as do often the difference in this discussion is a quantitative one not one in principle.

Running tests frequently is not a problem. Not knowing what the code will do is.

> If you write a small piece of code and can't tell if it is going to work or not you don't really understand what it does.

And if they don't understand it, they're going to B*&^% and Moan about having to fix the bugs once they come back.

The good news is their bitching and moaning won't last too terribly long. The bad news is that's because it will become clear they can't fix the bug, and someone else will end up doing it.

Testing code as you go along is not experimentation. It saves time by allowing you to verify your assumptions are correct in a controlled way.

You can sit there and plan all you want until you think you understand it, but you never actually know until you run the code. Being a good programmer is all about simplifying complexity. Testing your code as you go along to verify your assumptions is one more way to do this.

It doesn't matter how you call it. If you know the code works then there is no benefit of running it. If you need to execute it it is only because you don't know if it works correctly.

Sometimes this is fine. I do this when I execute external services because I may not be sure how they work exactly and I want to correct mistakes before I do too much design resulting from my flawed knowledge.

But when it comes to your code you should be able to tell if it works correctly. If you can't there is couple of ways to deal with the problem that I adopted over the years:

-- techniques that make it easier to ensure code works correctly are frequently also improving it in other ways. For example, making code simpler is important not just for reliability.

-- specifying your internal module boundaries and APIs is critical for anything non trivial. Abstracting parts of the problem is enabling you to solve each part of the problem separately. You can't keep 10k lines of code in your mind and tell if it works correctly, but maybe you can split it into multiple modules and make sure each works correctly, separately.

-- when I have written a module that I am just not sure works correctly, I refactor it so that it is easier to tell if it does what it should be doing.

-- make it a habit to write code that is what I call "obviously correct". One way to do it is to make sure it can never get into wrong state.

> If you know the code works then there is no benefit of running it. If you need to execute it it is only because you don't know if it works correctly

People make mistakes. You have to run the code. Do you submit PR's without running code? If not, then we're agreeing. Making code simple enough to reason about without executing is a great goal, but you cannot replace running code.

> You can't keep 10k lines of code in your mind and tell if it works correctly, but maybe you can split it into multiple modules and make sure each works correctly, separately.

But ultimately they need to work together, and that's the most important part. Again, you need to run the code here.

> when I have written a module that I am just not sure works correctly, I refactor it so that it is easier to tell if it does what it should be doing.

This is a good goal, though again, I hope you run the code. You don't know what you don't know.

Btw I’ve seen the “advice” part faceplant terribly, where the interviewer is pompously trying to “unstick” the interviewee but it turns out the interviewee already understands the problem in a deeper sense than the interviewer and the interviewer is effectively ignorantly and annoyingly trying to derail the interviewee’s thought process without recognizing what they’re doing.

In general I’m deeply skeptical of interviewers coming to the table with their framework full of assumptions for how to evaluate the interviewee when in fact they have a solid chance of being the less smart person in that room and can’t get a readout on the smarter person.

Nice. The things you focus on seem like the skills needed for actual work, as opposed to interviewing.