Hacker News Reader

Comment by dataflow | Hacker News Reader

For a junior-ish software engineer level (say, 1-3 years out of college) who claims to have experience in C, this seems like a fine question, but for a senior software engineer (which I think is what he was interviewing for) it doesn't seem all that great, except maybe as a first-round filter [1]? Not that it's terrible, and I guess if the candidate fails it then it certainly tells you a lot—but if they complete it successfully then it doesn't really seem to tell you much about their software engineering skills or depth of understanding. It's better than some silly challenges companies give, but there's so much room to ask more probing questions or provide more challenging tasks to help you gauge where the upper limits of a candidate's abilities are, instead of just the lower limits.

[1] Edit: Looks like it indeed was a filter question, as I suspected: https://news.ycombinator.com/item?id=31065819

View on HN

Sorry, are there people who do an interview like this and then just treat the result as a boolean pass/fail?

I mostly do pair-programming interviews. When I do it async and have them write something, we'll at least discuss the code together. So either way, I'm way less interested in whether they finished than how they did it and how they think about it.

And maybe I'm just weird, but I'm not super interested in the "upper limits of a candidate's abilities". If they're using all their cleverness just to write something, neither they nor anybody else will be able to maintain it. Software is a team sport, so I want people who are good collaborators. Indeed, the "upper limits" thing makes me think of Aphyr's series of fictional technical interviews. (Start with the last one and work your way forward.) https://aphyr.com/tags/interviews

Why do you think that's a bad question for a senior engineer? I think it's great. This is much closer to the actual work that software engineers have to do!

That's what I do most of the time when I'm working with code, it's always an existing system that you need to add a feature to without breaking something else.

What do you think is a good question for a "senior software engineer"?

Please convince this director to allow this very safe change to go to prod.

Not ... a ... chance.

You go through the normal change process like everybody else.

There is no such thing as a "safe" change.

The only exception to this is when stuff is on fire, and, practically by definition at that point, there is no such thing as a "safe" change.

Let me rephrase.

Given a Tech document that addresses the risk of change, the rollback plan, the security implications etc, and a suite of metrics demonstrating load and failure conditions with appropriate alerting. And of course the PR with appropriate test suite, shown to work in a test environment in concert with the rest of the ecosystem, that stuff is just table sakes. convince a director to approve the change. Ideally a director that doesn't report to the same vp as your org.

Because it doesn't tell you the upper limits of their abilities, only the lower limits. You hardly get any insight into their ability to design a new system, or into how they might deal with problems that require more than 2 seconds of straightforward planning, etc. Again, for a senior engineer, it's not really a bad question, just a very mediocre warm-up/filter question. It's just too easy to copy-paste or modify a few functions and get the task over with, and the result ends up giving you very little information other than "this senior engineer can, in fact, complete a very straightforward task".

Good questions aren't easy to come up with, I'm not an expert in interviewing by any means, but for a senior engineer you'd want to ask questions that start straightforward like this, but become progressively less straightforward (and maybe one that's a more subtle design issue) so you can get a better feel for their skills as you work with them.

It's only one question though. You probably shouldn't have a question that simultaneously evaluates many different things - the different signals would be mixed and harder to interpret. This seems like a good question to add a simple feature to an existing product and also seems very much like the work of a software developer - certainly much more like actual work than "Invert this binary tree" or whatever.

No one performs at the upper limit of their abilities on anything like a regular basis and it's basically a crapshoot whether your question and dialogue is actually able to test their limits. Most people, myself included, have a whole litany of things they need to perform well and doing it on-demand in a fundamentally stressful situation like an interview with a person you don't know is unlikely at best.

There's also a tendency for interviewer to choose questions from their personal expertise rather than something appropriate to the person or role.

Okay, I guess I have been in a position to interview senior software engineers who can "design new systems" all day on a whiteboard with artificial "interview" constraints but who could never actually solve this problem. I think this is a better question for senior engineers than junior ones. You seem to be assuming that a senior candidate who could blag their way through a "hard" design question in an hour could ALSO solve this problem and I don't know where you've worked but in my experience that is not at all true. In my experience, senior engineers often can't code much at all because they have worked at places with a ton of people writing code where code just "happens" and they are good at all the other processes that are code adjacent but "I haven't written much code recently". You better test that they can write real code and not leetcode.

I'm a senior engineer and I do a lot of design and architecture and code review every day but I would probably fail most generic leetcode tests and I think I would pass this one, so I feel kind of personally validated by that, so excuse the passionate response. :)

That’s, in my opinion, a good thing! It helps verify the engineer can competently code—in a real codebase, which is much more valuable (again IMO) than coding in a vacuum. Eliminating that question, which is often cited as the motivation for these sorts of challenges, is a cue that the candidate is qualified for further interviewing. That being out of the way leaves room to evaluate a lot of other things by:

- enquiring about the candidate’s thoughts about the problem and their approach to solving it, on which (again IMO) this candidate would really shine; you’d likely get a much less thoughtful or thorough account from someone less experienced or talented

- ask about concerns which would be typical in real world work but often get glossed over in interviews, like how they’d approach testing and SCM and review, or even just things they’d do differently outside the interview process

- get more conversational and learn more about the actual value the candidate would potentially bring to your team/org, starting this with some greater confidence, trust and familiarity on both ends

- dive deeper on technical validation if anything in the above feels questionable

Those sorts of conversations convey much more meaningful information than any coding challenge that isn’t very focused on very specific prerequisites for the domain/role.

The ergonomics suck. Whose laptop do you use? I’d want my own. Oh they want me to use sublime? Maybe I prefer Vim with all of my macros. I’m a professional and I rely on my macros and tools. Maybe it’s a PC and I only develop on linux.

It introduces an entire problem space where you’re evaluating something that’s neither important to you or your company.

Way too many moving parts.

Coding challenges in general, if they’re part of the hiring process, need to be much more accommodating yes. Even if they reassure you they’re evaluating “how you think” etc, this kind of thing is an anxiety amplifier for many (including me).

I haven’t interviewed on site with a coding challenge for nearly a decade, but last time I did I literally brought my own laptop and asked permission to use it (which was always declined). If I were interviewing today I’d literally walk out if that request wasn’t at least considered, and probably even still if it was declined without a really good explanation.

Filtering out prima donnas who claim they need the exact right dev environment to write out a function is another feature of this question. They’re never productive developers.

This is ableist whether you realize it or not. People tailor their dev environments to be functional and effective for them not just to cater to their whims, many of us actually need certain accommodations to be able to work and to be able to do so at our level of effectiveness.

I’m quite productive thank you. But if you stand me up in front of a whiteboard or sit me in front of an unfamiliar code editor, I’m gonna produce nothing of value. If you’re filtering me out for that: thank you, I don’t want to work for/with you either

That makes it seem like a great test, right? It detects environments with mutual lack-of-fit. You don’t want to work there. They don’t what you to work there. Outcome from experiment: both realize this.

As far as interviews go, it’s pretty good.

This is ableist too. You might be relieved to eliminate candidates who could be a great fit with some accommodation. You’re eliminating people unnecessarily, and it harms both you and the candidates.

As the originator of the question addressed elsewhere in these comments, they gave advance notice to the candidates and worked with them to provide an alternative setup as needed.

https://news.ycombinator.com/item?id=31065861

The “accommodation” of having vim macros set up.

Yes, hiring is supposed to be “ableist.” I don’t want to work with or hire mentally limited people who couldn’t write a function in Notepad.

The accommodation is allowing people to use tools which allow them to be effective. The ableism is finding people “mentally limited” because they may need affordances (often but not always) physical to contribute.

Hiring is supposed to be qualifying. Qualified people using completely irrelevant-to-the-task assistive tooling is none of your fucking business. Unless the job you’re hiring for requires coding in Notepad, your expectation that interviewers do is discriminating probably illegally but definitely immorally. And probably also limiting your hiring pool in ways you don’t want.

Not a problem for me in this particular aspect, I’m never going to seek employment from you. But disadvantaging people who need accessibility tools is shitty.

I see that as being largely a non–issue. Either you use your own computer and share your screen with the interviewer, or you use a pre–prepared machine. An ideal interviewer would have three laptops ready to go, one each for Windows, OSX, and Linux, and would ask you which you prefer to use. Or three types of VMs to remote into, or whatever. (The upfront cost of that is miniscule if you’re asking a bunch of candidates this question over a few years.) If the interviewer can’t provide that much then they have failed your interview of them, and if you can’t be productive under those conditions then you will likewise fail. This is beneficial for both parties.

I think that this is actually pretty obvious to most people. Arthur only mentions this to say that getting a working build environment wasn’t part of the test.

You'd be mostly reading code and writing <100 lines of C, much of which would be copypasta. I'm not convinced having the familiar editor would be that big of a deal here.

reading code (or more correctly, navigating an unfamiliar codebase) is difficult in a setup you're not used to.

I myself rely heavily on IDE click thrus and reference searches.

Purely using grep sucks, and a plain text editor's search function might also not function as smoothly. This all leads to friction.

This type of interview requires the candidate's own dev setup. If they could prepare ahead of time (e.g., give the repo to download and setup the IDE etc), then it would be fine.

This is an issue with many coding problems. I'm not sure if there is a term for this is detection / estimation theory, but it provides what I'm going to call asymmetric information. In this particular case, negatives are true, but all positives must be presumed false. If you make a problem too hard, you're faced with asymmetry in the other direction.

I think the usual solution to that is to follow it up with "what if you wanted to {introduce new/harder requirement}?" and ask them to explain & make as much of the changes as time allows. To give an example, if you were to initially ask someone to implement depth-first traversal recursively, you might follow that up with asking them why you might end up needing to do that iteratively, then you might have them do that iteratively for pre-order depth-first traversal (easier), then follow that up with post-order iterative traversal (harder), then you might ask whether/why it might perform poorly (e.g., re-visited nodes blowing up exponentially), then ask them how they might mitigate it (keeping track of visited nodes), then ask them how they might mitigate the space requirement (lots of possibilities), then ask them how they might handle parallelizing it for some particular problem... I can keep going, but the point is you can keep something like this going until they start to struggle, and that at least tells you the upper limits of their abilities on at least that axis. And you can pace it by e.g. skipping stages as you get a better sense of their skills.

I wish I could downvote this a million times. Do you literally have to do that at work and do you actually talk this kind of wine tasting algorithm bullshit in interviews? This has nothing to do with software engineering as far as building software for a business or any kind of meaningful work. Do you even hire people? This has to be a troll account. I enjoy code golf on a casual slack channel, but you can send me every engineer who fails that question in your interviews and I bet I'd find a few good ones.

That's a strange reaction.. Are you always able to answer any quesion that comes up at work? I know I don't. And so seeing people's reaction to qestion that is too hard for them is a valuable signal.

As long as one tells candidate in advance: "I am going to be asking a mix of questions, some easy some hard. I don't expect you to be able to answer every question", this sounds like a good question.

Reply on news.ycombinator.com