Bug squash: An underrated interview question | Hacker News Reader

Bug squash: An underrated interview question

blog.jez.io

434 pointsjez271 comments

I like this approach far, far more than coding tests!

> It’s fun. It’s fun in the same way an escape room is fun. It’s fun because of the dopamine you get when the test suite blinks green.

Keep in mind that for lots of people in a job interview setting (even excellent candidates) this is not fun, this is stressful. It may be fun on the job or at home, but the stress of a job interview both eliminates any "fun" aspect and makes even simple tasks more difficult. Just mentioning that to encourage some amount of empathy and understanding for the candidate.

The discomfort and stress is a given in any interview. What I think is nice about this is it doesn’t require you to emanate fully-formed, nicely designed code from your brain on the spot. Instead you just have to navigate the codebase, read it and understand it.

It’s a better demonstration of knowing how software works imo. But I am biased because this sort of thing is one of my strengths.

This approach then selects those who are relaxed. The candidates that have five more interviews at advanced stages, and likely to receive several offers, as opposed to candidates who badly need a job.

The former kind of candidate may be more desirable for the employer :-/

Depends very much on the company. Fishing in the unemployed-and-desperate end of the pool is a real tactic. The odds are good but the goods are odd, as they say.

What's the goal? Increase headcount at any cost?

Find the occasional spark of brilliance someplace where others aren't looking?

That’s it. Not everyone lives a life that leads to a perfect resume.

And for that matter, some people do live that kind of live but still fall through the cracks. Not everyone is lucky enough to graduate during a strong economy, apply to exactly the right jobs at the right time, etc. There's a lot of wasted talent out there that many businesses have little interest in seeking out.

This! One of the best developers I've worked with used to be a carpenter who had a kid with Down syndrome. He taught himself programming in his spare time. Smart, extremely driven and knew how good we all had it.

Finding talent

Improve the ratio of compensation to workload in the hiring employer's favor.

The reason someone is unemployed or is desperate might correlate (inversely) with their ability to produce high workload.

Or it might just be a totally unrelated layoff that axed a whole team / org / whatever.

Might correlate with ability. Which - let's be honest - gets rounded down to "is caused by inability", just in the next step, in the name of Efficient Hiring Process.

In my experience, it's just to hire the people that got passed over by other companies but still have some skill. You dont have to pay as much and they might also be more likely to stick around longer if they have trouble interviewing at other places.

I'm stuck in my job and I get paid less than people with lower tenure and credentials. Due to a disability, I don't have the social/political skills to get promoted in my org nor feel comfortable job hopping. It's funny though because every team I've been on has had managers or TLs telling me things that equate to 'you should be at the next level' such as "if you were preforming at this same level on another team you would probably be promoted". I had an interview internally recently and was told that I "found more errors than was expected" for my level.

So yeah, get disabled people like me so we are less mobile, work above our level for the technical work, cost less, and can't get promoted easily.

There are lots of good people who are unemployed through not fault of their own. The goal is (as always) to find good candidates. Because many companies have a weird aversion to those not already working, the pool will be stronger by including those with other job histories because it doesn't really indicate who they'll be as an employee. There are great candidates there.

It's just cargo cult hiring.

Hiring talent at a steep discount.

Even if I have no other offers I’m pretty good at maintaining a calm demeanor in an interview. I think it’s served me well over the years.

I've completely flubbed questions but thinking out loud in a personable way nets a lot of bonus points

Relaxation during an interview can also be a function of practice. Obviously when you're squeezed its more difficult to get that practice in, but if you can it can help tremendously.

for me cracking the coding interview is stressful

i prefer the bug squash question. i have 20 years of experience. i will probably do well.

a fresh grad might fail. do i care? no. leetcode has benefited fresh grads and is being used to discriminate against those with anxiety and other disorders and those who are older.

i think it is refreshing to try a different interview method, one that benefits those with experience.

I administered bug squash questions a few dozen times. Experience definitely trumped the smartest, most prepared young programmers, because no matter how much they studied, they hadn't really faced a real test suite in a large codebase. We had to rate them on a different scale, because otherwise very few of them passed.

why do young programmers need to pass? nobody gives a shit whether experienced people pass leetcode so give me one good reason why you should rate young programmers differently.

A) I, and a lot of other people, do give a shit whether experienced candidates pass leetcode, and try not to give leetcode interviews unless forced to by HR fiat

B) As an industry and as a society, it's not a good thing to refuse to hire new grads, since that leads to the kind of labour shortages we see in e.g. medicine and pulls up one of the best ladders for wealth mobility. Tech therefore needs some kind of funnel to evaluate candidates with less experience. A test that almost none of them can pass is not very useful for evaluation purposes

it’s not a good thing to refuse to hire older people and most older people do not pass leetcode without significant grinding

Yeah my first response to 99% of these kind of interview questions is why? I look at ROI and how working code impacts budget and investment. It pains me to say it but the biggest companies out there have horrible code in COBOL that frankly doesn’t make sense to change it over. Similarly bug squashing usually is how did it pass QA/static analysis/dev ops.

I don't think replacing an interview that fails older people that instead fails younger people is a good solution.

I thought the point of affirmative action was that being unfair is ok if it is to compensate for a prior unfairness.

We don’t typically do affirmative action by picking interview questions that make things harder for some candidates as opposed to others, though. We consider some questions to be easier for, say, men to answer (“tell me about a time you were asked to demonstrate leadership”) but we seek to remove that bias rather than asking questions that solely focus on things that can only be answered by those who experienced bias (e.g. “tell me about a time you experienced racism that affected your work” is not likely to be an interview question).

What do you think is a good solution?

Number one: there is research on how to interview find it and read it. All comments I've seen here give no indication that the commenter is even aware of it. I'm aware the research exists, but I have not read it (my company trained me in interviewing, and the class mentioned research but I haven't checked to see if it is the latest research that I'm trained in)

If the above doesn't answer your question, then different questions for different levels. You should know if you are hiring a junior, mid, or senior engineer. (you may have different levels), and ask them different questions.

> If the above doesn't answer your question

You misinterpreted my question, because I was asking what saagarjha thinks, not what anyone else thinks.

one thing I found interesting is the observer or hawthorne effect. in short, people perform differently when observed doing a task. this means that simply by observing people while they are coding, you might make them perform differently, and come to a conclusion they are not able to code, while if you did not observe them, they might have done well.

the reasoning is that it's ok to pass on excellent candidates. as a business owner I find that a real problem. I would prefer the best, not just people who had time to grind leetcode and rapidly crank out solutions to standard problems you can find online. i think it would be better if interviews erred on the side of caution and preferred to let people through especially if they have experience. you can always fire them if they don't perform. there is no job security in the US anyway.

It is my experience that caution during the hiring process means skipping on possibly excellent people to accidentally prevent hiring a dud. There are plenty of both on the job market most of the time.

In practice firing people is hard at most companies, even if there are no laws protecting people in an individual situation. In principle it can be easy but generally a manager who is in a position to evaluate performance doesn't also usually have unilateral authority to fire someone. And a technical manager telling someone that this person can't code or is disruptive on a team often can't communicate that to the managers with firing authority. This can be out of hesitance, budget concerns, or legitimate communication skills on either parties count.

So the hiring practices that most of the companies I have worked at tend to skip on possibly excellent candidates in preference of definitely excellent or at least definitely decent candidates. This is mostly about risk aversion then gain maximization.

> This is mostly about risk aversion then gain maximization.

I think you meant to say organizational dysfunction.

The inability to fire someone obviously incompetent is not risk aversion, it's risk accumulation.

I've seen very few people obviously incompetent. As such it takes a while to prove they are incompetent. They do write working code and get it through review. Often the only clear sign is nobody likes working with them - only after you get rid of them do you have concrete evidence that they weren't contributing (that is the team got as much done now since they no longer were stopping their own work to help the incompetent person on things that it is never clear if should have been figured out alone)

How does your comment relate to the topic here, job interviews?

squeaky said that companies are risk averse in their hiring process because it's hard to fire people. I said that companies are not risk averse if they can't fire incompetent people.

You responded that it takes time to determine whether someone is incompetent. Ok, but that just shows job interviews are ineffective at weeding out incompetent candidates, so there's still no justification for the so-called "risk aversion" of the interviews.

I guess I'm not sure why you responded to me rather than to squeaky.

No u ;)

Just Sqeaky.

I think the relationship between hiring and firing and interviewing is a little more subtle than a strict if/then statement.

It can be impossible to know up front if it will be easy to fire someone. Some people can hide incompetence of certain kinds for months or even years. Maybe the person that needs to be fired is competent but just toxic. Maybe the person rides right up to the line of the rules and makes an otherwise functional organization seem dysfunctional. Interviews might not be able to tell you if someone is excellent but some terrible people will absolutely reveal their cards. Most of the time there is an unlimited pool of applicants so even if an excellent one is passed up there will be another decent one somewhere. Not every job needs excellence some just need a baseline competence and an ability to be a team player.

These and other factors add subtle weight to risk aversion.

Also, difficulty in firing people doesn't automatically mean organizational dysfunction, sometimes it's an attempt at preventing abusive management. With how swiftly you are advocating for firing I would love to be at such an organization that made firing difficult if you were my supervisor.

> Interviews might not be able to tell you if someone is excellent but some terrible people will absolutely reveal their cards.

You appear to be equivocating on the definition of "terrible". The type of programming interviews that people dread, and are the subject of the linked article, are technical coding quizzes. These audition-style interviews do little or nothing to identify people who are "toxic". And grinding leetcode doesn't make you a team player.

> Also, difficulty in firing people doesn't automatically mean organizational dysfunction, sometimes it's an attempt at preventing abusive management.

Organizational dysfunction and abuse management are one and the same.

> With how swiftly you are advocating for firing I would love to be at such an organization that made firing difficult if you were my supervisor.

I advocated firing "someone obviously incompetent". Why would you fear that?

> You appear to be equivocating on the definition of "terrible"

Such a thing defies singular definition. Every company, person, and project will use different metrics. We might agree that someone truly awful is terrible, but more people exist on the margins. So how could I cleanly define it when clearly there isn't a single definition that matters?

I agree they don't find some kinds of toxic people, but I would argue that a system that does is magic.

> I advocated firing "someone obviously incompetent". Why would you fear that?

Because I don't know you and do not YET trust that "obviously incompetent" isn't a synonym for "politely disagreed one time", "didn't suck up enough", or "black". That last one is super important, somehow the places I have worked that fired very quickly, fired people that looked different very fast. And having a longer harder vetting process during might let a company see a red flag and avoid someone who might make such decisions and allow some time and process to review firing because the rate of terrible hires is likely lessened so firing rapidly is perceived to be less important.

> Such a thing defies singular definition. Every company, person, and project will use different metrics.

The submitted article is about bug squash interviews vs. leetcode interviews, in other words, technical audition-style coding tests. That's what I've been discussing all along too, and the relevant definition of "competence" is technical programming ability. You appear to be want to go off on a tangent about "toxicity", but it's unclear what bearing that has on the topic of the overarching conversation. Certainly no technical coding test is going to weed out toxicity.

> I would argue that a system that does is magic.

I agree.

Also, magic does not exist.

> Because I don't know you and do not YET trust that "obviously incompetent" isn't a synonym for "politely disagreed one time", "didn't suck up enough", or "black".

Well, you don't have to worry, because I've never hired anyone and won't be hiring anyone in the foreseeable future.

> And having a longer harder vetting process during might let a company see a red flag and avoid someone who might make such decisions and allow some time and process to review firing because the rate of terrible hires is likely lessened so firing rapidly is perceived to be less important.

It's weird that you've seemingly shifted suddenly from the topic of hiring engineers to hiring managers. Again, this is completely irrelevant to the topic of the submitted article, which is bug squash interviews. You want to argue that more "vetting" will help, but you haven't actually explained your method of vetting, other than "magic". In any case, if you're worried about racial discrimination in the firing process, which of course is a legitimate worry, then why wouldn't you worry about racial discimination in the hiring process too? After all, you don't have to fire someone who you never hire in the first place, right? Why do you think that black people wouldn't simply be "vetted" out before they get hired (especially since you yourself appear to want to increase the amount of vetting, and thus the opportunities to weed out whomever the hiring manager doesn't like)? It's truly bizarre to believe that you could institute a magical hiring process that could somehow nullify a racist hiring manager. On the other hand, if your magical hiring process could weed out racist managers before they get hired, then you wouldn't have to worry about the firing process.

If someone is not helping you, but you pay them for a year that is very costly.

I agree, of course, but once again I'm confused about how this relates to what I said or to the overarching topic.

Because many people who hire think more vetting reduces the chance of bad hiring.

> Because many people who hire think more vetting reduces the chance of bad hiring.

I don't know why you are answering for bluGill, and bluGill is answering for you. This is frustrating to me, because you don't appear to be of one mind. Unless you are secretly of one mind, one person behind two HN accounts.

In any case, bluGill's previous comment suggests that job interview vetting is rarely enough: "I've seen very few people obviously incompetent. As such it takes a while to prove they are incompetent. They do write working code and get it through review. Often the only clear sign is nobody likes working with them - only after you get rid of them do you have concrete evidence that they weren't contributing (that is the team got as much done now since they no longer were stopping their own work to help the incompetent person on things that it is never clear if should have been figured out alone)"

> I would prefer the best

How do you define best, and once you do, how much better than second best are they? What if the best person asks for several million dollars per year (an unreasonably high number as I write this), but the second will accept for $200k (a reasonable number though low if they really are second best). My guess is you cannot tell the difference between the top 20% of programmers in actual day to day work, and probably couldn't tell the difference in an interview.

This is Hacker News I’m on here to complain not find solutions

I agree. It's not a good idea to refuse to hire either young or old candidates. It's also not a good idea to refuse to hire experienced or inexperienced candidates.

These candidates have, as groups, different characteristic strengths and weaknesses. You need to give them multiple tests to see those. Leetcode is not a good test, but you do have to test a new grad on something, which might mean a vaguely leetcode-adjacent test like "go print this json tree of directories to the console then tell me what you did in your third year of uni".

Just because people in group A are treated badly, that's favoring group B, doesn't mean it's reasonable to do the reverse.

The real reason is you can pay them less and they'll put up with more BS.

You should have a mix of developers on any project, some senior, some mid level, some fresh out of school. The senior engineers should devote a significant amount of time to improving the junior. Your mix should skew to junior developers just because some developers will decide to switch to management, selling insurance, or some such.

Note that I did not say younger - you will get a few "old" people switching to programming as well, and they start off as junior.

Do I really want a mix of programmers on my project?

That's a very sweeping blanket statement and you're putting a lot of your beliefs on to other people. The smoothest projects I've worked on were always full of nothing but senior developers. But even though that is just my personal experience I know full well many projects benefit from Junior developers. Sometimes fresh perspective and energy are what you want to solve a problem instead of experience and expertise.

For a short time only senior is better. However eventually someone will leave and it takes time to get the new person up to speed. Your seniors will retire and you need juniors trained up and ready to take over. If you are only a short term next quarter thinker you are right, but if you think long term some juniors that you bring up to senior are the better value. (note that this also requires retention of existing people, if your culture is change the team every project there is no value in this - I'm against such cultures, but it is very common in the real world to not value experience on your project)

Of course there is value in bringing in external experts for a short time as well. You don't want to be isolated and not learn from the rest of the world, and of course sometimes you have money for a large project and sometimes you don't and want to fall back the the minimum staff needed to keep institutional knowledge alive.

Last, ignoring everything else, you have a moral obligation to society to build a better world. That includes training the next generation. (and this helps you - when you are retired they will be the senior engineers building the tools to keep you alive)

A lot of these are good points and need to be taken into account. But not every team persists, or even intends to persist. Not every developer is suitable for training others.

I've been on Plenty of teams that formed and disbanded in 3 months, we had a goal of building a prototype and turned it over to the customer. Just isn't a great place for new devs. Alternatively, I worked at slow and steady insurance companies that had all the appropriate processes for risk aversion in place and were great places to teach new devs. They let them stretch their wings knowing that they would be protected by good CI and good unit tests and other defensive practices at an organizational level.

Not every team needs every sort of skill or skill level. It's a judgment called each team needs to make, and some teams will decide wrong.

Sometimes you want to hire for a potential, as opposed to just experience, sometimes, the reverse. It depends what you're optimizing for.

What is wrong with capitals?

i don’t need them here, i am not writing a book

not wrong per se, but they take longer to type than lowercase

Wait, what?

Per letter, capitalization doesn't take much longer, but over the course of an entire sentence or paragraph, having to reach for shift can potentially add up. Sure, it's still small overall, but I think it's reasonable to give the benefit of the doubt to a stranger about whether the extra effort for them to type them is more than the amount of extra effort for you to read it.

It doesn't take longer for me as I have both of my hands on my keyboard and I can hold the Shift without any delay (or noticeable delay), simultaneously. You could even use "caps lock" in which case you wouldn't have to use shift at all.

The shift key is an uncomfortable stretch for my fingers. I can do it, but it isn't comfortable.

They don't if you have two hands!

NOTHING BUT THERE'S ALSO NOTHING WRONG WITH LOWERCASE

try pressing the the Caps Lock key

What's wrong with contractions?

NOTHING. CAPS LOCK IS CRUISE CONTROL FOR COOL.

I can't speak to anxiety or other disorders but I have a hard time swallowing that leetcode discriminates against the older crowd.

I'm not older, but I was in the industry prior to the leetcode style interview. I've never sat down and "grinded" leetcode but I've also never had a problem with those styles of interview once they became common. Neither have my former co-workers. I have a few friends that are still working, past retirement age, that would crush any problem you threw at them.

In my experience, it's not much different than the whiteboard interview. Sure the presentation of a problem can stump you for a moment, but with a few good follow up questions a solution, however naive, becomes apparent.

I had a mentor early in my career that repeatedly said "There is no such thing as an unsolved problem in computer science". I think that mostly holds true, you either recognize the pattern and implement, or you learn and implement next time.

That all said, leetcode style interviews are the interview version of a "bad code smell" and I'd strongly prefer a bug-squash style, even if it was in a completely unfamiliar programming language. Actually, an unfamiliar programming language might make it fun.

That is your personal anecdote, n=1. Not that your opinion doesn't matter or carry weight, but it doesn't say much more than the comment you replied to.

One reason they can discriminate against the older crowd is that they are hard for people with established jobs and familial responsibilities to study for. Working for a longer period of time leads to more experience bug squashing. Younger developers are much more likely to not have kids and more likely to have time to dedicate to grinding questions while having less experience with "real work" i.e day to day LOB software development that the bug hunt interviews optimize for.

I think the point is that competent developers can solve the problem on the go. You don't have to study for it.

I don't know, maybe some places do have ridiculously hard problems that you just have to memorize the algorithm for, but personally I haven't seen that. It's more like advent of code style stuff that you kind of just figure out by yourself.

You don't need to memorize the algorithm for, but if you haven't been exposed to those types of problems you will solve them much more slowly. Younger developers are more likely to have experience with more CS style problems than an older LOB developer who focuses more on writing clean testable code. I don't have an axe to grind in this as I am the younger developer in this situation.

Last time I did the gauntlet at Google I was able to solve all the problems.

However I wasn't as fast as others as I didn't recognize any of the questions and had to work through them. Interviewers expect you to be fast, or at least compare you to people who seem faster. Doesn't matter who studied or not, or who will actually perform better on non-leetcode tasks.

Also having kids is a real issue, it absolutely crushes the amount of time you have to anything besides working.

leetcode style interviews discriminate against older workers: older workers don't have time to grind leetcode. they have other obligations outside work. job descriptions require "x years of experience" but then the interviewer dismisses your experience and you get purely evaluated on whether you pass the interview, just like younger people who don't have the experience.

I don't know where you've been doing coding interviews. I'm pretty confident there are leetcode type problems you will have a hard time with and won't be able to complete in the allotted time. I'll share one of my experiences with a FAANG.

My interviewer first spent a lot of time "getting to know me" and asking trick questions, and stressing he also previously founded a startup, then handed me a problem and demanded I find the optimal solution in ~10 minutes and also wanted me to talk while working on the problem. I could not remember the arguments to a function and couldn't concentrate. It bombed.

A few days later I was sent prep material for an interview at a different FAANG. One of the videos was the author of cracking the coding interview solving this exact problem on a whiteboard. It took her more than an hour to arrive at the optimal solution.

So in short: 1) interviewer had unrealistic expectations

2) interviewer wasted time with introductions given his expectations

3) interviewer asked trick questions to determine whether I am a liar even though it was pretty clear from the introductory conversations that was highly unlikely

4) interviewer created an awkward interviewing environment and triggered my anxiety

5) interviewer thought I should be able to code while talking

6) interviewer probably forgot how long it took him to solve the problem originally and probably never timed that

> leetcode style interviews discriminate against older workers: older workers don't have time to grind leetcode. they have other obligations outside work.

This isn't discrimination on age this is discrimination on dedication to craft. Young people can have obligations outside of work just as easily as older people.

-- signed an old man

you can't call 20 years of experience a lack of dedication to craft...

I have more than 20 years of experience and I manage to find time to do leet code exercises before I interview.

I'm saying that testing people for their skill, in any way, will have imperfections and will wind up excluding people. If it excludes people based on something they can control that's probably not the worst thing. Not everybody can get time to practice, but do I really want to be working alongside people who don't practice? Maybe in some jobs I do maybe in some jobs I don't.

If someone really does have 20 years of experience then a single evening of practicing will do a lot more for them than it will a fresh faced kid from college even if that kid from college does it for two or three hours every day for a week.

I just simply don't accept the notion that leet code interviews favor people based on age.

i want to work with people who already know how to solve problems and don’t need to practice every night to be able to do that

I have played magic the gathering for 30 years. You better believe that if I am going to a tournament I am practicing in advance. Even if I don't learn a new specific skill it might hasten decisions I already know or show me subtle nuances that mught be clutch.

Why would I take my career less seriously than a game?

> I'm pretty confident there are leetcode type problems you will have a hard time with and won't be able to complete in the allotted time

I don't doubt that at all. But I've also had that on a whiteboard.

Also, it sounds like your interviewer was a jerk. Which seems orthogonal to the style of interview.

one of his colleagues was similar, so all I can conclude is that this particular FAANG has a higher than average number of jerks

Yeah, I've had a live bug squash interview (SSH into a server where a script is not functioning) and I didn't find it particularly fun. I did manage to get through it and the interviewer seemed impressed, but it's still stressful to do it with someone silently judging you. I'd much prefer take-home assignment and do it at my own pace, but I guess that might be susceptible to "cheating" (whatever you consider it being).

Doesn't a take-home task also prevent any chance to ask questions in the middle? And I mean questions both ways. An interviewee can ask questions about the code base that might be common knowledge to people on the team and an interviewer can ask questions about why a thing happened and gain insight into the interviewees thought processes.

I get that interviewing is a skill on its own, and not everyone has skills that work in an interview that transfer to the day-to-day job but don't most day-to-day jobs have a large communication component?

>Doesn't a take-home task also prevent any chance to ask questions in the middle?

Probably yeah. I don't remember my last take-home assignment having communication between the start and the end, although I would have probably been able to send them a question if there was something unclear.

>don't most day-to-day jobs have a large communication component?

I wouldn't know if mine has a "large" communication component, especially in terms of synchronous communication, since everyone's remote, choosing their own hours and so on. A lot of our work is very independent.

thanks for pointing this out

Isn't that what companies would look for, though? They want people who can work under stress. If you had two equal programmers, and one who was stressed and the second who was not, any company would pick the second one without thinking.

That assumes that during regular work, you'd be under stress. If it were the military, then yes, i would agree - working while you're being shot at means you need to be able to work under stress.

However, a lot of people don't do well under stress, and in many corporate settings, there's usually no real source of stress (except perhaps artificial stress induced by management). Someone might work much better in a stress free environment than someone else under a stressful environment, but you'd then easily miss these candidates by structuring the interview to be stressful.

All stress is not the same. Interview stress is actually very different from working stress.

Ask a firefighter who rushes into burning buildings for a living to give a public speech in front of a large crowd of people in a non-burning building. Which is more stressful? For the firefighter, it may be the latter. Fighting fires is what they train for, what they're experienced with, whereas giving speeches is not.

A lot of people, no matter the profession—firefighter, programmer, etc.—have a great fear of public speaking. Likewise, a lot of people, no matter the profession, also have a great fear of job interviews, especially the crazy audition-style interviews of programmers. It's simply human nature, with little bearing on job performance in general. We shouldn't be giving auditions to people who aren't stage performers. Personally, I never work with someone watching me—definitely not a judgmental stranger who will fire me for any perceived peccadillo—nor do I ever have some specific narrow time limit on my work in the range of 15 minutes or so.

Interview stress is nigh indistinguishable in my experience from a bad customer call. Indeed, if I replaced the bug fix part with writing an effective ticket or test for the issue, this becomes an even better method for tech support, technical account managers, or sales engineers than for developers, unless it's a company where devs regularly interact directly with customers.

Some organizations (although admittedly not all) put a lot of effort to offer a creative environment for their creative staff. So they optimize for something else.

A bit like 100m race vs. marathon.

I'm still at university, so despite my previous post having a bit of accusatory tone, I was genuinely curious. At least my profs told us many times that we should be prepare that stress at work is absolutely normal and that if we can't handle it, just drop out.

In my experience, being a workplace or even academia, there's as many types of work environments as types of teachers.

In the end, management (or founders if it's a startup) set the tone and who they end up hiring (thus, who will manage your work). And management is just normal people in the end.

Working minimum wage jobs of most stripes or in the service industry (retail, restaurants, that sort of thing) is far more stressful in my experience. Working conditions tend to be worse and people treat you worse. Not that tech isn't stressful at times, just that I've never felt like I needed to go to the walk-in to cry about it.

>> They want people who can work under stress.

Not exactly. Interviews are a different kind of stress that is usually a direct result of being observed in an unfamiliar and somewhat high pressure situation. You might have to deal with an outage at work, but typically your employment isn't on the line (less pressure) and you have a great deal more familiarity.

Some hiring practices intentionally try to create a more welcome, less stressful experience because they don't want to bias for interviewing skill (as opposed to fitness for the job).

> It may be fun on the job or at home, but the stress of a job interview both eliminates any "fun" aspect and makes even simple tasks more difficult.

Suggestion here: leave the room and come back in 40 minutes. Let them debug, fix the problem, and present to you when they're done.

At one place there was a bug squash interview like this. Rough idea was we wrote a very small version of a system we had in our app (some data-syncing-then-displaying service), with a handful of bugs and a very simple feature request.

It was very helpful for sanity checking if a person was able to be in front of a computer. There's a bit of a challenge because I think there's a pretty low ceiling of performance (strace on a Python program is fun, but our bugs are not that deep!), but we could use it to also talk about new features on this mini-system and whatnot.

General feeling on it was "this is a great filter", because ultimately we needed people to just pass above a minimum skill bar, not be maximally good at coding. And having the exercise really helped to communicate the day-to-day (which can be tedious!)

I've interviewed a few people who have a great resume and came off well in the interview, but I was left wondering if they could write code. (HR only allows me to ask specific research based questions in an interview and so it is easy to answer well without giving any indication you can write code). We now have some filter exercises that are about if you can write code at all.

> (strace on a Python program is fun, but our bugs are not that deep!)

They don't need to be deep to be useful. I once used it on nodejs on a hunch, looking for anything that felt "off", and discovered it was accessing settings files I had no idea even existed. Turned out the other dev was having problems because of one in his home directory the rest of us didn't have.

This.

Keep in mind people that a job interview is to check wether you want to work with the person, not to judge if they "pass" the test.

Or to say it differently, they don't have to find and fix the bug, they have to demonstrate ability with the tools, a good attitude, a match with your work culture.

I've done the equivalent of this by asking them to describe an interesting bug they've encountered in past lives; how it came up, how they hunted it, how they fixed it. By listening to them describe the work, asking questions, and following their thought processes, you can come to a fairly good hire/no from this single walk-through.

I know, it's short and humane, so not a good fit for current Sillycon Valley culture.

There’s no chance that I can come up with a good bug story on demand without any preparation.

For past experience I reviews like this it's a good idea to give the candidate advance notice.

compare this with the technical interview bomb of demanding you write a full blown program right fuking now, complete with tests. Which one do you prefer?

I prefer to describe a bug in details, but it is hard to come up with the details on the fly without prior preparation

Writing the program. Recalling a bug for the purpose of impressing a stranger does not sound fun.

Ok, but why do you think you should be hired in that place though? :)

Agreed. It's a great interview question. For a story-telling position.

Live bug squash is better, imo. I was part of the production support at my $DAYJOB. My work was literally finding and squashing bugs all day. And yet, if you were to ask me for interesting stories, I won't be able to tell you anything. Maybe I'll remember the latest bug I solved but most of the time, once the bug is resolved, its off my mind. I will have to go through my jira and github history to first see which bugs I even worked on, then filter for the interesting ones and then try to remember the story instead of just the root cause.

Ugh "tell us about a time" questions. I'm so glad the tech industry generally doesn't do that nonsense.

If you do get asked these questions just lie.

I straight up lie and make something up for these "tell me about a time..." type questions

Unless it was literally a week ago, I work on too many things to ever remember anything, and for me it's just a fact of the work that I'll be fixing bugs or whatever the questions asks. Nothing stands out, because I don't consider any of them to be some sort of special occasion worth keeping track of

> It’s easy for the candidate to self-assess their own progress.⊕ When the candidate isn’t doing well on it, they probably already knew it without needing to be told as much by the recruiter. This is a much better candidate experience than the whiplash of thinking you solved a question perfectly only to realize that the interviewer was looking for something else entirely.

Not to say that I think this is a bad type of question overall, but IMO, this is an anti-feature. The candidate does not need to accurately self-assess their performance on the day. They need to have their confidence preserved so they don’t tilt and tank the signal for the entire interview round.

I don't know, i think the best questions are the ones where candidates fully understand what is being asked about them. If they aren't able to self-assess did they really understand the question?

Candidates walking away pissed is itself also a problem. A significant percentage of candidates avoid buying from companies they're rejected from [1]. They also love sharing their poor interview experience with future potential applicants.

[1] https://www.wayup.com/employers/blog/how-a-positive-candidat...

> A significant percentage of candidates avoid buying from companies they're rejected from [1].

When rejected, or from an otherwise negative experience.

It's not necessarily negative emotional associations. Usually it's that I picked up on signal that the company has serious problems -- either overall, or with a key person -- which suggests I shouldn't depend on the company as a vendor.

What you want is to prevent floundering death spirals. They need to understand the criteria, but if you need to hint and nudge them, they need to not feel like that means they’re bombing. Of course, you can absolutely do that in a bug squash interview, but my point is that it defeats this claimed advantage.

"We don't want you to know you've already failed so we can waste your time for a month in the off-chance nobody better shows up."

What an incredibly bad faith reply. Candidates should always be given an answer within a few days of finishing the interview. But it’s not good for a candidate to think they’ve bombed during the interview.

If someone fails my interview, it is completely possible that they’ll still get an offer. I have one data point, and my colleagues are going to collect more. I’ve changed my vote from no to yes in debriefs many times once I saw other feedback. But that’s a lot less likely to happen if my interview wrecks their confidence.

Nope, this is as straight forward as it gets. There are only two ways not to know if you're failing or acing an interview:

1. You don't know what metrics you're being judged on.

2. You do know but you either can't assess your abilities in said metrics or you can't tell how well you've presented them.

The second one is up to the candidate, not much you can do about that, but if during an interview you have no idea what the interviewer is looking for, that's a terrible interview.

There is a vast distance between “accurate self-assessment” and “no idea what the interviewer is looking for”.

I’ve already explained why I think obscuring poor performance to preserve candidate confidence is crucial. If you think that’s a “terrible interview”, maybe you could elaborate on why, rather than just asserting it.

I wonder if this is an improvement over a conventional white boarding question.

It seems to me that debugging in particular is often dependent on the developer realizing some edge case or scenario where things circumstances line up to produce a bug. In that case, wouldn't a "bug squash" end up being similar to a "gotcha style" white boarding question.

The author, quite correctly, says that the interview should be scored not on whether the candidate can immediately vomit up a solution but on whether they can demonstrate their an organized thought process. However, isn't that true of white board questions as well?

Overall, it seems to me that the real problem with conventional white board questions is when the interviewer focuses on whether the candidate gets the right answer rather than on whether they demonstrate the ability to problem solve. While it sounds like the author is a good interviewer who focuses on assessing the candidate's thought process, it's not clear to me that giving debugging questions is actually what causes that.

"It seems to me that debugging in particular is often dependent on the developer realizing some edge case or scenario where things circumstances line up to produce a bug."

I don't think that's the case if there's a test failing. If a well-written test fails, that means there's a completely scoped out requirement for how the feature should work. The candidate should know how to use a debugger, so they should be able to start diving in and figuring out how the code works and what might be going wrong. I don't think that's likely to produce gotchas.

I've only had one "find the bug" interview, and it was awful:

- Didn't set you up with a way to reliably run/test the code, nor a step-through-debugger which, jeez is it 1980? Like setting up a repo in a language you aren't familiar with (say getting your py-env exactly right) can take a whole hour if it's not your main language.

- Didn't have standardized questions, which is hugely problematic (some bugs are 2 orders of magnitude harder than others)

- It also just seems like there's a huge random element, like am I debugging code written by somebody who thinks at the same level of abstraction as me? Did they write comments I understand?

Yea there's a huge element of randomness to these kinds of interviews. I wasn't a big fan of them at Stripe. You can get some signal like:

- do they methodically approach the problem? - do they understand the bug? - do they know how to use debugging tools? - do they know advanced debugging tools? - do they work well in an unfamiliar codebase?

But in practice I'd say most candidates check those boxes, at which point it becomes an arbitrary evaluation unless you pass/fail them based on whether they solved the bug.

As it happens the place I had a really bad experience with it was stripe. I had pretty much aced every other question, but basically was setup with a problem where I didn't have a way to reliably get it running (and I wasn't even clear what the app was supposed to do, it was just some random project off of github).

Fortunately I already had an offer onhand from facebook.

I gave this interview 100+ times and my criteria boiled down to: did they propose a hypothesis and then follow it to a conclusion? Doesn’t matter if it was wrong, just mattered that they were able to falsify it or find the solution.

At the end, I would have them walk me through the bug, its cause, and the fix in very high level terms to make sure they could articulate the path we took.

> - Didn't set you up with a way to reliably run/test the code, nor a step-through-debugger which, jeez is it 1980? Like setting up a repo in a language you aren't familiar with (say getting your py-env exactly right) can take a whole hour if it's not your main language.

How frequently do people interview in a language other than their "main" one(s)?

> - Didn't have standardized questions, which is hugely problematic (some bugs are 2 orders of magnitude harder than others)

How do you know they're not standardized? (You say in another comment where it was, and indeed that's where the blog post author works. It's described as a pretty standardized process by my reading of it.) You can pass the interview without actually solving the bug, but I get it's easier to blame the problem than admit you struggled with it.

I seem to interview in a different language every time I switch jobs.

> How frequently do people interview in a language other than their "main" one(s)?

I would say most of the time.

However i still think debugging is a core skill you should be able to demonstrate in languages that aren't your main one.

In Java, would you catch that these two snippets of code do different things?

        List<Integer> l = new ArrayList<>();
        l.add(1);
        l.add(2);
        l.add(3);
        l.remove(1); // invokes List.remove(int)
        System.out.println(l); // [1, 3]

        Collection<Integer> l = new ArrayList<>();
        l.add(1);
        l.add(2);
        l.add(3);
        l.remove(1);  // autoboxes 1 and invokes Collection.remove(Object)
        System.out.println(l); // [2, 3]

Bugs very often lurk in language specifics. Python has its infamously static default arguments as an analogous quirk.

I once interviewed (and got the job) for a java dev position with a mostly C/C++ background.

One of the issues to solve during the tech interview was fixing a bug in a fairly involved web app with a java backend. Long story short, it boiled down to "==" being used to compare two strings instead of ".equals" somewhere deep in their class hierarchy.

I found it and fixed it, even though I was not familiar with the reference vs value equality difference in java. General debugging skills acquired over years should allow you to track down exactly where things go more wrong than expected, and then help you reason about why it goes wrong.

Probably not. However i think (after copius println debugging and lots of time) i'd probably be able to narrow it down to which line the unexpected behaviour was happening. From there i would google the language construct, and hopefully get something useful.

The art of debugging isn't memorizing all the foot guns, its narrowing the problem down enough that you can deal with the foot guns you have never heard of before.

A good chunk of the debugging skill is just having seen a lot of bugs before, and knowing what to look for based on the symptoms alone. This is an extension of your ability to mentally model the behavior of the code, which is fairly central to any programming activity.

Another part of debugging involves various methods for reducing the size of the haystack, but it's not really realistic to rawdog every bug like that, as that would mean each bug would likely take hours rather than a few minutes.

Any half-experienced Java developer should spot the stink in that code immediately.

> setting up a repo in a language [...] can take a whole hour if it's not your main language.

Even if it is your main language.

> How frequently do people interview in a language other than their "main" one(s)?

A lot of companies have a set of "standard" interview languages like Java or C++ that may not be used by the role that is being hired for.

> It also just seems like there's a huge random element, like am I debugging code written by somebody who thinks at the same level of abstraction as me? Did they write comments I understand?

Isn't that part of what professional software engineering is about? Unless you worked with the same group of people for a decade and have had the time to mind meld together professionally, _any_ random developer at a new company is nearly guaranteed to think in a different way and have their own philosophy of code comments.

Checking for mental flexibility and adaptation to varying approaches for others is a great subject for an interview as a software engineer.

My favourite thing to do for "coding" interviews is to give the candidate a piece of absolutely awful python code that a friend of mine came up with for use for interviews.

There are code smells, side effects, errors, confusing syntax, a whole slew of things in it. I give them the code, tell them I'll help with every bit of python syntax (and really emphasise that I don't mark people down _at all_ for any lack of familiarity with python / python syntax), and ask them to work through the code and do a code review.

There's some python specific quirks that I largely consider bonus points at best, but anyone with any appreciable familiarity with at least one language should be able to spot a number of things. As they call stuff out, I'll dig in (or not) as seems appropriate.

So far it seems to be working out great for me as an interviewer. Candidates seem to be way less stressed than they are with a "whiteboard coding" exercise. Good discussions are showing me their development skills, etc.

Would you mind sharing it?

It does seem a bit unfair - ok sure, anyone familiar with C-family syntax should be able to work through it and spot errors. But anyone already familiar with Python would be able to do so a lot faster.

I think the idea is good, but to be equitable it should be given in a language the person claims to be already familiar with. Or alternatively, only give it to people in a language they are not familiar with.

If the goal was to objectively evaluate ability in a broad population, I 100% agree. However, if my dev shop primarily uses Python then selecting for people familiar with Python is pretty reasonable.

Thankfully there is a giant corpus of terrible code out there for any language. Especially that code written by the most evil terrible coders of all time: past-self :D

I'm dying to see this code

As was also pointed out in the TFA, while you may not be "marking people down" because of unfamiliarity with syntax, an interviewee who has more experience with the language (and it's debugging tools) used in the interview will have an inherent advantage over someone who isn't as familiar.

Right... Python.. Its syntax is so ugly. So I guess you can have really awful gems there with hidden bugs :)

I myself, shoot myself in a foot once doing Python code. I declared variable called 'len' and boom! What an interesting failure mode the script had. Took me a while to figure it out.

When I write Python it's an absolute hard line requirement to run it through Pylint and Pyright. They catch sooooo many issues you'd have to be a complete idiot not to do this (many people are unfortunately).

In this case Pylint will tell you about this mistake:

https://pylint.pycqa.org/en/latest/user_guide/messages/warni...

Hmm, nice stuff.. Luicky, I avoid python, so case closed. Ruby is so much nicer for me.

Ruby is even worse in my experience, but I guess some people have strange priorities.

Hehe.. Well, most importand is, use whatever language that makes you happy using it. I ve started using Ruby around 2006 I think. I was evaluating Python too, but syntax annoyed me and portability issues as well back then. I never regreted that I choosed Ruby. Its just solid for me :)

Please show some examples of such code, everyone will profit from this, no reason to hide it, thanks!

Yes! I love these kinds of interview questions. For some reason these feel more like a "normal" pair programming (/ code review) session, and I totally forget I'm in an interview.

Still a tiny bit stressing at the beginning while I read the code for the first time and try to understand it because I worry that I might be taking longer than the interviewer expects, but if I'm thinking out loud then that helps both of us. After I get reasonable context I can then forget about the interview and just focus on the normal pair programming / code review.

Great idea, that's exactly what I like to do as well.

Because many people were interested in some actual code, here is an example that we used for Java:

---

public class UserDao {

    private Provider<Session> sessionProvider;

    public UserDao() {
        this.sessionProvider = new DbSessionProvider();
    }

    public boolean saveUser(User user, ApiConfig config) {
        try {
            Session session = sessionProvider.get();
            session.save(user);
            System.out.println("User saved!");
            return true;
        } catch (DbException e) {
            EmailUtil.sendEmail(config.getTechnicalSupportEmail(), e);
            return false;
        }
    }

}

---

"Solution": We always ask the candidate "What would you change about this code?" or something similar. We expect the candidate to come up with some selection of: - Database sessions should come an injected provider. - Configuration should probably be injected as well and not passed as a method parameter. - Booleans are not canonical Java as return value. - Don't write to stdout but to a logger. - Exception handling should probably happen outside the DAO. - Don't use static methods for sending Emails without good reason. Non-static methods are easier to test.

Finding these things is as important as being able to talk about them and give background on advantages / disadvantages of doing things different ways. With good candidates one can talk easily half an hour just about this example and adjacent topics (e.g. error handling).

---

Edit: Formatting

Edit 2: added "Solution"

This is canonical Java. Commit and deploy!

I love the exception suppression via email.

Sr.Dev : "UserSignup failed, ask tech support what happened in our code"

Genius.

> Booleans are not canonical Java as return value

What should you return instead? Or should the method return void and raise an exception on failure?

You could return the saved User object, if `save` changes it in any way, to allow the caller to work with it functional style, ie. make it explicit that it is an updated object (or if it is immutable, although typical persistence frameworks expect mutable objects, "bean" style).

You could also go fancy and do it properly functional, so return something like an `Either<User, Error>` instead of throwing an Exception, but that's definitely not canonical Java...

Probably boolean. If you don't see the difference, welcome to Java :D

Ah, but the code already uses boolean instead of Boolean.

Whoops, so it does. In which case I have no idea what the solution is referring to.

No serious Java code uses booleans to indicate failure or success. It's on the same level as not using a language's standard variable naming scheme. It's not wrong, but something experienced Java developers working on good code will immediately notice.

In good Java code you would return either void to indicate side effects or the newly saved user. Exception handling should be done outside the function, because depending on the context you might retry etc.

The main idea is to check if people who say they are very experienced understand conventions and have experience with common Java libraries.

This code looks like normal enterprise Java ugliness to me - I object to it on principle, but apart from having to pass ApiConfig when saving a user I probably wouldn't identify any of the official problems you listed without familiarity with the rest of the codebase.

Same here... and funny enough the only thing that really looks wrong to me is they used "this" in only one of the two places they used "sessionProvider".

It's not a bug ("this" is implied, so it works just fine), but leaving it out hints to me that it was originally a third argument to "saveUser()", and either the original person writing this didn't have a solid API in mind or "saveUser()" was intended to also be static like "EmailUtil.sendEmail()" and it was switched around later on. Overall hints towards two conflicting styles in the same codebase and no good guidelines. Or maybe there are such guidelines (as hinted in the "solution"), and whoever updated this class only did it partway. This is the point where I'd poke around in the repo history to see why it's like this and whether anything should be changed further in either direction.

The "secret decoder ring" for this sort of question is that it's really "say something intelligent about this code that indicates you have real and good experience" even more so than "find the bug". So your answer would generally be off to a good start as well.

Given the Java code in question, while I understand the idea that "well, the session provider may just be hard coded" and such, I would definitely want to hear something about deficient exception handling. It's obviously an important part of the code and while I may not be able to spew out the exact correct exception handling without knowing more about the context and what exceptions may occur, it is obviously very wrong as written.

"Don't write to stdout but to a logger."

Not great advice given the fact that a logger paged pretty much every SDE on this planet.

That's the intention. So people see the messages.

That said, modern Linux OSs send stdout to journald by default. journald should forward to some centralized logging server.

It's an extra dependency. But I guess we will never learn. Just import log4j and pray.

Ever get the answer: this code should not exist?

I work it BI and am hoping to expand the team. Does anyone have a good universal SQL code for that?

In my interview, I was given a T-SQL code, where I did point out the mistakes, but was told my solution wasn't optimal, but that was because I've never used that flavour before (nor was it specified in the hiring docs), so I'd like to avoid doing the same to others.

I wrote some entry-level PHP interview questions years ago and the number of people who couldn't spot simple syntax errors or identify the security hazard of echo $_GET['var'] was staggering.

I have an interview question:

step 1: the candidate is shown the specification for a method and the results of running the test suite on an obfuscated version of the method. All tests pass. The test suite is minimal, and test coverage is abysmal.

step 2: the candidate is asked to come up with more test cases, based on the specification. The code is run against the updated test suite - most new tests will fail because the method's implementation has several known bugs.

step 3: The un-obfuscated source is provided and the candidate is asked to correct any bugs they discover.

step 4: the changed source is run against a full test suite.

I like this because the candidate's ability to think of test cases, debug, and fix existing code are all tested for.

This has failed me because code get pretty gnarly fast, many things I thought was obvious was missed by people smarter than me. I think as long as it is a tool for discussions rather than grades it is good though.

Curious if any interviewee has tried to unobfuscate the code.

Do you allow property based testing?

One of my favorite interviews was like this, but it was a long time ago and they printed out the code and told me there was a problem. Here's a pencil. What's wrong? Most were a function call or two. In one case I remember someone was doing a loop that incremented the index of a SQL query in code and using each result, instead of querying the set and looping through it in the code.

The fun part was it was a discussion. Two people, paper and pencil, talking about the code. And the examples were from bugs they'd already squashed in a code base they inherited. It was a project that I ended up working on that had... quite a few interesting bugs like that.

I somewhat disagree that this is reflective of a person's everyday work. Usually we're only ever working on at most handful of codebases at a time, and getting oriented in a completely new application (including getting it to build and run on your machine) doesn't happen every day.

I've done interviews like this and one major pitfall is the start-up time. It's possible to spend an unreasonable amount of time debugging cold-start issues with getting the repo set up, dependencies installed, and the app to build while the interview slips away. These things are representative of the first week on a new team or project, maybe. Figuring out why bundler can't seem to download the dependency in the gemfile isn't my idea of a good use of interview time.

I'm not too proud to admit I took the general concept from https://sadservers.com/ and turned it into something I could use for interactive debugging interviews.

I had golden images with some scenarios I wrote myself, and the images automatically shared a tmux session over http so I could follow along without requiring the candidates to screen share.

I did have to ask for IPs so I could ensure the machines were just available to me and the candidate, but it was otherwise pretty seamless!

Though now that https://sadservers.com/ has a paid service it might be worth looking into.

Hello, SadServers author here. Zero shame in adapting an idea you found somewhere; I'm super happy you built something useful.

I've thought of adding programming debug scenarios (I even got sadbugs.com lol), may implement in the future.

I'd love to see what you've done, please feel free to connect :-)

I was given a similar interview problem and thought it was one of the better interviews I've had. In my case the interviewer pulled up a web app and said this particular page is loading too slow, how would you approach the problem? We went from opening dev tools to look at the requests driving the page all the way through database optimizations and every layer in between.

This kind of interview didn't feel like a gotcha and was much much closer to real world work than the toy and/or algorithm problems that I have encountered. More companies should adopt these types of interview approaches.

I had a bug squash interview today. I found it nice, but also frustrating.

It was nice because I didn't need to practice and I knew exactly how to debug the thing.

It was frustrating because my personal laptop is from 7 years ago (from college), is slow, and the dependencies and editor don't work out of the box for an a new repo. Additionally, I'd prefer to use IntelliJ like I do at work but again, that's too heavy for my computer to handle so I resort to vscode and have to figure out how to use it. So then the interview becomes debugging my environment instead of debugging the problem. Maybe that's a useful signal, but it's not really bug squashing anymore then.

So overall, it was still requiring learning but there was not a very good way to test in advance (how do you test all possible repo structures?)

This is one of the interview questions at Stripe, and I agree that it was the most "fun" part of the interview (It is still pretty stressful giving an interview, but magnitude times much less so than converting a BST to doubly linked list, and converting it back )

If you can’t sit down with someone, have a simple tech conversation with them and be able to tell if they know what they are doing… if you need to “give someone a test”… the problem is likely not the candidate.

On the flipside, I've spent an hour with a candidate who seemed great, but could hardly _actually program_. It was a bizarre dichotomy. Dude was clearly smart as hell but was so caught up in building perfect abstractions in parts of our software that _didn't matter all that much_ using languages that nobody else knew or wanted to use.

I get it, your fun language is fun. But if the dev shop is 60% one language and 39% another, I don't really care about the small improvements. You need a strong reason that it actually _helps the business_

Ask them to explain what 4-5 different bits of code do. When they do, ask them to explain it like they would to a junior and to a senior/principal/fellow.

It works very well and takes only an extra 10 minutes on top of the conversation.

That biases heavily to candidates with good people skills and the skill to take over a conversation. Yes, you are not in fact immune to this even if you think you are, that’s why it’s so dangerous.

I disagree. That would indicate conversations skills equate to knowing what you are talking about.

"Yes, you are not in fact immune to this even if you think you are, that’s why it’s so dangerous."

I think that I am. I'll think about that a bit more. I don't care for popular people so, in fact, my bias might be the other direction. However, I don't think I have ever met a person who I thought was technical and proficient, but turned out not to be.

I have been forced to hire people I knew were not proficient because they passed a test, but not the other way around.

> I don't care for popular people so, in fact, my bias might be the other direction

It's not about someone being popular and you saying it just makes me more certain you don't realize the trap. People can have well honed social skills and not act like your standard popularity contest wining politician.

The most carefully honed social skills in the world won't help somebody plausibly answer a question like "What is the data type of this variable?" or "Show me the line where the null pointer dereference is happening."

They can add all the smiles and chuckles and other social niceties they want, but at the end of the day, whether they come up with the right answer or not, their logic will tell you pretty clearly whether they know what they're talking about.

That's the point of a technical interview: to distinguish those who think (or pretend) they can code sufficiently well to do the job, from those who can't.

aka Talk is cheap, show me the code.

So true. A test will only measure its set of criteria. What is a test for drive — desire to understand and learn? What is a test for how a person will respond to challenging and overwhelming prod scenarios? What is a test for if they will burn out in a month because they want to prove how rockstar of a programmer they are?

My most successful hiring has always been based on a conversation with 3-4 practical questions. I even worked in a company that had testing down to a science with all the psychometric nonsense and in the end, it just hired many sociopath-adjacents.

There's both culture and technical elements to consider in a potential hire. I don't think anyone would contest that vetting for the culture/drive of a candidate is important. But I do think the demonstration of skills is a necessary part of technical hiring, at least for non-senior positions.

I agree with that to some degree. But I do lean more into hiring on potential though, it has worked out for me.

Skills can be learned. Tools can be provided. But the employee’s personal values are very hard to change. These are core components of performance in some perf management theories.

I think context is important. If a company can hire on potential, I would say it will be a better hire in the long-term. But if employee turnover is high and tenures short, and you need work done now and not 6 months from now, I agree with you more.

If all I needed to do was express my thoughts out loud, I wouldn't need to hire anyone. I can do that myself.

What you're describing is a low hiring bar. Low hiring bars make sense for companies that pay low to average salaries, or don't require particularly deep technical skills. But it's an awful strategy if you're trying to filter for exceptional talent and willing to pay top dollar.

Strong disagreement!

Sure, there is some correlation between talking and doing, but I've worked with people who talk like geniuses and code like crap. And also the opposite.

For some skills, there is just no substitute for actually have people actually demonstrate using them.

That sounds like a great way to end up with a dev team full of smooth talkers who you then later discover can't code their way out of a paper bag.

I agree — this is very much the type of interview I like to give. However it took me a while to get good at steering the conversation and digging for details.

It’s really easy to let the interviewee talk through talk through all of the great things they built from a product and business perspective — and assume they understand the technical perspective. There are candidates who are really good at talking in an impressive way, but manage to talk around any true implementation details or deep technical understanding.

It’s also really easy to come away with a bad impression of someone who has a tendency to give short answers and not elaborate on things, even when it turns out that person is really skilled when you dig in.

I imagine a big part of the reason for the test-style interviews we have today is that it’s easier to train an interviewer on how to give them.

We tried this approach. It did not work. Source: SendGrid while ramping up to become a public company. The interviewing experience involved was fairly extensive; I had probably given a couple hundred interviews in my career by then and I was not the most experienced by a long shot.

We tried to have a zero coding interview. All behavioral and experiences questions followed up by "how would you design a system to do blah." Got a candidate that did well. Hired. Turned out they could talk the talk but not walk the walk. It was wildly unexpected. They simply couldn't code well. Too long to deliver, too poorly written, usually didn't work right. We insisted on _some_ coding in interviews going forward

Triplebyte (one of the best hiring experiences I ever had) gave me this during their initial interview.

They dropped an archive of a medium-ish codebase to my machine, that had failed unit tests. My task was to simply fix the code so the tests passed.

Not only did I feel engaged with the interview because I could speak aloud as I debugged, I also found it fun!

Too bad that Triplebyte then turned around and decided to sell everyone's data after it turned out their business was not scalable after all.

You're describing the parallelized web-scraper that they pointed to their own internal site? Yeah that was fun. Too bad everyone got the same question.

Had an interview question like this where the interviewers were very impressed. Said it was the fastest and cleanest they'd seen anyone do it, and in an unfamiliar framework no less. Unfortunately it wasn't enough for me to get the job. I had a couple of answers in the next section they didn't quite like.

> I had a couple of answers in the next section they didn't quite like.

Could you say more about this? Were they technical or behavioral interviews?

Technical. IIRC:

- I did the system design at the wrong level of abstraction (too low)

- Got asked about lowish-level database details. I took the question at face value and said I didn't know. What I should have done was explain what I did know (which was at a level slightly higher than the question), make an educated guess, and say how I would find the information. I still don't understand why Id didn't do that as I've always done it in interviews before or since.

This is my own assessment, they didn't give detailed feedback. Just that I wasn't strong enough in some areas (paraphrasing).

I agree. Great question. Academic LeetCode exercises are really not what I consider useful in evaluating candidates.

However, I'm biased. I'm really, really good at finding and fixing bugs, and pretty much stink at LeetCode, so take my support with a grain of salt.

One problem is that, if the exercise gets known, expect an underground economy of solution cheats. Same with LeetCode, but that's sort of expected. Bug fix solutions hit harder.

> You’ll need at least one question per language that you want to allow people to interview in. ... “Was this performance poor because they’re bad at debugging, or just unfamiliar with this language’s tools?”

Not necessarily a con. Different languages/tech stacks have very different tools. I.e. debugging a GC issue is different than debugging a network issue is different than debugging an embedded issue. If you chose a language for a very good reason, then it's not a con to filter out those who aren't familiar with that language enough to debug an issue in it

> If you chose a language for a very good reason, then it's not a con to filter out those who aren't familiar with that language enough to debug an issue in it

The language is based on/chosen by the candidate. We want to map the candidate to whatever language in our repository of "this question in different langs" is closest, to control for the candidate's familiarity as much as possible.

If you're screening to only candidates who can code in the language your company primarily works with, you're missing good candidates. The best are going to be able to pick up a new language rapidly, particularly if your language is a mainstream one that's probably imperative or OO-ish, or both; adding a non-esoteric language to one's repertoire is just not that hard a thing to do. But in the interview, I don't want them struggling with syntactic bull, as it's not a useful signal; I want to know how they think, whether they've seen code before, and can they reason from problem statement to debugged bug.

> If you're screening to only candidates who can code in the language your company primarily works with, you're missing good candidates.

I agree with that. I work somewhere that uses a lot of OCaml, we don't screen out those who don't know OCaml, but it usually helps them, since static analysis is easier with an ML family language than say, javascript.

> We want to map the candidate to whatever language in our repository of "this question in different langs" is closest

If there's not a close language in any of your repositories for the candidate, then I think that's a good signal that candidate may not be a great fit. As mentioned before, we don't necessarily ask candidates to write OCaml, most functional languages will have a similar bug + fix.

> problem statement to debugged bug what if the problem you might normally run into is perf issues related to the language? I.e. HFTs usually use C++ and perf = $$$. I would agree that yes, those who have worked on say Rust could be good candidates, but if someone has years of experience in writing highly performant C++, and that's what the job requires, probably easier to look for those candidates.

I definitely agree with you in general, just pointing out that there are times where being familiar with the language used is desirable.

I did a bug squash interview when I was interviewing for new grad positions and it was one of my favorite interviews (both as an interviewee and an interviewer)

- People were allowed to use their favorite IDEs - so you could see how proficient some people were

- Great engineers are really really good at debugging - it was great to see how people debugged and it helped me pick up a few things as well

- People couldn't leetcode their way out of this

Although I like this interview question, it has a big downside. You make the candidate spend quite a lot of effort while you as an interviewer don’t learn much. The answer to “Can this candidate find and fix this bug?” is not necessarily equal to “Is this candidate a good expansion of the team. What do they bring that we need?”

It's kind of a fizzbuzz-ish weed-out question: If they can do it, sure, you haven't learned that much, but if they cannot, then you know with pretty high certainty that they're not going to add to the team.

Astonishingly, I’ve found fizzbuzz to be a shockingly good weeder question for tech screens.

To me this sounds like a relaxing break from the effortful part of the interview.

It rarely makes sense to hire for a specific need. I want people that are smart and high agency. Seeing how they approach problems like this is generally enough to tell.

I've done similar interviews in the past and they are remarkably high signal.

The point of the interview is not to answer "Can this candidate find and fix this bug?" but rather "what is the candidate's approach to fixing unknown problems?"

A good performance looks like making a hypothesis for where the bug is, testing that hypothesis, and repeatedly narrowing in closer and closer. Finding and fixing the bug is irrelevant!

A bad performance might look like

- running out of hypotheses for what might be causing the bug

- making hypotheses, but never testing them

- failing to interpret the result of their test of the hypotheses (thinking it confirms one thing, when it doesn't actually confirm that, or it confirms the opposite)

So like trial & error? Modify, compile, run?

That could be one way to approach this, yes

Care to elaborate on other approaches? I am curious.

A candidate could, for example, debug the issue without modifying the code. Or they could alter the system it runs on.

Sorry, but do you have any data that shows 'great' candidates run tests and use your approach of finding / fixing bugs? Read the book 'Coders at Work' which describes some of the best developers of all time - most of them use print statements to find bugs / debug. Btw, I've solved bugs in 2-5 minutes which some developers spent hours or days working on (to their amazement) -- I've done this countless times in the past without using code or any debugging / testing tools -- all it took was reading the code and figuring out in my head what was going on and which data elements could introduce problems -- so I suppose I belong in your list as well? Sorry I just want to see the thought process here, to me it makes 0 sense and all the HN commenters commending the parent article are also making me absolutely scratch my head since the suggested process makes many many assumptions without having any data to back them up.

I agree with you, print statements or just simply reading the code (or modifying the code and then testing) can do wonders.

This sounds like a lot of fun to me. A lot of people dislike debugging, but I always enjoyed it. It's a bit of a murder mystery. You not only need to understand what's happening several layers beyond the initial problem, but why the code may have been written the way it is even when you find the issue.

I had a very similar intro experience to dev. I started with some friends as a hobby, and my first paid jobs were when I was hired to do a bunch of small tasks in different projects. Add some animations in this Ember project, fix some bugs in this Wordpress project, add a very specific feature in this Angular project, etc. I probably did a terrible job at that time and I'll always be grateful, but it also taught me A LOT about jumping into a foreign codebase and start working on it almost right away, especially since I was paid hourly and I was super-aware I was paid to fix the problems and not to learn the code. IMHO what this article explains is one of the most valuable bits I retain from that time.

I've never done a bug squash interview. I have done PR review interviews that are similar - given a PR that technically runs but has tons of problems, then see how many of the problems the candidate identities. I really liked that interview.

I was thinking, wouldn't it be fairer to do the test in an online code sandbox where being able to build and install isn't an issue? Then I thought, isn't letting the candidate use their own machine and debugging tools about as biased as hiring a limo driver based on how fast they can drive their own car? It does tell you who's a real "car guy" or "Unix guy", it just has a slight disparate impact by screening for people with money and hobbies. I mean, I think you should be allowed to hire these people because they will be better at the job, but hopefully they could also show that in a sandbox test.

Decent product idea? I’ll stay tuned for your Show HN.

your job doesn't let you configure your own environment and tools?

I find editors to be deeply personal things. Throw a vim user in vscode and you might be disappointed, and vice-versa. I don't care about your tools, I care that the sum of you and your tools make you good.

I think the good middle ground is having both a canned environment and allowing candidates to use their own... Unless the job is about debugging production systems where only vi is available, in which case that interview might as well represent the actual job.

We would run stuff in repl.it, you get the shared codespace and also don't waste time on the ops issues (which are maybe interesting and you really want devs to be able to handle but also entirely incidental complexity for most devs).

In abstract I think it's great to have people use tools their comfortable with, but I dislike dinging people because they write good code but are not good at unborking Python venvs (less of a problem now) (yes you can be a good programmer and still lose time on dumb environment stuff)

yikes. any time I'm asked to use a sandbox I ask if I can copy/paste to my own editor and share the screen.

Otherwise I'm easily 4-5 slower and can't think as clearly because of errant keystrokes.

I think there's the chance to do both!

Some candidates will not have problems installing dependencies and want to use their own tools.

Other candidates will want to not have to think about their environment.

Providing an online code sandbox doesn't preclude allowing candidates to do it on their own laptop!

This is exactly what we've built!

We take in these bug-squash/Github based repos, serve them in VScode Web/Jetbrains/etc, and give you instant results

Email is in profile if anyone's curious to see it live

I've written some interview questions before and can confirm that having people find and fix simple issues in code is a surprisingly great way to weed out people who have no clue what they're doing.

> Cheating effectively is indistinguishable from debugging skill. Even with knowledge of the exact bug ahead of time, if you just open the file with the bug, barf out the code to fix it, and run the tests, you’re going to fail the interview.

Did you mean "distinguishable" here, by chance? The first sentence seems to contradict the second.

A similar exercise but much easier to do in an interview is to give a short piece of code that does something a bit non-trivial and ask them to write (paper or whiteboard) test cases for it. Can then either run the test cases and see if they can find (all) the bug(s), or simply review vs a known list. Classic example in Myers The Art Of Software Testing is a method that takes 3 numbers as the length of the sides of a triangle and prints out whether it is scalene, isosceles, or equilateral. Many people miss a lot of corner cases.

This is indeed a good interview challenge, and as a candidate I’ve enjoyed taking it the couple of times I have encountered it.

I also like the related challenge of, “add a feature that does X” to a codebase.

In computer security, this sort of interview often takes the form of find the vulnerability in this app.

I like it. It feels like much more directly measuring skills than most interview questions.

How much time do you allot for this thing? Cloning the repo, dependency installs, server/DB setups, possible build steps, configuring the IDE to the project ... even if the whole thing is containerized, just going from nothing to running code in an IDE alone could be expected to take 20-30 minutes of the interview.

You just need to rewrite the dev tools in rust!

Joking aside, modern fast tools like bun can make this type of interview way more viable.

I once conducted this type of interview where the candidate had to install node on their windows laptop and... we spent most of the time debugging their install :/

FYI just small tip on the Windows / Node front, you should just install & use NVM instead of Node directly - will make many headaches go away.

In my experience doing these interviews in Node and Python, it takes about 5 minutes to get running. If it takes more than that... maybe use a different codebase.

What exactly is the interviewer looking for? Familiarity with tools, or actually debugging skill? I like the idea of debugging questions, I'm just not convinced all this setup and "use your own tooling" is necessary,and that a long enough snippet of given whiteboard code sample can morr effciently test for debugging skill. I can always teach (and learn!) new tools, and frankly, there is value to me as an interviewer to see someone manage without his tools.

The bugs are chosen in projects that don't require this much setup. (There is no server/database, they use standard tools for the given language, and they are generally solvable without needing an elaborate IDE/debugger setup. Of course that may help, and it's why you bring your own computer and choose the language ahead of time.)

Agree! This is one of the core pains we set out to fix. Using a CDE to package everything nicely for the candidate goes a surprisingly long way for their experience.

I think there's a valid point that any IDE != a candidate's local setup. But I think there's a compromise there - we try to offer a variety of common IDE's + a few mins of prep to download extensions, etc before you're thrown in the thick of it.

Compare and contrast https://sockpuppet.org/blog/2015/03/06/the-hiring-post/

I ended up doing this for an entire week, once.

I was hired and presented with a system that didn't work. I knew nothing whatsoever about their proprietary system and wasn't even given access to their codebase. I wondered why they were being so spectacularly unhelpful with onboarding tasks. At the end of the week, I found out - the whole thing had been a test to see how well I would do, and needless to say I failed miserably.

They told me I was a bad fit, and I had to agree, but probably not for the same reasons they were thinking. I really dodged a bullet with that one.

I like this idea but I'm curious how others would implement it. Does the developer have to write the failing test themselves? Are they instructed to do it first, or is that behaviour worthy of bonus points? (Most write a passing test afterwards, which is worse).

> I’ve seen candidates wield their text editor like it was an extension of their fingertips.

So am I allowed to install my own text editor and other tools (Emacs with my own config that requires some build time)? Or is it on my own laptop (I don't have one)? It seems unfair if some candidates get to use their favourite tools and others don't.

This is essentially what I do with candidates. I ask them to pair with me on adding features to a codebase, and also to offer a code review to a PR. 100% success rate on candidates thus far.

I agree. The best interview experience that I had was to be sat in front of a small app that had lots issues, both big and small, and also a lot an not-really-issues-but-still like inconsistent code styles and names, and code comments and code that weren't aligned. This was a great starting point for both me and the interviewer. The interviewer had a list of things to ask into about the code, so that we didn't get stuck.

Having done nearly 100 technical interviews (with multiple questions each), the bug squash question gave us the biggest signal on candidates.

Our question was far simpler: it was a simple class (java + python variants, no fancy syntax) and ask them to describe what it does, then find the bug, and finally ask them what they would change.

It reflects a true test of what the day to day is, and whether or not the candidate would succeed in the role.

Thought it was going to be about squashing a bug or releasing it outside your house, which would tell me a lot more about a potential employee than this.

"Work Sample Test"

Our best received interview question was in this style. Pick something your team fixed or did, distilled down do a 15min thing a teammate could do. Give the candidate 3x the time.

For us, it was a db value not being set as expected during a cron. The candidate got our forged bug reports, cloned a repo, ran the script, verified the unexpected db state, and off to the races

I’ve always liked the Gilded Rose refactoring[1] which is similar but about improvements/feature requests instead.

[1] https://github.com/emilybache/GildedRose-Refactoring-Kata/bl...

I could be wrong but given how fast things change in this business, this seems to test very micro level skills not the learning and adaptability skills.

What if a guy could, given the business or technical issue the code solves, write something faster than he could fix others code?

May be the job is about heads down coding so this bug fix test is appropriate..

Then you could do that and showcase that you can write code incredibly fast.

How is debugging a "micro level skill"? Jumping into an unknown project/system/whatever is something you will likely have to do at some point. (Sometimes it might be code you wrote yourself years ago, but revisiting without any context makes it feel unknown.)

Is this hypothetical guy always writing 100% correct code?

I am quite skeptical of item 6 in this list, but otherwise it looks pretty good:

> Cheating effectively is indistinguishable from debugging skill.

If you know the code or problem ahead of time and how to fix it, you can certainly be convincing in your walk-through of how to do it.

> It’s fun because fixing self-contained, reproducible bugs like this is what so many of us enjoy the most about software engineering in the first place.

Do really "many of us" enjoy that? In the "day-to-day business", not interviews.

”what we’re observing is X, but we see Y instead.”

What?

Anyway, I’ve found these kind of interview questions rarely helpful because the scenario is either overly simplistic, or it’s some obscure bug they identified (which they only caught because it caused an outage, i.e. they also didn’t “solve” this when they wrote it).

A similarly poor interview question happens in IT operations when the interviewer asks super specific details about some CVE from 5 years ago that he’s particularly proud of pulling an all nighter to patch a bunch of servers.

> what we’re observing is X, but we see Y instead.

I think this might be a typo - we're observing X but want Y, or some similar variation. In any case, there's a bug.

Sorry about that! edited to

> What we're observing is X, but we want to see Y instead.

it's kind of strange to me that ad-hoc debugging is considered such a valuable skill. I thought it's mostly a juniors' perspective. I typically set up extended loggers and logger methods, write everything to the some formatted file, and have a diffable, persistent, provable, multithread-friendly and versioned bug demonstration, which scales well to bugs of any complexity.

(I once found some 10 bugs in a pretty old and tested bond calculation engine while migrating it to the cloud, which nobody could initially believe.)

My kingdom for this approach to take the place of (some/all) leetcodes.

> I’ve seen candidates drop into strace to debug a Ruby program

strace is great. 99% of the time, your program is looking for a missing file.

A pity it's so hard to use in modern macOS due to OS integrity protection.

Kind of like the "install this software" as part of the interview process.

"My hourly rating for figuring out your bug is $X".

There is academic research on how to interview. Most comments here give no indication the commenter is even aware it exists much less what it says.

I'm aware this exists, but I'll admit to not having read it directly. (HR gives me a list of questions I'm allowed to ask in an interview and they tell me those questions are based on research but I only have their word they are)

I like this approach. We did something similar, but for a position as support engineer, where coding ability was one of the requirements, though not the only one.

They were given a piece of code on a toy, easy to read, english-like pseudo-language, and were asked to examine it and explain what the output to a specific input would be.

They were told that we would answer any question, except what the answer was or what the code was doing, and encouraged them to explain their reasoning as they went ahead.

We wanted to asses the candidates ability to problem solve, communicate, how would they react under preassure, and how they approached the issue.

We were not necessarily looking for perfect answers, we rather rated the candidates willingness to ask for help when stuck, how well they communicated while doing the task, whether they tried different approaches, their ability to grasp what an unknown piece of code was doing and so on... all important properties for a role supporting mission critical solutions with near to zero down-time requirements.

Our reasoning was that technical knowledge is easy to acquire over time, but problem solving skills, how candidates tackle difficult and unknown challanges were more important for the role.

In my current role I try to design interview questions like that, looking both for the candidates current skills as well as their future potential.

Not easy, but IMHO more rewarding for us and them in the long run.

It's easy to write this question. I can cut a branch with gnarly UI bugs in it, no sweat.

Unrelated: why there are no stories about interview questions for managers?

Would love for others to share non-technical questions that can filter bad candidates!

Ah the humble 2 point bug ticket

>Cheating is indistinguishable from debugging skill. Even with knowledge of the exact bug ahead of time, if you just open the file with the bug, barf out the code to fix it, and run the tests, you’re going to fail the interview.

Umm hopefully you don't fail for this. I've definitely known and hired devs that can do this with surprising frequency. Just because the interviewer may have taken hours to find the bug doesn't mean someone else must.

The only downside I see is that I need to share my screen with a person I don't know. Don't get me wrong, but I normally have access to some crypto on my laptop, banking notifications turned on for my 2FAs, private messages I'm getting from my spouse, maybe some borderline NSFW chatgpt dialogs, screenshots of my ideas, emails, private notes, personal projects, whole bunch of tabs I am not willing to close or share, etc.

I don't have issues to show the screen to a friend of mine. Or showing the screen of a work computer to a colleague. However, demoing the whole screen is violation of privacy. Please don't do that, unless you grant VNC access to a machine with a set of popular tools, code editors, etc.

How about reverse bug squash? Interviewer shares the screen with all his/her private info, notes, tabs, development environment settings, shell command history (with maybe autosuggestions turned on), and the interviewee needs to guide the interviewer towards finding the bug?

True, this would discomfort me, too. Also in webconferences, I never share the full screen, only one window. This should be enough for bug squash as well.

This is common in many interviews, not just "bug squash" interviews. I recommend having a plan for it if you're interviewing. Perhaps dedicate a virtual desktop to interviews if your system supports it.

Thanks for recommendation. I just finished my rounds of interviews, gotten a decent offer (SF company), being 20 years in industry. I have my own rules, and luckily they align very well with industry standards. So I never had any issues not following the scenario when I need to share something to land a job. I also don't do any take-home assessments, and moreover, I only use one programming language while interviewing. Never had any issues with that.

I think I don't need a plan or virtual desktop. I just say "no" if it doesn't work for me. Unless I am really desperate, I do not share.

Reply on news.ycombinator.com