How to Interview Engineers When You're Not Technical – Part II | Hacker News Reader

How to Interview Engineers When You're Not Technical – Part II

greghausheer.com

195 pointsgreghausheer87 comments

Some good questions in here, with one exception: > Programmers are unique. They're one of the few professions where outside the boundaries of their working hours they choose to do the same exact thing they do at work: programming.

Hiring managers should definitely not be assuming that all programmers do apps in their spare time. It's one thing to expect them to have a code sample (anyone should), but many strong programmers have interests and responsibilities that take them away from computers in their spare time. This doesn't make them less effective or dedicated programmers, so while it's always interesting to hear about the projects of those who have them, it shouldn't be assumed (and you shouldn't consider it 'points off' in an interview if a dev doesn't do side projects)

For me it varies with how much stress I'm facing at work. When I log out feeling fine, I'm going to develop things in my free time for sure. But when I'm just constantly stressed out, I don't have the fortitude to keep the pace up at work while making things on the side. One of them has to go.

> Hiring managers should definitely not be assuming that all programmers do apps in their spare time.

It baffles me how much this is an assumption. Like all the developers should not have any hobby that doesn't involve coding.

but don't you think its a fair guess that someone who hobby is X might be better at X than someone whose hobby is Y.

No, because that doesn't say anything about the quality of his code. He could be sitting around writing complete chicken scratch if he doesn't have a team with good leadership and code review guidelines.

Not necessarily. My hobby could be, say embedded programming and I could be a front end engineer.

I see your point, and I partly disagree. First because, they are doing it for fun, to solve things they want to make it work. That doesn't say they are learning something, or following clean code guidelines.

But let's assume they DO. They are better at coding. But are you sure coding is the only skill you want? I've seen my fair share of developers that lack "real world" experience, to the point they cannot see that what they are doing is not what a user wants, or how they want to do it.

In some cases, open source projects are made for developers, and managed/prioritized by developers, not someone with experience in project management, or with a roadmap with the users in mind. So, they might be lacking a lot of other important skills, and I would dare say that some also lack empathy towards non-programmers - the paying customers.

I'M NOT GENERALIZING GUYS!

I'm just point out things that I saw in some instances, with some people.

yea ofcourse, if someone is better at skills you are looking for then by all means coding hobby is irrelvant.

But what if all else being equal, why would you not choose coding hobby candidate.

If we are going the "all else being equal" route, one could argue the hobbyist is a worse candidate because they have spent more time working and thinking about programming but still haven't been able to demonstrate their skills are superior to someone who has spent much less time refining them. Give me the person who is able to get the same thing accomplished with less effort and time.

Also I would imagine that the person who also code as a hobby is more likely to have a burnout later on. Just a guess though

There's no way to compare that "time" of course.

Could go either way, right? I've seen folks who have side projects spend most of their work day working on their side projects. They might be "better programmers" (also debatable), but if they don't get their work done - does it matter? I've seen folks who do really solid work for 8 hours and then go home and do other things, some of those folks have said that the separation is helpful so they can really be at their "programming best" at work.

(I say this as someone who generally have multiple programming side projects going on at any given time and also freelances...)

I write hobby code sometimes but I don't write unit tests for it and I almost never go back to refactor it. Once I achieve my initial goal, I tend to be bored with it and set it aside.

This seems to be a pretty sensitive claim, but it also seems completely reasonable to me that people who practice something more are better at it than people who practice something less. Even if it's a completely different domain than your work, it's still developing your intuition and exposing you to new abstract concepts.

As someone who runs interviews occasionally, when you're sitting in an interview room, you're not making an immediate decision whether the candidate is better or worse - that comes after the interview. During the interview itself, your job is to learn as much as you can about the candidate. The biggest problem for both you and the candidate is that you might get to the end of the interview and still not be sure whether they're a good candidate, and therefore have to reject them.

You've got two probabilities to work out, "how likely is this person to succeed in the role we're hiring them to do?" and "how confident am I in the previous number?" Side projects usually don't change whether you think the candidate is a good fit (at least for me), but they usually do increase your confidence in your prediction, and turn some no-hires into hires.

If "Time technically participating in an activity" directed overall skill that clearly, yes. Unfortunately there are plenty of 10 years exp devs with thousands of hours in their chair that also just aren't very good.

I don't think I've ever run into that assumption.

Then again, maybe those shops just don't hire me, so I don't get exposure to it...

We also should not propagate the caricature of the developer. You know, the all I need is gallons of coffee and I’m off grokking code all day in a hoodie. I’m off to my Kubernetes meetup! Whoops, just dropped my giant math is fun textbook.

/me looks at a pile of empty coffee capsules, /me looks at the hoodie, /me looks at the (unread) giant math is fun book, I hide in shame.

I do agree though, especially in hiring I find it weird to expect people to have their hobby being the same as their job. I'm quite curions if there is another profession where this is the case. Photographers maybe?

Well if you’re the real deal, you’re the real deal :p I guess only celebrities/insta influencers have to keep up the image.

Casually hides Cracking the Coding Interview book

> Casually hides Cracking the Coding Interview book

Ironically, I'm pretty sure that owning that book is part of the image :)

No, the image is that you own CLRS. Cracking the Coding Interview is to the programmer image as Cliffs Notes are to the Shakespeare nerd image.

I feel like owning a print copy of CLRS just implies that you studied Computer Science at a university, not necessarily that you're some super gifted hacker. But then again, most people who own print copies of Shakespeare are probably not uber-Shakespeare nerds, but took some English Lit courses.

> I'm quite curions if there is another profession where this is the case.

Any artist, really. The line between work and leisure are blurry at best.

Musicians https://www.forbes.com/sites/quora/2017/10/03/programmers-an...

I'd say anyone who is a maker. Someone who creates things creatively.

There is a good chance your carpenter has a workshop at home. If you have the kind of mechanic who likes will modify your car, they probably do their own car too.

Pilots to some extent (the senior airline pilots who can afford to fly privately), but that is often to get back to the joy and freedom of roaming the skies at will in light aircraft vs the demanding and rigid world of commercial flying, so arguably not the same thing.

Please read the article further down. The author explicitly says:

> It's OK too if an engineer doesn't have any side projects.

I always ask about side projects to give candidates an opportunity to highlight their side projects, not because we require candidates to have side projects.

I'm sure some hiring managers out there have side projects listed under their "must have" criteria, but in my experience the good hiring managers are just collecting as many data points as possible. In fact, I've known many hiring managers who take side projects as a potential risk factor, because they can pose a distraction to the candidate if they become too demanding.

All other things equal, a candidate with extensive side projects is going to have more experience than a candidate who has never worked on any side projects. In the real world, you never find two candidates who are identical in every way except for their side projects. The goal of an interview is to collect as much information as possible about the candidates in the limited interview time. Asking about side projects is just one more place to search for those data points. It would be equally unfair to disqualify side project experience from the consideration process.

In my experience, have side projects is largely a boost for junior candidates who haven't had enough opportunity to build a long professional resume yet. I can't remember the last time we interviewed a senior developer where side projects were the tipping point in our decision making process.

fair point, I did miss that further down in the section. It's still unfortunate to lead with a generalization that plays into a common trope/assumption about developers that does play out in the hiring process, but it's good to see it called out explicitly below that it shouldn't be required. I certainly don't disagree that side projects are a good data point especially for junior developers who don't have the direct professional experience.

Fair criticism, point taken. Thanks for the feedback here. Def did not mean to breed a culture of always being "on" and "crushing it" which might also be part of the trope.

One thing that I've never disliked about hiring managers asking for side projects, is that when I work on one, I do what interests me. I like trying new things, experimenting on stuff that I don't know how it will turn out. When I get bored or I hit a dead end, I stop. A lot of it is unfinished and messy. It's for me, not anyone else. My time is valuable, especially outside of work.

Yeah. I'm building an http server for fun. Last year I wrote a JSON parser one weekend. A few years ago I wrote a YAML parser. I make things like this fun & learning, not because I'm trying to impress anyone. When someone asks why I'm reinventing the wheel, or is generally unimpressed that I didn't do something they view as more useful, it can be awkward.

Not only that, but when you are programming for yourself, you can take on practically infinite risk. There is no downside. When you are programming for the company, you have to err on the side of conservatism. Sometimes the only way to break through to another level is to take on that risk and see how it turns out. It says a lot about a programmer that their portfolios consist of lots of efforts that don't end up with a "product". They have tried a lot of stuff that you can't reasonably try in a work setting.

Not only that I’m sure programmers are not unique in that regard. Surely musicians do music in their off time. Artist do art. Carpenters build. Mechanics have beaters, etc. etc.

There's also a general technique for filtering experts when you don't know their domain well: ask them do something known to be impossible. [1]

Very bad answer: "Sure, no problem." [2]

Bad but tolerable answer: "Not possible but it's too hard to explain why."

Good answer: best attempt to explain why it's not possible, and the nearest solvable version or what you would need to change to make it solvable.

There's a great Better Call Saul episode (s4e5) where an expert is idenified as a unsuitable (in part) because he insists a project could have a firm maximum duration without caveats about what could go wrong, and he could do it without something it obviously needs.

[1] That's arguably what the unnamed MP was doing when he asked Babbage if he'd get the right answer if he put the wrong numbers in the machine:

https://en.wikiquote.org/wiki/Charles_Babbage#Passages_from_...

[2] At best they might silently replace it with the possible version, but that's still hiding the tradeoffs from you and means they could be misrepresenting other answers.

As a professional programmer I spend almost all of my free time NOT programming, but I do spend it taking care of my family and social obligations instead. While I love what I do I would love it less if I did too much of it.

Right I love programming. But I do it 8 hours a day. That's quite enough!

Exactly. When I was a junior I had plenty of side projects that I rotated through. Then that got old, I developed deep passions for other non-computer things, have a family, enjoy knowing people, and what do you know? Not many side projects that involve programming. I haven't updated my blog in 15 years. I'd rather go fishing with my kid, rock climbing with my bud, or banging together a garden with my wife.

Seems to me it still makes sense to only hire people who program in their free time... if your real goal is to hire people willing to do a lot of overtime work. (I am not saying this is a good thing to do.) Things that prevent you from programming in your free time are more or less the things that would prevent you from working overtime.

Or it could simply be thinly disguised ageism. "Hey, I am not saying we shouldn't hire people who have kids... I am just saying I don't trust people who are not passionate enough to write code in their free time!"

As someone who specifically chose not to have kids because I want to spend more time programming (crazy, I know)... I'm not that interested in working overtime for the company. I'll do it in emergencies, but my time is my time. I only have so much of it and there are things I want to do. No amount of money, or recognition, or pride in helping someone else get rich will give me that time back. Just because I want to spend my time programming, does not mean I want to give it to someone else.

How do you calibrate yourself on these open-ended questions? How can you compare one good answer to another completely different good answer and figure out which one is better overall?

You can't rely on a "The very best engineers have a checklist they apply depending on the type of bug." type of framework here -- each "very best" engineer might have a completely different way of doing things.

You have to have at least some questions that are difficult to answer but have a countable range of responses, and watch for specific lapses that can show how well a candidate demonstrates specific experience required in your job description.

For example, if you require some experience in distributed systems or scaling things to many users, have them talk about "Could you talk about an or program you built to scale to n users, what performance or reliability issues did you see (or foresee), and how did you solve them?" Have a loose set of industry specific things to watch out for, like caches, load balancers, monitoring/observability, etc.... You don't have to have the specific knowledge yourself, but if the candidate can explain to you what they did, and how/why, in the framework of the specific experience you're looking for, that's good signal!

> The very best engineers have a checklist they apply depending on the type of bug.

This is misleading at best.

Masters do less work than beginners. They have years of intuition built up that let them apply heuristics extremely cheaply and save their energy for solving the novel parts of the problem. There is no 'list'. There is only the verbal part of the brain trying to introspect on what just happened. Sometimes accurately, but quite often not. The ones who get it even close to right become mentors, instructors.

I can spend as long as you like explaining my debugging process and the first time we sit in a war room you'll call me out for not following my own 'list'. Every situation is either unique or the result of a lack of self-reflection (why do we keep solving the same problem over and over? Are we not programmers?)

The biggest danger to people who can't explain at all is when advances in their field arrive and they can't introspect enough to adjust their behavior. There are limits to how much you need to address this in an interview process. For many of these people, they'll already be in that space when you meet them and easy enough to spot.

Most problems aren’t special. They occur because some good general guidance wasn’t formed or applied.

Having a checklist does help masters. It helps make sure they don’t miss the obvious issue in front of their face.

Checklist manifesto is good reading on this.

Hmmm. I think the checklist exists, for sure, but tends to be more Plan B. If your initial strategy is exhausted with no progress, fall back on the checklist.

As a programmer I have an opportunity not everyone else has. I can codify parts of my checklist in a way where I don't really have to think about them at all save for a few times a year with the big outlier problems, teaching someone to do basic troubleshooting themselves, or starting a new project.

You know, one of the things I found most amazing about interviews and the hiring process was that to a large degree, the things that a person says in an hour session determine whether they will get hired.

Yes, not their references or resume (at a certain stage) or what they're able to prove that they coded, but the things that they are able to come up with to say. Doesn't that strike you as sometimes amazing?

Now, that may be a crazy notion and flawed in many ways, but I came to realize the following. The reason that that is often acceptable is that it frequently is a good proxy for weeding out the unqualified. In this sense:

If someone doesn't even know to say that something is important, or that they solved a problem a certain way (regardless of whether you have proof the actually did it), they probably have never even thought about those things being important.

If someone tells you (just offhandedly) that they solved something by jumping right in to doing x,y,z at the detailed level and coming up with the technical solution (the algorithm), but not the project planning or scoping part of it -- they probably have no idea or experience about what's necessary to plan out a project. They may not be good at estimating or communicating. They may be someone who gets lost in the details. If they have no pitfalls to warn against, probably they haven't even seen how a project has gone wrong before. They are probably not a good candidate for a senior developer role.

Whereas, if someone knows to say that they had to do some top level estimating or quick pass of which solutions would be the best from a cost/resource standpoint, consider the timelines to implement and risk of doing so, how to decide who on the team would be put on the project and in what sequence, etc... Then I know that they at least realize these things are important.

(Yes of course you should probe deeper. And I know that this applies to some kinds of roles and not others, but since the article is about interviewing for non-technical people and is often about hiring for a role requiring a higher-level hiring decision)

> There are countless ways engineers choose to spend their off hours. Some contribute to open source. Others take online classes. Some are tutors, and others work a second job as a freelancer.

It's called off hours for a reason. Being a tutor or a freelancer is a second job.

> [From the preceding article] how were teams split, tasks delegated, and projects managed throughout Sprints?

> Did they use Agile, Kanban, or Scrum? How long was their Sprint?

How about asking whether they used sprints, or formal agile methodologies, at all? Why assume that every engineering org has drunk the kool aid?

The context is companies with 30+ engineers. At that point you're going to want some form of formal project methodology that allows everyone to co-exist somewhat peacefully. Generally, the results otherwise are varying level of dysfunctional.

I worked in a handful of teams at Google and none used Agile (I believe some do, but they're a minority), and very little formality besides a weekly team meeting. From what I hear, FB and other big companies are in a similar situation. Sure, you can stretch the definition and say they're dysfunctional ("omg lol google kills projects") but you can't deny they're successful.

First of all, I used the word Agile in my response zero times. I used the words formal project management for a reason. Agile tends to be assumed but if someone asks then either you can infer what they actually want or you're going to be a terrible hire in a startup anyway.

Second of all, as people love to say here, your small or medium sized company isn't Google so you shouldn't make decisions as if it was. What works for Google does not necessarily work for random startup or small company number 7521. Every small and medium company I've seen without a project management process was a disaster. Partially because, unlike Google, they can't throw 10 million at a problem and shrug if they waste 9 of it or cancel the whole thing in 2 months.

> What's the one thing you would not compromise on as you write code?

Code that works.

I'll take a gordian knot that works over an over-engineered system that fails or never ships. Some of the worst code I've seen comes from engineers who focus on "unit tests, documentation, and refactoring code." Those are all good things, but they aren't #1.

The "done" question is a very good one.

I completely agree about the unit test point. I'm not arguing that it's useless - quite the opposite. But it's damn hard to write useful unit tests. I've seen a case where an internal IT team took ownership of code I delivered. A few months later I was asked to make some enhancements, and I found that they've added a lot of unit test code. It was absolutely useless. Essentially, they were testing the compiler, not my code. Complete waste of time.

Or get two candidates and ask them to ask each other questions. Ask them to score themselves and "opponent". Infer score yourself. Promote winner to next level to interview other winners, until final battle. Brutal, but what can you do when you're not technical?

Jesus Christ. Somewhere a hiring manager is explaining the benefits of this to human resources.

Step 1 - regular interviews.

Step 2 - group interviews including memory check with coding task

Step 3 - this thunderdome you just created.

We never did group interviews but did try panel interviews as we noticed the order of interviews ended up skewing how we felt about a candidate.

You joke, but an interview for a Australian government agency I did (can't name it for fear of assassination) I was asked by interviewers which other candidates in the group interview I would work with and why. They were not at all technical.

Ha. If I got paired with some loser by a manager that pulled this, I'd make sure that said loser got the job.

Devious, but delicious. The punishment fits the crime.

There are some good questions here.

Still, if you're non technical when interviewing, how would you know the right answers to technical questions?

Eliciting underlying thoughts about problem solving is important, but is more than just saying the right keywords.

If interviewing is not much more than a game of keyword DDR, it makes successful outcomes even more challenging.

Maybe hiring is broken because it is designed for linear working paths, but engineering is not linear.

> if you're non technical when interviewing, how would you know the right answers to technical questions?

My reaction to the article was similar. My answer to the title question would be: "Find a technical cofounder you can trust."

Absolutely. Don’t take advice from someone who doesn’t have relevant experience.

I find a much better success rate in not hiring he traditional HR ways. Tech folks are not linear workers with floors and ceilings, they are growing much faster and on multiple layers at the same time.

It’s humorous when non technical ppl try to manage things they don’t understand.

That table seems to be currently being turning in society because of covid, if you ask me.

Technologists will have to move into each area of management, because maybe it’s technologists turn to learn and manage business after such a long time of business trying to manage tech.

The biggest tech companies you know are founded and run by techies.

Yup. Lots of non tech companies (pretty much most) which are still doing tech hiring

Thought I'd write a Part II to my first article on questions non technical founders and managers can use if they need to interview engineers.

I consider myself good at programming in general. But let's say the interviewer was hiring me for a technology stack that I'm pretty much clueless about... let's say assembly language or cobol...

Even then I can bullshit my way past all of those questions, and I'm sure most people who can't program can also do the same too.

"What's the one thing you would not compromise on as you write code?" Its not hard to figure out the right answer to that one: "I never compromise. Must be perfect." But of course that's not true. There are always pathological conditions that can reasonably ignored especially in the first release.

That's easy. Just say I will never compromise on writing unreadable and messy code. I will also never compromise on documentation. That's just a disservice to my team members and disrespectful to anyone who will ever read or modify the code.

That bullshit will work for both technical and non-technical people. Of course do I do follow my own mantra all the time? Does anyone follow it at at all times? No.

You need some kind of coding question at the very least to measure coding ability and a non-technical person just can't do that.

Engineering is practical work. Of course you compromise among all your mutually unreachable goals.

That's funny, I came to the almost opposite conclusion that the obvious answer was that everything is tradeoffs and compromises.

I suppose very easy things like consistent indentation width might be done perfectly in a codebase, but any non-trivial thing always could be "more X" or "less Y".

"I came to the almost opposite conclusion that the obvious answer was that everything is tradeoffs and compromises."

Probably the smartest answer there is.

No this is not true. Definitely not smart either. Not everything in the universe is a nice yin yang tradeoff, this is a philosophical and scientific mistake. There exists in this world solutions to problems that are the most optimal.

When humans think in terms of design tradeoffs they are thinking in a domain where there exists no theoretical or mathematical technique to find the optimum. This does not mean a technique can never exist, it just indicates a lack of knowledge. The keyword is design. How do you design the best looking website vs. how do you design the shortest distance between two points? I would say for the later, we have enough knowledge and theory to say that the shortest distance between two points does not need to designed... it can be calculated.

Design implies alternative designs and tradeoffs due to the lack of techniques to determine or find an optimum. Calculation implies greater knowledge.

Most real world problems have constraints that render global optimums moot. Usually opportunity cost, but also risk, experiential components, satisficing ...

In fact, any interview problem with a global optimum is worthless for evaluating candidate's ability to handle ambiguity or demonstrate capacity for creativity, second order thinking, communication ...

Expecting candidates to lead with absolutism like in your post os a good recipe for finding Brilliant Jerks.

Dude, the entire field of Machine learning is an attempt to find a global optimum. It is an attempt to turn what is otherwise a design problem into a calculation.

To say that global optimums are moot or don't exist is categorically wrong. Global optimums and trade offs exist in equal measure everywhere. I value good judgment in both.

Catering to one extreme is unwise on every level.

>Expecting candidates to lead with absolutism like in your post os a good recipe for finding Brilliant Jerks.

First off I never said or implied this. My post is just saying that not Everything is a design tradeoff. Second off are you implying I'm a Jerk? Did you just call me a Jerk?

It used to be that 'it has to compile' but since so many people are using Python and Javascript I guess that is no longer true

> "What's the one thing you would not compromise on as you write code?" Its not hard to figure out the right answer to that one: "I never compromise. Must be perfect.

That is actually the worst possible answer I can think of. Pretty much any answer would be better than that. It demonstrates that the person either:

1. isn't honest or

2. doesn't understand basic engineering concepts

3. will be a dangerous, pain in the ass to work with

I mean based on that answer I'd say you're obsessive and likely hard to work with in an environment that requires compromise (ie: a startup like in this context). So I'd consider that a rather bad answer personally.

edit: Given it's a non-technical interviewer I'd say something like "I never compromise on making sure the business objective is met. I value code quality and documentation but sometimes trade offs need to be made especially in a company this size."

I've been hired to perform "technical-interviews-as-a-service" for a handful of non-technical founders / leaders. Has anyone partook in this format, either as leader, interviewee, or interviewer?

I really like problem solving and estimation questions during interviews, and I favor them over coding skills, etc. A question that I frequently use is how long would it take to evacuate a city like Phoenix, if there was a 24 hour warning that some dictator du jour is going to drop a nuke. It boils down to assessing a throughput of various modes of transportation: cars, buses, trains, airplanes, and yes walking. One candidate responded "by helicopters" at which point the interview was pretty much over.

Google used to do this kind of question but found it wasn't predictive of future perfomance.

I'm a good enough developer who has worked on enough projects that if someone tries to tell me that I need to show them personal projects I'm working on, then that ends the interview. I used to be a recruiter, I know how to conduct an interview and sit on the other end of that desk, and I know what I won't put up for in the workplace.

My personal time is my personal time. If there's a good work-related reason for me to put in more hours, fine. But you get none of them that you don't pay me for.

Another angle of this question is often to elicit what kinds of problems you have solved in your personal time, ideally technical, but not necessarily so.

I have had great conversations with someone who has built an automatic pet feeder that lead to the work.

It shows a different facet of one's personality. This, in turn may come in to play when creative problem solving is involved. Quite often even non-technical problem solving around the home can transfer to work

It can delineate the difference between a developer who sees what they do as a profession, vs those who started with it as a passion and continue to solve problems.

The guys who try to heal people with crystals are way more passionate about their jobs than the people who went through med school. Passion doesn't equal talent. I'll choose the guy who gets his job done on time and under budget all day long over the guy who needs to reinvent the wheel because he saw a blog post and thought the idea would be cool to implement on company time.

This is presumptuous of a level of trust that is not there.

There is no talent shortage, and if there is, it's of management, not the ability to write code. The difference between a good developer and a code monkey is that the first is self-managing. They have to be, to do their job. But even they can't do it without clear communication about the project---and cash, of course.

That the shortage is of management, not developer talent (if you protest that there is a developer shortage, I have bad news for you about where the shortage of management ability is), means that the interviewer is generally in the power position.

Ergo:

How great can the conversation be on such unequal terms? I'm not saying I haven't had good conversations with interviewers or interviewees, but it can't be expected to be a regular thing. What I really think is a smorgasbord of hunches, folk wisdom, and Alan Kay quotes---but I would never be so rude as to express such in an interview.

Buy the girl a drink first, sheesh.

Pre-pandemic I've found that in many markets there is a shortage of good senior developers. Good defined by those who can interview well in whatever the current in-vogue interview style is. Or more specifically those that could get jobs at Google, Facebook, etc.

I feel like the caveats prove my point.

It's more that you can't lump all developers into one category and make blanked statement like you did. Some developers in some markets aren't in demand. Some developers in some markets are absurdly in demand.

Reply on news.ycombinator.com