Hacker News Reader

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied.

I mean, you're talking about a different thing than "job performance" here.

Job performance is a satisficing measure: you only need to get someone who's good enough to perform the duties you need of them. (Yes, even in software engineering.)

You're talking about ranking measures. Does it matter if someone is "the best" if they're still not good enough to do your job? Does it matter if someone is "the bottom of the barrel" if they are good enough to do your job? I would say no, in both cases.

IMHO, the hiring bar is way too high in most bigcorps; they care about vanity metrics about having "the best", when 100% of the work assigned to these hires could be easily done by people who are not, in fact, "the best."

The point of a job interview, in the abstract, is to filter out the candidates who aren't qualified to perform the job duties even after a few weeks of on-the-job training. This leaves you with a pool of people who are all qualified. At that point, you should either just hire one of them at random, or try to optimize for a second criterion, such as "is willing to take the lowest compensation to do the job" (which usually translates to "has the fewest formal qualifications to do the job, while still actually being qualified in practice.")

> Job performance is a satisficing measure: you only need to get someone who's good enough to perform the duties you need of them. (Yes, even in software engineering.)

Is that really the case for software? I mean, I'm very skeptical of the "only the best and brightest" nonsense, as I think it selects for the wrong people. But I'm also skeptical of the "eh, good enough" heuristic.

Right now I'm helping a friend deal with a code base that was built by an outside dev shop. It undeniably works; the business has been running on it for a number of years. And honestly, like most software, it's not a complicated app; mostly CRUD stuff. But overall it's a spaghetti snarl. Any given concept might be in the code, or in a config file, or driven by SQL data, or even driven by JSON blobs stuffed into the database, mutated by the main code base, and passed along to the JS front end.

Now that I can see the code, I understand why the dev shop needed so many people working on it, and why all features were hard to do. It's because when you use a lot of mediocre programmers, they all create work for themselves and one another. Not intentionally, but just because they aren't able to see how the differing approaches and sloppy work slowly diminish productivity.

So although I agree with you that a lot of companies hire on vanity metrics, I'm really skeptical that "whoever, as long as they barely qualify" is a better approach.

> Now that I can see the code, I understand why the dev shop needed so many people working on it, and why all features were hard to do. It's because when you use a lot of mediocre programmers, they all create work for themselves and one another. Not intentionally, but just because they aren't able to see how the differing approaches and sloppy work slowly diminish productivity.

To me, that sounds like a leadership problem at the outside dev shop. Properly supervised, properly supported, and with reasonable deadlines, mediocre developers can provide extraordinary results per dollar spent on probably 80-90% of dev tasks.

I'm familiar with the theory, but I've never seen that actually happen. And really, that just shifts the problem. If you need really amazing managers and really amazing senior engineers to make it work, you now just have a different hiring problem. As well a robustness problem, in that choosing a senior person badly means the whole team can go off the cliff.

Personally, I'd rather start with the amazing senior people, hire bright juniors, and mentor them into being amazing developers as well. As well as investing in tooling for non-developers. If a chunk of work is boring enough that a mediocre programmer can do it without risk (e.g., creating reports), it's perhaps boring enough that you can just enable the users (with, e.g., report builders).

> Does it matter if someone is "the best" if they're still not good enough to do your job?

Except we are hiring for a mix of performance and potential - weighted towards potential.

I don’t want to hire a junior engineer that can never make it to senior.

When I am only hiring for performance I hire contractors - and the bar is a lot lower.

Part of the idea of a universal standard in bigcorp is so that you can shuffle engineers between projects as they want to and as your strategy/needs change over time. And for cohesion.

Also I think there is a huge benefit for having generally capable employees, even if they are not directly utilizing many of their talents, and not counting the fungibility: they will make generally better decisions when reasoning about things outside their specific work domain and do better in ambiguous, not perfectly-specced situations. And in software at least, having general/complete knowledge and problem solving ability is absolutely vital if you want to be self-sufficient in diagnosing and solving problems, IMO.

I can't help but object to statements that begin with "The point of a job interview is..."

It seems like we tend to lose the basic reality of gatekeeping and social ritual in these discussions.

It's never going to be empirical. It's how you strike the balance between the empirical part and the social ritual part, and no one ever wants to discuss that.

But I've certainly sat across the table from hiring managers who I knew weren't going to choose me based on a cursory glance, or just seemed like deeply frightened people.

Agreed that interviews are both a gatekeeping and social ritual, but from the perspective of the employer I do think "the point of a job interview" should very clearly be to identify candidates with the highest probability of being successful in the role.

Sometimes when I delegate it costs me more time explaining and supervising than it would to do the work myself.

Sometimes I get back a perfectly decent solution, but in 5x the time I would have considered reasonable.

Most often I get back something that’s not amazing but looks fine, and works at first but turns out to have had some sloppiness or impedance mismatch down the road.

Sometimes I get back not only a solid and thorough solution, but a coworker who went way deeper into the topic than was necessary for the particular task, and is excitedly teaching me new things about it.

From certain people I get back an explanation of why what I’m asking will turn out to be a mistake 8 moves from now in the 10-dimensional chess game, and thing I actually needed instead.

The third group is technically “good enough.” They pull their weight, and we turn a profit on having them around. But if I can manage it, I’ll always try to get the fourth group for junior positions and the fifth group for senior (between those two it’s the same set of people at different stages of development). Those people are maybe 20% of my division, but they do 80% of the work, for what I expect is closer to 30% of the cost.

> if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel

And where you set the bar will depend on, roughly speaking, your prestige. Google can afford to be extremely selective when hiring technologists. Similar thing goes for Harvard and the Air Force.

> what a person works on (developers) outside of educational or work settings gives a ton of information on how they work as well as what they know

The usual counterpoint: nuclear engineers and surgeons tend to have hobbies with no bearing on nuclear engineering or surgery. Nothing wrong with a competent technology professional keeping it to work hours.

All teams should be selective. But if you don't have a Google-sized brand and budget, you need to develop a clearer understanding of your teams strengths/weaknesses and needs.

This understanding will enable you to figure out which hiring criteria do you need to maximize vs. which do you need to satisfy and where the bar is.

Also, by interviewing more effectively, you punch above your weight by spending more face-to-face time selling your company/team/role.

I love this point—the more time you can spend selling the candidate the better. And I think the more effective your interview process, candidates take notice and probably need to be "sold" that much less.

Great points - there are things we're trying to address over at Interview GPS. We're building a Wikipedia/Stack Overflow-style knowledge base of the best interviewer guides/rubrics: come join the cause at https://www.interviewgps.com/community/

We tag questions in 3 areas:

Personal Values: instead of "culture" we help you assess shared values. For example: do you value fun vs. achievement?

Competencies: does your software developer need attention to detail and critical thinking? Or creativity and collaboration?

Skills: what skills are necessary vs. what can be learned on the job? E.g., should you assess for React skills? Or assess for programming/debugging skills in general?

Also, P.S. you have to be careful/thoughtful when asking about what developers work on outside of work. Worst case, you end up breaking the law by discriminating against protected groups. I'd suggest only looking to that if their work history is relatively lacking and you're trying to develop a positive case for the candidate

Love your site—thanks for sharing! And +1 with regards to the potential legal issues.

By what they do outside work/school... I specifically mean if they work on software projects, using skills related to the skills they use at work. I've seen both sides of this and tend not to discriminate when someone has more experience vs. less... it's an offset, more important for interviewing someone out of a bootcamp, or with no formal education.

It really takes all types, so to speak. At this point, I've interviewed and met a pretty broad spectrum. The same goes for on the interviewee side as well.

Again... it's a balance... work history, personal projects (github activity), the pre-interview code challenge, as well as the technical interview are a matter of finding what's needed... someone can be less in one area and make up for it in another. To me at least, independent drive makes up for experience in a lot of ways, especially for more junior-mid roles.

> In the end, it's a crap shoot... Even their highest marker of validity is only about 50%.

Here's the thing though: all these different tests aren't completely independent. You can combine a couple of them and get a criterion validity value of up to .65.

See https://books.google.com/books?id=IvKYhrXAfRUC&pg=PA198&lpg=...

I completely agree that if you're designing an sort of assessment process, you should go through it yourself or ask other engineers to go through it so it's reasonably scoped in terms of expected time commitment. You can also use assessment tools that specifically limit how much time an engineer can invest in working on the challenge, which is also useful in showing how much progress each candidate made in a set amount of time. The problem with this approach can often be that the candidate feels like they're working against the clock though, which isn't "real world."

That's fair... a few days ago, got a solution (the problem is generally a cli program to transform a csv) that included a full API, data access layer and React ui. It was frankly impressive, and total overkill... turns out the request was unclear from management on our end.

I'm still surprised that almost nobody has used an actual library for csv parsing (naive solutions reading a line and splitting mostly), which to me results in so many headaches in practice.