Hacker News Reader

> All these "competency tests" are good for is catching blatant lying. Is this a serious problem?

Of the people I interview that claim they have years of experience, mastery over multiple languages, and expertise in various frameworks, a solid 80% or more can't pass fizz-buzz in any language they choose.

Claims made on resume or in person? On the seeking side, I've felt pressure to put every technology I've used even in passing on my resume, to satisfy "buzzword bingo" and get through the initial screen, but in an interview I would give a (truthful) answer along the lines of "I don't have a ton of experience, but I know the basics; enough to be confident that I can quickly pick up whatever else I need to learn on the job."

> Claims made on resume or in person?

Both. And it's not "putting every technology I've used even in passing on the resume" that bothers me. It's "I don't know how to write a loop or a function in any programming language."

Having administered over 600 interviews, I second this. Most people can’t code, despite degrees and years of experience.

> Is this a serious problem?

You would be astonished, absolutely flabbergasted, at the number of "senior engineers" who interview well, talk great about projects, but cannot demonstrate basic programming skills (e.g., convert an integer to a string representation, or write fizzbuzz).

How does this happen? What (apparently huge) swath of industry am I insulated from where this can take root? Is this just a bunch of wannabes after tech salaries? Or is this is a side-effect of industrial java programming, where their "job" was basically filling in boilerplate? (I am probably unfairly slandering java shops, it seems unlikely, but everything else seems even more unlikely)

It boggles my mind.

I think it usually involves 'helper' type folks. They end up on a project, and they 'help' the project get done. But the work they do tends to be just the boilerplate (or cut-and-paste style work), or just the gathering of test resources, or a surprising amount of work that you would assume that the PM or Manager would do, but instead is done by this person (organizing meetings, setting up checklists, emailing people about current issues/priorities).

This isn't to say that those people don't add value, but if they do this for a few years, they may find that their ability to do greenfield coding has significantly diminished, so much so that they struggle to write objectively simple code because they've fallen out of practice.

Source: I've been one of those people, and I've worked with many. It's an easy rut to fall into, particularly in Test Engineer role (what MS used to call SDET roles).

It boggles my mind too. I've conducted a fair number of interviews both phone and in-person, and I don't think I've ever encountered anyone remotely that bad.

I work in a non-tech company (bank), so I assume the general caliber of candidates is quite lower than what a bonafide Silicon Valley style tech company would see.

Would the typical candidate I interview be able to pass a DS&A leetcode medium or hard, getting the superoptimal solution on the first try within <40 minutes? Probably not. Maybe most of them wouldn't be able to do so on a harder leetcode easy problem either.

But most are still able to code up an acceptable solution to a more "realistic" in-person coding exercise we give them.

There's the concept of the "expert beginner" who works in a role when they're never challenged, so they quickly assume they're great at the thing when from a global perspective, the just stopped learning very quickly.

There's also the type that accidentally falls into a manager-type role way too soon, before they've actually had a proper chance to learn programming well. Then whatever basic skills they had atrophies quickly.

It's partially just a numbers game.

If you assume the top third of candidates only need 2 interviews to get an offer, the middle third need 5, and the bottom third need 15, then 15 / (2 + 5 + 15) = 68% percent of the candidates you interview will be from the bottom third.

This, of course, ignores every human factor but it is a starting point.

I must be one hell of an outlier, then, because of the dozens of people I have interviewed over the years, exactly one could not demonstrate basic programming skills. Most of them couldn't get much further, but that is a different matter.

You are lucky. I end up doing 2-3 screens a week. Ask any of the typical questions were you to google for interviews for Java/Spring/etc.. you get the same canned answers. Push any practical bits, and there are a lot of people who apparently don't know exception handling and some of the other more 'basic' programming things outside of flow control. Few things sadden me more than the senior developer who appears to have done junior coding for years.

> Why would a practical test be more effective than a discussion about prior work experience? Short coding tests under pressure are extremely unrepresentative of real work and I'm not sure homework-style interviews are much better. I personally don't want to spend an entire weekend on your test and I've dropped opportunities for that reason in the past.

These are good points, and the tests I have given are specifically designed to avoid these issues.

Personally I do this:

* Come up with a simple but realistic 1 or 2 hour task, and write up the request in plain English (do not write a tech spec). It should be task you'd expect anyone to be able to easily do on day one without any assistance.

* Include a short narrative about the end user that clearly implies a specific requirement or two, but don't spell out it out as an explicit requirement.

* Omit a small but important detail that any reasonable person would ask.

* Instruct them to provide a solution to the users problem, and ask questions if something isn't clear.

These are pretty simple to evaluate too:

* Does the code run? (i.e. can the candidate write basic code?)

* Does it meet the user's explicit needs? (i.e. can the candidate follow basic instructions?)

* Does it meet the user's implicit needs? (i.e. does the candidate write code to spec, or do they think about the bigger picture?)

* Did the candidate ask a question about that obvious missing detail? (i.e. does the candidate speak up, or will they make assumptions?)

You would be surprised with how many people don't have the ability to apply their ability to write code to solve simple problems.

Competency tests have helped me get in the door.

That said, they seem to work best when they're part of an overall interview process. I think it's a mistake to look at only the highest scores and interview those people, they should only weed out the bottom scores and follow a more typical (do they fit? can they learn it?) process for the rest.

> Competency tests have helped me get in the door.

How so? The door was already open if the company decided to give you a test.

My impression is that coding tests either offer neutral or negative feedback during an interview. It's not like a candidate can manipulate linked lists in a way that shows particular brilliance.

Nope, the first hurdle is usually some bs resume filter which removes a lot of competent candidates from consideration

The test comes before the interview, after speaking to a recruiter.

I imagine a good compromise might be a small piece of "contract work" where a company presents a small representative task as a project for a candidate to complete and the candidate actually gets some form of compensation for their time. There are definitely problems to this approach, but in a best case scenario, the result is a win-win situation even if a candidate doesn't get hired. At the moment every candidate that doesn't get hired represents a net loss for both parties (time for both and opportunity for the candidate)

> All these "competency tests" are good for is catching blatant lying. Is this a serious problem?

It absolutely is, there are plenty of companies who have no staff capable of verifying a candidates technical abilities

Agree on the test stuff. When talking to someone who can goes deep in details you mostly know that he's not a lier; then you have the trial period to fire him if he is.