Hacker News Reader

Comment by rsweeney21 | Hacker News Reader

It's because in software engineering your work product is confidential/proprietary and work samples take a very long time to produce.

If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants. And it would be much easier to move from job to job.

Instead we have to simulate work samples through whiteboard coding, pair programming sessions, take home coding projects, coding knowledge tests, etc. We know that none of these methods are particularly good, so we err on the side of failing candidates when we are on the fence. They also take a long time to complete, so software engineers are reluctant to change jobs because they know how much of a pain it will be to go through interviews again.

It is much easier for product designers and product managers to pass interviews. You can see their work product.

At my company we're working on a way to create standardized coding projects that can't be faked so you can use them across job applications. If you are interested in this space, shoot me a message: robert at facet dot net.

View on HN

I have had good luck hiring developers by asking them to do a code review of something WE wrote (but framed as if a junior developer wrote it)

This helps us look for good signal around their communication skills, mentoring skills, empathy, reasoning skills and software development skills.

We used this review as a diagnostic - there are no wrong answers since we know they’re not aware of our style guide and conventions.

It worked for us. Maybe it could work for others.

I do something very similar and it works quite well for anyone except fresh grads (I've a different variant for them).

I pose a situation and introduce the v1 code. They look for bugs and propose fixes. Then I add on a new feature, and they look for more bugs. I've maybe 4-5 levels overall each having different classes of bugs, some of more or less subtlety. Fits on one whiteboard / single piece of paper in the end.

There are no syntax errors, just logic. Errors are in categories like: robustness, failure handling (internal or external), data/thread races, unforseen side effects, etc. The more experienced you are, the more problems I expect to be found. A "I don't know" is just fine (and often the following discussion is MORE informative).

I _really_ like this variant! In my experience new grads never seemed to have too much trouble with the exercise but then I do explain that it’s diagnostic in nature and use the exercise to see how coachable they are.

I'm using the same technique. Taking MRs from Junior Devs together with the Ticket description, reducing the scope a bit and sprinkling different errors or incomplete implementations here and there.

In the interview, I'm checking of problems that the applicant notices and see how he approaches the problem.

So far I've been satisfied with all hired who passed that test. And they liked not having to solve stupid whiteboard problems.

That is such a refreshing approach to the process. May borrow this next time I hire a developer.

Thanks for sharing.

> If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants.

I disagree, somewhat. In positions where I work with a medium or large team, any code of mine you reviewed would give you an idea of how the team leads want code to be written, not really how I as a candidate write code. I've often disagreed on design/architecture but was forced to go through with it because I really didn't feel like arguing and getting on the shitlist of a high-ranking engineer with influence in the company.

If you want an idea of how I would design something, it's probably best to have me write something specific to the job. I'm not talking a whiteboard red-black tree implementation, but something that is actually indicative of the work I'll be doing.

    Instead we have to simulate work samples 
    through whiteboard coding, pair programming 
    sessions, take home coding projects, coding 
    knowledge tests, etc.

Very small sample size but we've had good results skipping coding exams entirely, and simply talking through some scenarios with candidates.

- give them a hypothetical problem ("the page is loading slowly") and role play how they'd troubleshoot it - do they understand how different bits of the stack fit together? have they ruled out network issues before digging into the code? etc.

- "tell us about a tricky technical challenge you solved"

- ask them what makes a for good code review on a pull request / what makes good code review culture in general

- ask them about things the team did well at their previous role(s) and things they'd improve, and how they'd improve upon them. (not looking for "correct" answers or opinions - but they should have some opinions)

- what are some technologies/frameworks/languages you'd like to learn next? again there's no "right" answer. but if they don't have any - red flag, right?

- etc

This of course presumes that some minimal screening has been done to make sure they're actually a software developer; an initial Fizzbuzz screen etc.

At each step of the way we discussed relevant technical bits in a natural and conversational way. For example if they mention that they "optimized" or "refactored" or "fixed" something at a previous job then naturally we ask them for the details and usually wind up swapping some stories as well.

If they passed this conversational interview, we usually extended an offer to them within 24 hours or less. Sometimes we waited longer and they got snatched up by another company.

Of course this sort of interview doesn't tell you about their work ethic and how reliable they are and so on. But, neither does a lengthy coding assignment-based process either. And the "conversational" method gives you a sense of how they'll fit in with the team.

Considering how quickly good candidates are snapped up, doing "fast" interviews like this can be a big advantage for your hiring efforts. If worst comes to worst, and they don't work out, you can part ways. It's not like either side is locked into the employment agreement. Depends on your specific locale a bit, of course -- and at my current employer we're helped by being a remote team, so it's not like we're asking folks to relocate or anything.

> - give them a hypothetical problem ("the page is loading slowly") and role play how they'd troubleshoot it - do they understand how different bits of the stack fit together? have they ruled out network issues before digging into the code? etc.

I don't know. I think I'm pretty good at troubleshooting on the job (since I can be the one to troubleshoot & solve the trickier bugs) but thinking about doing it in an interview setting makes me nervous. It feels too fickle & up to the biases of the interviewer that day, not to mention that any serious production bug relies on knowing how the software stack operates & which things are relevant, all things that are ambiguous in an interview the interviewee would just be stuck grasping in the dark to try to figure out which scenario you might be describing.

The rest seems fine, but I'm curious about your experience with the live debugging & if you've considered cutting it.

    but thinking about doing it in an interview setting makes 
    me nervous.

I felt it went pretty well in the interviews and didn't seem to introduce any extra stress.

Personally, if I was being interviewed, I'd really prefer this to coding exercises as those really crank up the anxiety for me. (But of course everybody has different preferences!)

    not to mention that any serious production 
    bug relies on knowing how the software stack operates 
    & which things are relevant,

Oh, I think maybe I didn't communicate this aspect properly in my post.

There's no right answer. We're just looking for their thought process and how they might look up and down the stack to zero in on something. Of course they would need to ask questions about the architecture along the way. We do communicate all that up front - it's their questions that we're looking for.

    The rest seems fine, but I'm curious about your experience 
    with the live debugging & if you've considered cutting it.

Well it's only been a handful of developers, but I felt it was helpful.

I think that kind of exercise is good for distinguishing "recent code school graduates" who are smart and can write code for one or two parts of the stack, from more experienced devs who have experience understanding how all the pieces of the stack fit together in a production environment.

I'd really do well on these types of interview questions. I've spent a great deal of time troubleshooting the most difficult software issues for the teams I've been on. I'd rather not dox myself here, but troubleshooting is a pretty big thing I've done for the past 10 years.

Just today I discovered the biggest issue we've been encountering on our platform, and I'm not even in a developer role here. The developers didn't track it down since I got them involved, but here I am coming in and figuring it out, and it didn't even require access to the source code or how the platform works.

At the end of the day these types of issues come down to creativity and outside the box thinking than it does knowing how the specific software platform works. Looking at bugs and issues in a way that isn't subject to preconceived ideas and existing knowledge.

Sounds like a fun interview for someone like me lol

I've done these sorts of questions before and the point (more me at least) was two-fold:

1. Do they understand the basic machinery of a web application (assuming the job is about building web applications)?

2. Can they take a vague problem ("the page is loading slow") they might get from an actual non-technical user and drill down to identify what precisely the issue is? Is it DNS? Is it a proxy server/load balancer? Is their latency at the DB layer? How would you tell the difference between all of these?

I'll take high level architecture design and software coding problems over live debugging any day. Which is kind of my point. This kind of session is going to potentially weed out candidates with different strengths & a good team is a mix of a variety of strengths.

Surely an interview scenario is far less stressful than an actual outage though? Your performance in the real one is a lot more important for your job/career than the interview is.

Not necessarily. People put a lot of stock into interviews because they could affect not just today, but years of their future. Whereas nobody really expects to get fired if they can't resolve the outage in 45 minutes (instead of 3 hours), say.

Especially if you're junior and your current job sucks and you've never (yet) passed an interview for a better job.

This is counter-productive, but the brain doesn't always work how you'd want it to.

page loads slowly ... it is the database query man ... make sure to join the fields properly :-)

Yeah this is generally what I'm trying to lead them to.

If it's not a network issue, it's probably the database, right? Generally you've got an n+1 and/or some individual huge slow queries. I'd interested in knowing how the candidate would zero in on the database possibility and how they might narrow it down from there.

"tell us about a tricky technical challenge you solved"

I have so much trouble remembering previous challenges I've overcome. I don't ruminate on them, I just move on...

Then just do a few canned answers before getting into interviews, so you don't have to wait and think about it and just shout out the most appropriate one you can think of the list you made?

Yes, I have found that this question just selects for people who can recall well on the spot, or worse, can make up some a believable canned war story and articulate it well. As such, it is really testing a particular set of social skills which have minimal bearing on an engineer's ability to do their job.

    it is really testing a particular set of social skills

I need my engineers to have strong communication skills, so this is intentional and desired.

    ...which have minimal bearing on an engineer's ability to do their job

In positions I've held, communicating and remembering are pretty desirable qualities. For engineers and others. I need my engineers to be able to do those things. You obviously don't. That's fine. We'll hire different engineers. Everybody wins.

You don't need to ruminate on things like this but there is a lot of value in keeping a light dev journal of sorts about what you have done and the results.

Makes it way easier in interviews or ever just in a normal performance review where you can refer back to things.

Well, if I was your interviewer, I'd say "no worries."

I'd be fine if you had to skip a few questions like this and did well on the rest.

That's a fairly common interview question, though, so it's a good one to rehearse!

I've generally found more people flunk this sort of interview than flunk a whiteboard one.

Namely, if you've had only boring, uninteresting jobs so far, you might not have a truly interesting "tricky technical challenge you solved," and similar gaps around the other questions. Like, you can't name many of the ways a big, high-traffic site might slow down, because the internal tools websites you work on don't need caching or proxies or load balancing or...

Whereas tossing an algorithm question or two at them too to see if they can figure it out at least potentially screens for potential in a way this conversation doesn't.

So I always make sure we do both on interview loops, because I don't want to reject you just for lack of good past jobs, OR because of lack of studying algo problems recently (the good senior candidates crush the conversational one even if they're rusty on the algorithms).

     Like, you can't name many of the ways a big, 
     high-traffic site might slow down, because 
     the internal tools websites you work on don't 
     need caching or proxies or load balancing

I feel like it would be tough to spend any amount of time developing web apps, even if they're just internal tools, without hitting a basic performance issue like an n+1 query situation where you accidentally write a page with 500 queries or something. But maybe that's me... maybe devs are better than me these days. I hope so =)

There are of course innumerable network related reasons an app could slow down and we wouldn't expect a software dev to dive into them... I'd expect them to just be able to do some ultra basic stuff to rule out a network issue and zero in on the application stack.

Do medical doctors face this? Their work is confidential as well - patient data is highly confidential.

No.

As a software engineer for 20+ years, I've talked with my sister, who is an ER doctor about this often. Most folks outside of engineering think our interview processes are crazy.

Keep in mind, there's significantly more training that goes into the medical field, so there's more emphasis on the institutions. Where did you go to school? What hospitals or facilities did you work with? Your resume and references speak more strongly.

Other hurdles in software engineering hiring:

- It's both a technical and (often) a creative field.

- Most work requires teams to complete and there's very little insight into who contributed what into a project.

- You don't have many of the signals from other fields, such as published papers, law cases, or hard numbers such as sales, certifications, etc.

Long story short: we don't have good mechanisms within companies to measure engineering productivity, let alone measure programmers outside of an institution.

I'm reminded of Steve McConnell's intro to "Software Estimation":

"Meanwhile, the typical software organization is not struggling to improve its estimates from +/-10% to +/-5% accuracy. The typical software organization is struggling to avoid estimates that are incorrect by 100% or more."

> Most folks outside of engineering think our interview processes are crazy.

They do, but not for the reason you think.

A hospital administrator would be mad to hire some rando from the street who calls himself a doctor, who has no formal schooling in medicine, a dropped two-year degree in chemistry, but who reads a few medical blogs, on nothing more than 'They have four years experience working at a startup and a good reference and they can answer some second-year whiteboard questions about the lymphatic system.' [1]

Other professionals front-load the interviewing effort into a rigorous schooling, testing, and accreditation process, that costs hundreds of thousands of dollars and years of your life. Meanwhile, software engineers grouse about whiteboard interviews.

[1] Despite the highly dramatized performance by Leonardo DiCaprio as Frank Abagnale, that's not generally how doctors get hired.

One key difference is licensing. As a physician you have to take the USMLE Step 1,2,3 and then potentially other board certifications. We engineers don't have any of that.

Yes and depending on the field, you may have to do mandatory additional certification and training throughout your career.

We have certs in software engineering, but in my experience, they've largely failed as an indicator of qualifications. Some hiring managers see them as nice to have, but I've also met hiring managers who are actively skeptical of certifications.

We also don't have differentiation of software levels or skills. If you're a programmer, you could, in theory, go program anything from web to AI to robotics. You don't have to be certified to work in nearly any industry. And we don't have formal hierarchy of skilled work, such as EMTs, nurses and doctors.

We're a relatively young industry that just hasn't worked things out yet and, to be honest, in the United States, there isn't pressure by employers to get our act together. The engineers themselves are in charge often and we're skeptical of boards and authority. We have too many stories of folks making it big with no formal education. Those with the money aren't demanding change because the current system "works."

It's possible software engineering will someday evolve into a boring respectable profession, but right now it's still a creative field of fads, trends, rock stars and super stars (let alone ninjas and wizards).

> Some hiring managers see them as nice to have, but I've also met hiring managers who are actively skeptical of certifications.

To me they are a red flag. Someone couldn't get a good job in a bull market straight out of undergrad and felt the need to get additional training...

> We also don't have differentiation of software levels or skills. If you're a programmer, you could, in theory, go program anything from web to AI to robotics. You don't have to be certified to work in nearly any industry. And we don't have formal hierarchy of skilled work, such as EMTs, nurses and doctors.

It still exists, it's just that it's informal.

I imagine that amazon would still grill you in an interview even if you have one of their advance AWS certificates. I know those certificates are not the same, but I don't think it's about certifications.

FYI 'real' engineering (electrical, mechanical, civil etc. ) do have certification exams in USA and many other countries

Can you imagine?

Hi I'm Ed Barns I'm here for the interview of ER Physician.

QUICK Put on your scrubs we've got a patient here with a bone dangling out of his arm.

No you don't understand I'm just here to interview.

THIS IS THE INTERVIEW HURRY UP.

Some nursing boards pretty much have this; you get to a room with a medical actor that's behaving like a patient with X issue and you have to handle the situation.

It's also a big part of residency (having to treat patients with diminishing supervision from senior doctors). So it's not that far fetched.

You should write for Grey's Anatomy. That sounds like one of their ridiculous plot lines.

Medical doctors have licenses which include exams and all sort of other barriers before they’re a ‘real’ independent doctor.

I'd be curious to hear peoples thoughts if they would subject themselves to this sort of professional board and licensing for software. There's clearly a pre-determined set of information and skills that are fundamental enough to be tested, but it is also fairly creative and subjective at the end of the day.

With medicine there are probably like 4 ways to treat a bullet wound. If you go too far outside of that you're probably not a good doctor.

With software you can write a TODO app with a million lines of code or 7.

... Every serious trauma is unique and challenging in its own way.

Given the amount of people with CS degrees who fail the most basic interviews, it would be hard to set something good up. I assume universities are trying to solve the same problems as the hypothetical board setting up this test.

That, and the field moving so quickly and being so broad. I interview for ML. I assume interviews for front end are extremely different.

Disclaimer: I’m educated as a technologist not an engineer in Canada.

This exists in Canada for some areas of software development that falls under the requirements of being a licensed software engineer.

Many people, including CS graduates in Canada, think it’s ridiculous. The requirement for certification is pretty strict in that you need a computer engineering undergrad degree or a degree in software engineering from an accredited program. This is probably how most people would be accredited, but there are some other ways too. All of them, as far as I can gather, require formal educational credentials.

You then need to go through a formal apprenticeship while working under someone who is licensed. It’s probably an 8 year process in total.

Further to that, there is a limitation on what you can practice. There are different disciplines (e.g., mechanical, civil, electrical) and there are fields and scopes of practice within each of those disciplines. What this results in is someone who is licensed in, for example, embedded systems wouldn’t then be able to go and work on websites or back end databases.

The field is further divided into technologists and technicians that require less education than engineers and have much less autonomy and scope of authority.

I would likely take issue with whatever criteria people chose for judging someone's software engineering ability. Creating the best performing system is rarely anyone's goal (not that performance should be discounted,) they want to make something with the hard to quantify qualities of robustness, scalability, maintainability and expandability, to different degrees. They also often want to use languages or frameworks (or supporting software like databases) based on what boils down to fashion. Often the most desirable quality for candidates is an ability to learn and leverage new tools, given varied and shifting standards of good design, which again is hard to quantify.

Realistically, I think I'd not be allowed to practice with that kind of system. I suspect there's an entire class of programmer (maybe this is the dev vs eng distinction the article refers to) which would be excluded by a system like this.

Fundamentally though I think that system of high priests and laypeople who are barely allowed to code at all can't be sustained. I strongly suspect all professions will eventually require a little programming, and certification will be hard for those cases.

Sure, but that doesn't mean they're a good doctor. Most of us have had experiences with bad doctors. The exams they took could have been decades ago, but what does that tell us about how they've performed recently?

Bad doctor is relative. I expect that most doctors who have pass through that gauntlet can pass the doctor equivalent of fizzbuzz, which already is a lot different than our industry.

A doctor equivalent of fizzbuzz could be “what do you prescribe for a viral infection with no indication of a secondary infection” and from what I’ve heard there are a lot of doctors failing that one during the COVID pandemic by prescribing antibiotics.

They are highly effective at making patients stop begging for antibiotics they don't need. Patient satisfaction with their doctor has nothing to do with medical outcomes.

> prescribing antibiotics.

I'd hire that one over anyone pushing horse de-wormer, anti-malarial drugs, or homeopathic remedies.

Well, it's less dangerous than the first two, generally, but roughly equivalent to the third, for covid.

It's quite hard to be really bad physician - because Pareto principle applies to medicine. As far as I remember, 5 autoimmune diseases (of >120 described; ~4%) make up for 80% of total autoimmune diseases. My relative did research on kidney diseases and results were similar.

No, but they have a legalized cartel to make sure there's never an over-supply (that would lower the price for the consumer, we don't want that!) that also gatekeep credentialing.

Meanwhile in Software there are initiatives for "Gender X of Minority Y can Code" and "Everyone can be an Engineer after 3 months of BootCamp".

I disagree on 2 points:

> It is much easier for product designers and product managers to pass interviews. You can see their work product.

You don't see any more work of a product manager than of an engineer. What you see is always the finished product, and you have no idea who contributed what parts. If the overall product that can be publicly experienced is good, then some engineers in that team seem to have been doing a good job. You won't easily know which ones - and you especially don't know whether a product manager on the team really a lot of influence on this or whether the products vision and execution was mostly driven by engineers.

And the second point:

> If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants. And it would be much easier to move from job to job.

Sometimes employers can - because candidates have open source code. But experience has shown that these datapoints are not used, and employers still fall back to the default process of assuming nothing and doing whiteboard/leetcode style interviews.

And, about reviewing code from someone's previous job -- they might have had to write sloppy code because of time pressure

That's one advantage to working on open source. Your work is all just a click away.

Have you ever gotten a job from just showing off your open source work, though?

I was a committer to Elasticsearch. My name is thanked in the (very old) official book. I will still get asked questions.

No one looks at open source work, just as no one cares about your resume.

Other people I know have. Quite a few have leveraged their contributions to the D language into a much higher paying job than they had before.

Reply on news.ycombinator.com