I have interviewed 100s of candidates for software engineering positions
mastodon.cyborch.com
mastodon.cyborch.com
Some first level of the hiring funnel (HR, recruiter, or something) screens the candidate by looking at the resume and maybe doing an initial phone screen, and then this person gets dumped on the interviewer, and they 100% could be anywhere on that skill scale. So you always seem to have to start at "how do you turn a computer on?" and "write code that finds a substring in a string." It's really frustrating for the interviewers and I'm sure also insulting to actually good candidates.
I've never been in one of these businesses, but I bet if a hospital interviews a doctor, they can at least count on basic knowledge of anatomy, and if a law firm interviews a lawyer, they can assume the person has read a law book. We have no such guarantees in software.
If the cost of those guarantees is formal specialized training costing hundreds of thousands of dollars, I'll stick with the world of phone screens - speaking as someone who's been both rejected-after-phone-screen and rejected others after a phone screen.
We should consider an inexpensive, very basic "programmer's bar exam" to establish a (admittedly very low) proficiency baseline. I'm not asking for a test that gatekeeps software engineering only to people who can write a compiler. I'm asking for a reliable document that says this person can write a loop and a conditional statement and can reason about how it works. Is this a lot to ask for?
Or take this array of items and give me a map that uses the name as a key. I had one guy who could do a lot of front end work well but he could not pivot data to save his life. It was super weird. Like Derek Zoolander ambi-turner weird.
> [...] but I bet if a hospital interviews a doctor, they can at least count on basic knowledge of anatomy
Does the same situation arise if you pre-filter requiring a CS/EE degree from e.g. a top 1000 university?
MD degrees are highly regulated, so you would need some stringent pre-filtering to make both situations comparable.
Otherwise, I guess it is natural to find a lot more variability in candidates?
I've worked with a couple folks from MIT - which I imagine is "top N" for a pretty small value of N for most anyone - who knew a lot but could build very little quickly or effectively.
What you get graded on in schools for CS or even SWE is pretty different from what you get graded on on-the-job.
Funnily enough, one of my best jobs in tech included an interview with who would become my hiring manager. (I don't think the company had actually figured that out at the time as they were writing the job description for me.) I don't think I had ever had an interview that I just couldn't read like that one. Years later, after he was no longer with the company, I sort of asked him if I had really bombed and he basically laughed and said he was always tough on interviewees. (And he was always really tough to read.) It also probably didn't help that I hadn't had a real interview in at least 10 years.
This, plus grades and class rank. And there is a series of interviews, obviously.
For a young associate, grades/class rank are a decent proxy for how hard working they are and whether they have an aptitude for the basic practice of law.
All of this combined with seniority not for coding, but for the mercantile aspects - if you formed and sold a company then you are senior, otherwise no.
I don't mind it much. I don't have great social skills but I don't mind answering simple questions. If someone is insulted by simple questions, they're probably gonna be a bad mentor to junior devs, right?
The point is that, if they can answer very well more than 75% of your questions, you increase the difficulty, otherwise you decrease.
And the better they answer, the more you try to challenge and argue them.
What I do hate however is leetcode. I'll avoid that completely as an interviewer, as I feel like it would only help me determine whether they are the type of people to spend hours on leetcode.
I'll want to try to understand their passion to build and produce a product that solves something and provides value for their customers. Leetcode culture is contrary to that to me. To me leetcode culture is something where you play the game to increase somewhat irrelevant skills to get to a prestigious company to optimise for your total compensation. Which is fine, but it's not something I think that would be great for me to try to find best members for my team.
I haven't looked for a job in a while, but I'm bringing this up because in my experience when I was the candidate I remember people being quite rigid in their questions, for example questions being predetermined, which meant they ask me non-sensically easy questions, which considering what kind of questions I had answered before it would make no sense that I wouldn't know answer to those questions. So a lot of time-wasting.
Or in the other direction they could be asking similar questions in similar difficulty and area of topic, where someone clearly shows they no experience with, instead of trying to understand all talents of the candidate.
Simple coding questions are invaluable at filtering, but of course do not guarantee the result. In last 3 weeks I interviewed 3 candidates, two of them failed to write 40 lines of code based on written requirements. One of them even used Copilot.
"I just want to be left alone to program" seems like a perfect fit: will do the job, will not ask for anything.
but let's not derail, "no i dont want to go to company parties" = "doesnt want to socialize" - "doesnt want to submit to the company" - "will probably be a hassle to work with" - "can't communicate" - and so on.
again, think hard about a different career angle in tech if you dislike these things, you might enjoy it more
I've once been rejected in an interview because I found a solution that the interviewer didn't think would work, spent half an hour trying to prove me wrong rather than trying to expend the smallest effort to understand - even after I proved it was correct. I was quite glad they showed me a bit of the culture though
While this dev you encountered might have had no idea what they were doing, a most charitable explanation is that they may subscribe to the idea that the "unit" is a business usecase within a single service (often a 1:1 correspondence with an endpoint, but sometimes even more than one, e.g. a form upload with a separate signed upload step to a 3rd party service).
FWIW the concept of small/medium/large as presented in the Google testing blog is a much more useful denomination.
What is a component? A global variable, a pure function, a procedure, an endpoint, a business usecase, a microservice, a microservice fleet?
What is the whole, if not the same list of questions just asked in backwards order?
It gets worse: Once you arbitrarily come up with one definition that you stick with in your project, does the definition you chose leave little to no room for interpretation? Do all kinds of tests that are worth writing fit neatly within those definitions?
Technical terms that are so loosely defined we can do without, for only two things ultimately result from their use: Apathy, or madness.
That’s what I see interviews as: I’ll show you my culture if you show me yours.
There’s a lot of fuzzy factors like willingness to admit lack of knowledge, eagerness to learn, ability to cooperate.
To be fair, it's our own damn fault. "Poor stewardship over professional standards" is a succinct way of putting it.
The reasoning comes down to poor training, lack of standard competency tests (which is a stable goal to train for), and probably heavy disagreements about what a candidate should be capable of as a junior/mid/senior/principal, or at year 1, 2, 3, 5, etc. of their career.
Communication of these standards over blog posts and what you can find via search engines just won't - and hasn't - cut it for raising the competency of the industry as a whole. If you are competent, you can publish a book which becomes popular, but there would still be disagreement about what books are important.
As an example, Designing Data Intensive Applications is probably one of the more well-known books, but it's not a requirement to read that. The reason it's not a requirement is that you can meander through most jobs by just googling stuff and no one denies you a job if you can't talk about its content.
This problem has gone on long enough that most of us can't give a comprehensive list of what the "basic knowledge" is that we need. Most of us could try to make our own lists, but we'd be much less confident that that list should be the standard because it hasn't received any critical review and approval by a large sample of people.
We need to get out of this free-for-all thunderdome. We do that by getting the companies aligned on what they expect. Then they simply don't hire people that don't meet those expectations. Then we can start writing competency tests towards meeting those expectations and simultaneously develop training material to pass those tests. Most importantly, those standards cannot change too quickly otherwise we end up with the same moving target situation we have now.
You give me the training material, I pass a test, then I get a junior job and continue training for the expectations at various job title levels. It should be that simple.
Hopefully not as many, or as bad, but still.
On the other hand, if the person is annoyed or frustrated by the idea I don't blame them.
* Caveat for people with high levels of experience going into more senior job titles.
I've also done 100s of interviews (small and large companies). What I learnt:
- CVs are useless except for leveling the candidate (which is done for me),
- CVs are dangerous because they create a bias ("looks like an amazing candidate!" -- an then you get someone who is good, but not amazing as you expected and then you rate them lower than you should -- I intentionally skip the CVs as much as possible),
- majority of the people will solve a simple problem at the algorithmic level, and then fail to code it up,
- the best value is to give the candidate a SIMPLE coding question which produces a decent amount of code (25-50 lines), and observe them while solving it (their approach, bugs, etc.),
- the signals I'm looking for: (1) can the candidate code fluently, (2) do they approach the problem in a systematic way, (3) can I communicate with them without friction.
I am not sure what you mean by this. This is a question I ask and I’m concerned there’s something I’m missing here. Could you elaborate?
> I intentionally skip the CVs as much as possible),
That seems very odd. How do you comb through and make an initial choice of who to interview then? Also, seems unfair to not look at the work they’ve done over all the years.
Personally, I’m a GitHub person. Show me what you’ve written and how active you are.
> I am not sure what you mean by this. This is a question I ask and I’m concerned there’s something I’m missing here. Could you elaborate?
Easy :). Take me: I hate Go and think Java is the best programming language there is. (My daily drivers today are C++ and Python.) NodeJs should've never been invented. I'm pretty opinionated when it comes to this. I'd still accept the position where I have to code in Go (been there and it was OK for both sides), but I wouldn't keep my mouth shut during the interview -- I wonder how that would go with 80% interviewers.
> How do you comb through and make an initial choice of who to interview then?
Sorry, I didn't put it explicit enough. I don't do the initial selection or leveling. This is done for me. The CVs are useful for this. But if you can avoid reading the CV (e.g. assessing only the raw skills), then you should.
I think this is unavoidable. Teams have cultures (I don't mean ethnicities, though of course there is some overlap), and ill-fitting candidates will struggle even if they are hired. Or they'll change the team and stress someone else out. As un-PC as it sounds, probably small, values-aligned teams are the easiest/most comfortable to work in, even if that means individuals aren't exposed to more diversity.
> the signals I'm looking for: (1) can the candidate code fluently, (2) do they approach the problem in a systematic way, (3) can I communicate with them without friction.
Isn't this a sort of cultural preference on its own? You're looking for (and prioritizing) people who can code a certain way and communicate with you a certain way. A brilliant programmer from a different code style background, who's maybe an introverted poor communicator (or just have a different communication style) and codes in nonstandard (to you) way might not pass such an interview. That's fine... they probably wouldn't work well with your team anyway. But IMO that's the point of cultural fit interviews, to respect both your team and the candidate's time upfront by evaluating not just their technical ability but how well they'll fit into your team.
The "signals" you're using are just a masked variant of that. Maybe (as an example, just an extrapolation) you prefer async communications with clear, simple assignments -- like self-contained Jira tickets -- rather than group architecting, pair programming, or whatever. But those are cultural values.
If you ignore all those signs and forcibly include someone, sure, you might increase diversity on your team... and stress for everyone, including the new hire who finds themselves alienated from the rest.
> Isn't this a sort of cultural preference on its own? You're looking for (and prioritizing) people who can code a certain way and communicate with you a certain way.
Fluent coding doesn't imply style or something non-measurable. You can say if someone is fluent or not even if they are using a programming language that you don't know. I interpret it as: do they struggle while converting their idea to code, or not. Not how they structure the code, etc.
Same for the systematic approach to the problem. If their explanation is all over the place, or failing to uncover the important bits of the problem[1], bias doesn't kick in here. We can argue about the appropriate layering of the solution (the level of detail). That's where different people are different. But you can see if their strategy works or not once they put it in code (i.e. maybe they should have put more structure in their approach).
[1] (I: "Code me a string invertor". C: "Yes Sir! Here you are." I: "That's not what I had in mind.").
> A brilliant programmer from a different code style background, who's maybe an introverted poor communicator (or just have a different communication style)
Well, if I can't communicate with them (there's high friction), I shouldn't work with them. This is a bad fit by definition. They might find a better fit at another company.
Honestly, I think a lot of it is akin to medieval quackery. "I've treated hundreds of patients with leeches and the large majority of them eventually recovered - you should listen to me!"
I wish that we, as an industry, would spend less time reinventing interviewing from first principles. There are decades of research on how to interview people effectively. I highly recommend Schmidt + Hunter's meta-analysis (https://www.researchgate.net/publication/232564809_The_Valid...) as a starting point.
From my read of the research:
1. Use work-sample tests. Yes, they're annoying, but they're high-signal. Take-home tests, online tests, in-person tests, trial periods - there are plenty of options here, but it seems clear this is the most effective way.
2. Use structured interviews. Engineers love to just have loose conversations and judge people based on that, but structured > unstructured. Plus (just my guess here), I suspect unstructured interviews leave a lot more room for bias.
3. Plenty of techniques are effective on their own (eg references), but add minimal effectiveness when you're already doing a more effective test (eg work sample) and are therefore pretty much wastes.
I'd argue that you're basically pushing the workload onto the applicant. Now, sometimes it makes a lot of sense to the degree that the applicant can just share a work sample with you--GitHub, published report/paper/other writing, a video of a presentation. In fact, I've hired for non-developer roles where some sort of work sample is pretty much non-negotiable. But recognize this creates biases (and acts as a filter) where something needs to be created from scratch.
Everyone feels pressure differently. For some devs a job interview whiteboard can feel like trying to code with a gun to your head.
I've known brilliant devs who froze under the pressure of an interview. And other brilliant devs who needed to meditate on something, or sleep on it, before coming back with the optimal solution.
I've worked with devs who tended to just run with the first solution that popped into their head, with no patience to mull over a variety of possible approach looking for the simplest one. These devs are fine for straightforward tasks, but become a liability when designing or writing foundational core code.
Yeah, this is me. Not that I'm brilliant, good side of solid is maybe a better description. But I freeze on whiteboarding and l33tcode. I taught myself binary search and insert sort in high school, but ask me to write insert sort in 40 mins with someone looking over my shoulder and judging and I'll struggle.
Every job I've gotten has come from a take home test.
I'm guessing it's far less prevalent than some people seem to think, and that unfortunately, there's no good general system for solving this.
The other possibility, which I don't know why shops don't offer more often, is a contractor position. I've never taken a job that I wouldn't have been happy to work as a contractor for a while first if they asked, and two of my best jobs start out as a contractor.
I personally dislike "whiteboard" testing as well, and I think it's usually very ineffective. I think some form of work-sample is best, in my 15 years of interviewing we'd usually do: one "open ended architecture" question (which can involve a whiteboard but usually only for diagrams, not code). Then a 1-2 hour programming task, on a computer with full access to the internet.
> [...] I don't know why shops don't offer more often, is a contractor position.
This is usually a completely different position, from many perspectives. It's usually a different budget for the group hiring - they can't easily convert "salaried" positions into "contractor positions".
It's also, in many jurisdictions, a very fraught legal situation. Contractors in general have different rights, but if a company is shown to hire contractors and treat them the same as employees, the contractors can often sue to get the rights of any other employee. This makes sense - it's so companies can't use contracting as a way around whatever legal regulations exist on employment. (Whether those legal regulations are good or not is a different question - but for sure you don't want there to be loopholes around your laws.)
So no matter how predictive they are, it's ineffective to use them because you'll identify good engineers... who will never want to work for you ¯\_(ツ)_/¯
That said, I disagree about lack of any exercises. I think practical but not too time-consuming take-home assignments are nice and not too annoying (as long as they’re not the first step! That would be wasting candidates’ time.), and I’ve rejected tons of candidates based on the results of those, which is different to the author’s experience.
Overall, I recommend people to really think about who they’re looking for, and heavily adjust their process to that. There’s no silver bullet, and the right process will vary greatly depending both on the seniority and the skillset you’re looking for.
Even worse, they might not be able to communicate the actual job requirements correctly, if they even understand the requirements and necessities themselves.
Even worse than that, they often aren't as strong socially as they believe themselves to be. Being able to establish a comfortable atmosphere to connect with someone quickly is rare skillset within the human population, let alone in specific domains.
There is a PRESUMPTION that people in the interviewer seat or management position can and should do this but honestly, after a few decades? I don't see it.
We should be working to solve the "interviewer" problem first, not the other way around.
What's your favorite editor, what language do you like using for scripting, what's your favorite Linux distribution, do you have a homelab...
We were having a decent back and forth and getting along, and then he hit me with the final question, "we really aren't supposed to ask, but what do you like to do in your spare time?"
I mentioned that I had a 2 and 3 year old and joked that I didn't have much free time anymore, and said I liked working in the garage with my hands to get a break from computers.
I didn't realize I gave the wrong answer until an hour or two later (based upon the rest of our conversion and his questions). It would have been the right time to show a few personal projects on GitHub.
Leetcode challenges are a waste of everyone's time. If you are a competent developer then you can google for algorithmic solutions if you really need to, but honestly how often does the need come up, and is googling for suggestions really the hardest part of the project ?!
The only thing that doing well on these type of problems proves is that you've been willing to put the practice time in to get good at them.
I agree with the author on having a plan built for what you're looking for. I don't like the general "culture" questions because they're quite biased ("oh you're reading on HN! Me too!")
I like behavior questions ("tell me about a time when") with a reference scale that explains your. It's not perfect, but it seems slightly more data driven than the rest I've experienced with. I take lots of notes and try to base my verdict on these. I hate leet-code, except at screening level, to validate that the person knows how to type on a computer. I like somewhat job-related exercises for more senior people, code-reviews, and with junior trying to teach them about something new you taught to multiple people and see if they catch it. I betcha half of the people here hate all these methods.
In addition to what you're looking for, I advise trying to see what you miss with people you currently have in your team. (eg if they annoy you, find why and look for what they don't have).
Generally what I want to learn about a candidate is "are they good at understanding plain-English requirements, can talk through their thought process as they problem solve, convert that solution into something that resembles working software, and we can have a decent conversation about what was built." As it turns out, whiteboard-type algorithms and system design questions are very good at this.
Asking if someone keeps up with tech news (and call it a red flag if they don't) is just a way to keep your candidates within a certain social bubble. Which, honestly, is preferable if you're a < 50 person startup. But this doesn't scale. Plenty of software devs at IBM, Amazon, etc are just clocking in and out of their shifts and do a fine job of it. Not everyone needs to make career advancement in the tech industry their life.
Why? Why can't I have a moment to think and focus without talking?
Obviously the expectation here changes on leveling--with juniors if they just code in silence before explaining their solution at the end, I'm less concerned. But if a senior just starts coding on the whiteboard I'll interrupt them and ask them to explain what their approach is, in English, before coding it up.
... which is typically written after hours of understanding the problem and then minutes to hours of writing out the document. Not 5 minutes after reading a problem statement.
Not sure I've ever had to do that on a whiteboard or under pressure though. Also don't think being good at doing that correlates with writing good documentation
If you're afraid of making the wrong decision then if one person makes 1/5 bad hiring decisions then 3 people make 3/5 bad hiring decisions not 1/125.
It’s all pretty easy if respect for the human is genuine, and destructive for all when faked.
No, no it won’t. I’ve done 300 interviews when i stopped counting and I’ve had all sorts of people who talk a good game but absolutely can’t code
Whenever I start thinking the problem is the candidates I start asking myself “why?”. Why we came to the conclusion we did, why they applied, why we need more people on the team, etc.
A person who "cannot write a loop" will not be able to earnestly hold a conversation about programming.
I asked very basic coding questions that didn’t require “advanced” algorithmic knowledge like DP which are popular at some shops
Maybe it is a test as there is an evaluation, an assessment of fit and ability of compliment the team’s current skills and way of working. That evaluation is uncompromising, not an “easy pass” by any means.
I can't, so far, think of anyone who faked it past my interview teams over the years. It's not that hard to figure out if someone can actually code, has a sincere interest in it, respects others, craves improvement and measures their work by the value created. I really don't care if they test well.
And, like I said, the bar is actually very high to join my teams. But my teams aren't prototype or manifestations of any theory of teams -- it's just folks getting the work done and enjoying it along the way.
A team of different folks will need different things by definition and still be valuable individuals, each different, each worthy of respect.
My interview questions eventually settled to having the candidate bring in a sample of their code and then teaching me how it worked.
We'd talk about what it did, what they thought they'd change to make it better, and so on.
Learned a ton really quickly and didn't waste anyone's time.
The last tech job I interviewed for they had me spend a lot of time working a sample application. When I thought about how many person hours were going into that across all the candidates, I immediately felt the company didn't have a sense of efficiency. Sure, it didn't cost them, but the fact they were fine just wasting time like that was telling.
Didn't get that job. But got a better one. :)
You can get a candidate that nails the tests and interviews. Meets the team and first impressions are good.
Then 2 months in everyone realises the guy has a tantrum issue and is just an awful person when you get to know him ...
Do it that way.
Whiteboarding interviews are a good way to make the applicant feel bad about themselves so you can lowball them in salary, however
On the other hand, the inability to write a nested for loop is a reliable predictor for a bad hire.
Clicking buttons in jira alone isn't going to go very far.
What were you saying about proficiency?
1. https://thedailywtf.com/ I'd link the actual post but it was over a decade ago and
How do you make sure you don't hire these people?
Why are you hiring the guy if the job is that easy? Just because someone can do leetcode questions doesn’t mean that they can design a system or even design an API. That actually requires creativity not memorization.
The list reverse (or anything similar - count word occurrences in a list) doesn't qualify someone for the job, but is a good way to quickly tell if someone is lying about their experience, which unfortunately some do, especially for low paying contractor jobs. For stuff this simple I'd expect people to just write it down in 30 sec once they've asked any clarifying questions (such as can I use list.reverse() - no).
Here’s the problem though: most software development is done alone and most software engineers are introverts. They do not like to be put on the spot and be the center of attention. When you ask them to do things under pressure (which basically means your company is always having fires to put out - management problem) they can’t think straight because half their mind is focused on how you’re judging them because the interview matters to them much more than it matters to you. You’re not giving them a fair test.
>> How do you make sure you don't hire these people?
> I simple whiteboard test such as reversing a list will do the trick!
Agreed. I did a lot of hiring interviews of experienced embedded C programmers. A few of the applicants could not read two bytes from memory and combine them into a 16-bit integer, no matter how much handholding they got.
But they had good resumes and interviewed well other than that.
So some kind of check of their actual technical skills is valuable.
We spent a much larger portion of the interview having them explain things like "how would you implement a firmware upgrade architecture where the power can be lost at any time during the upgrade?"
But I always warn first that there's no problem if they don't know the answer so they know first-hand they can say I don't know.
Very easy to detect bullshitter that will invent stuff or those on the first level of dunningkruger.
And the best moment of my life was when I interviewed someone who had all the answers and the explanation of why it was working this way. I told hr that we had to hire him in a higher position than me!
-------------------------
> Anders Borsch (@anders@mastodon.cyborch.com)
I have interviewed 100s of candidates for software engineering positions.
I’ve done take-home tests, in person challenges, pair programming with the candidates.
All of them were awful experiences for me and especially for the candidate.
I can only think of a single instance where a code challenge exposed a poor software engineer and I could definitely have made the same assessment just by talking to them.
Lately I’ve stopped doing any software or mental puzzles.
I don’t do any of that when I interview designers or QA people or HR people, so why would I be particularly toxic towards software engineers during the hiring process?
Instead, I actually read their resumes (which is significantly quicker than doing interviews, asking them to repeat the same information), and then I ask them questions like:
- Where do you get your tech news?
- How do you learn about new technologies?
- What do you most appreciate in your coworkers today?
- What is a perfect workday like for you?
I specifically avoid trap-style questions like “what is your greatest weakness?” or “why are you leaving your current job?”
I recommend that you make a plan for what you want to learn about the candidate, e.g. “are they good at acquiring new skills?” or “do they share the same values as the team?” and then structure the interview around that.
Be a non-toxic manager. Make your company look good during the interview process. Get better candidates.
Modern Wordpress hosts do both, and it's usually very very quick to load.
But basically, it's an essential part of a modern Wordpress stack (lol, what an oxymoron, I know), not an afterthought. It's not something a lone-wolf admin using a generic Varnish/Nginx config should do, but part of a large a community-driven effort to improve Wordpress performance. These plugins are very well integrated into the Wordpress base system and cache many parts of it.
This isn't so much to defend Wordpress (whose ancient architecture requires the use of these techniques), but to express surprise that a modern software like Mastodon doesn't have this built-in.
https://docs.joinmastodon.org/user/run-your-own/
Many people run theirs on a small VPS, raspberry pi, etc. and there are a number of 'paid hosting' providers for Mastodon servers too.
In practice, this all means yes: unlike a blog, it is a complex web application, and caching and whether a CDN is in use are up to the person hosting it.
Mastodon can also be fairly expensive to run, depending on how many images/videos people upload, traffic they bring to the instance, etc.
In hindsight, I should have used an indirect link via mastodon.social or something, apologies.
Seems wiser to invest that time in health & entrepreneurship.