It took me about 6h to develop and iron out the edge cases.
My take away from this is that:
I will not do take home assignments for free if they take more than 1h of my time.
Apparently the approved HN method is to arrange the resumes on a dartboard, throw darts at it, and commit up front to hiring whoever you hit unconditionally for at least two years without no possibility of having to fire them. Or something like that.
Setting limits on what you will do is good, and ghosting is bad, certainly.
But in the end, you're going to have to do something unfun to get hired. There's no way around it. I know we'd all love to live in a world where Google one day just logs on to LinkedIn and is so impressed by the sheer brilliance of our awesomeness which just somehow oozes through our listing despite the fact we didn't have to do anything to demonstrate it, as it's also unrealistic to expect programmers to have a lot of publicly-available demonstrations of their skill such as open source projects (forgot to mention that one!), that instead of hitting you with a job interview they just lead with a job offer for their best position, but that's just not going to happen.
I've done exactly this, and everyone seemed happy.
Most contracts I’ve had will actually state “without manager approval” but that’s just as bad in many instances.
[later] Actually this became illegal in Illinois at the beginning of this year, though the protections don’t extend to clients or competitors.
It's unenforceable, according to everyone I've interacted with in the state :)
"The solution is this, but I'm SURE people wouldn't like it, so you all are complaining too much"
What was the rate you chose?
I've seen multiple comments in past HN threads of programmers complaining about any after-hours coding homework (even if paid) because "they have kids and don't have time or energy for that".
Probably the only suggested interview method that gets the fewest complaints from candidates is the "let's have a casual chat and just shoot the breeze about past programming projects". The problem is that you get complaints from the hiring managers that it's too easy to bullshit and is the weakest signal to determine competency.
I agree with gp (jerf). Ultimately, candidates just want to be hired without demonstrating skill. E.g. the college diploma should be enough. Or the resume should be enough. Or just low-stakes conversation should be enough. But the hiring managers don't want to do that.
Hiring managers need to actually spend time constructing a process that has questions (and if needed, exercises) that specifically determine whether a candidate has the specific skills needed for the job at hand. But that is actually fairly hard to do right, and it's easy to just grab some semi-random "take home exercise" and hand it off and call it a proper assessment. A proper technical interview will have increasingly specific questions that will both assess skill level and a person's ability to communicate with the team. If there is an exercise, it should reflect the specific skills the person will be expected to use in the job at hand. And bonus points for flexibly adjusting between in-person or take-home depending on the interviewee's strengths and wishes in that regard.
The problem is you're using your own perspective of "reasonable middle ground" and thinking that other candidates would agree with you. Many would not.
Here's an example of "you shouldn't have to go through that crap when your resume clearly states you are capable." : https://news.ycombinator.com/item?id=11596553
So regardless of any "job-specific" skills assessment you come up with, you will still get complaints from many because they feel their resume of experience, (or github portfolio, or college degree, etc) -- should already prove that they know how to code. This is not an uncommon sentiment! E.g. "Making me demonstrate my coding skill is an insult when you could just look at my resume credentials. Does a pharmacist show how they dispense pills to get hired?!?"
Or variations on your "reasonable skills assessment" with creative exercises such as "here's an existing bug ticket for a code defect; show us how you debug it" ... still has some complaints of "demonstrating my coding skill is unfair because I get nervous in interview situations because there's an imbalanced power dynamic between interviewer and interviewee."
Ok, instead of interview because you get nervous under pressure, here's some coding homework. Of which this "reasonable" alternative has pushback from some comments in this subthread.
Various threads about "hiring is broken" is evidence of the ever elusive "reasonable middle ground" that we're chasing after.
I don't have a lot of sympathy for this argument. If they have time to take half a day to go to an interview, then it doesn't seem like it should be much different to take half a day to work on an interviewing problem.
But a lot of companies use the multi-hour coding homework as a preliminary filter before asking the candidate to take the "next step" to the half-day onsite interview.
The coding homework is asymmetric work because the company doesn't have to expend their existing employees time on prospective candidates. In fact, a lot of companies let recruiters coordinate the coding homework so their existing employees spend the least amount of time on it.
The onsite interview is more symmetric effort because both the interviewer and the candidate have to expend time together. There's a bit more commitment on the part of the company doing the hiring.
I'm not complaining. Just stating how the different perspectives play out.
if i interview at 10 companies suddenly a lot of vacation time is gone.
so i am not going to take time off until i am reasonably certain that i want this job and can get it if i pass.
likewise i'll be willing to take time off for a take home assignment only when that is the last test i need to pass.
if companies want the assignment before they'll even look at my CV, then that's to much.
steps should be:
look at the CV to see if the candidate is basically quslified.
talk to the candidate to get to know them, answer any questions, agree on salary and benefits.
once that is all sorted, bring on the coding test.
It took up a lot of my week, though, and I know it would be tricky to be interviewing at multiple places at the same time like that. Others have mentioned portfolios, which is an idea I think makes a whole lot of sense.
…
Now, I want to eat pumpernickel and am shaken by wanderlust in the hinterland.
AFAICT the null hypothesis has yet to be disproven in tech hiring.
Actual hiring results look a lot in abstract like dartboarding after trying to filter out the obviously unfit applicants, and while people can reasonably disagree on how hard to filter, not at all seems like a good way to put yourself out of business.
In any case, the dart throwing method can't be criticized simply because it produces lemons, because no hiring process is perfect. Every company from top to bottom makes bad hires. So you've got to compare the overall results, which will contain both good and bad.
One big advantage of the dart throwing method is that it will randomly select good people who tend to be overlooked by hiring managers because they don't interview well.
Yes, now your process is indistinguishable from existing hiring pipelines. The darts happen to be weighted in all sorts of ways you don't have full insight into, but hey, so are literal dart throws.
> In any case, the dart throwing method can't be criticized simply because it produces lemons, because no hiring process is perfect. Every company from top to bottom makes bad hires. So you've got to compare the overall results, which will contain both good and bad.
The problem is not that it makes some bad hires. The problem is that you will make >90% bad hires if you don't do filtering, and once you bring a few of them on and word gets out you will be making net >100% bad hires by this process.
How so, when there are no interviews? Everyone seems to take interviews to be the most important part of existing hiring pipelines. Interviews are certainly the most time-consuming and expensive part of existing hiring pipelines.
> once you bring a few of them on and word gets out you will be making net >100% bad hires by this process
To be clear, I was only entertaining the dartboard selection part of the proposal and not "for at least two years without no possibility of having to fire them". You should definitely fire people who can't do the job.
I think you're also assuming that if "word gets out" then you will only attract more bad candidates, but it will also attract more good candidates who hate the existing hiring pipelines, some to the extent that they even refuse to apply for many jobs.
>do something unfun to get hired
I don’t know of any other industry who hires senior people the way we do outside of performing arts.
I personally don’t mind putting together a portfolio the way most other creative industries do it. At least then I could use it multiple times.
What I don’t like is being treated like a new grad when I have close to 20 years of experience.
We do have apprenticeships in programming, we just don't call it that name. You start out at the bottom of the ladder and more senior people help you learn. Most medium and sized and bigger companies have some variation of that. Sure you can start as the only programmer in a small company and thus not get mentored, but you still learn on the job like an apprentice.
They are useful in that they at least convey a tiny bare minimum level of skill--something many, many software engineering candidates lack. Passing the bar exam for lawyers and the medical board exam for doctors signals at least some minimal knowledge and qualification. They don't tell you the professional's whole story, but enough to allow you to skip the "Where is the patient's knee?" interview questions.
Software desperately need at least some "barest minimum" gate that lets interviewers at least skip the Fizzbuzz questions.
Still, if I need someone who knows Linux, something like a current RHCE tells me they have at least some minimal level of competency. (Certs seem to be more of a thing for sysadmin knowledge than general programming however.)
To the larger point: I'm very much in favor of hiring junior developers, but there's a limit - I find you need at least 2-3 mid+ developers for each junior, ideally at least one senior/staff per if you want to do a good job of mentoring and development at the same time. Since the industry has grown exponentially, it's impossible to hire proportionally to applicants and maintain this ratio.
I am counting down the days until our recent senior hire is fully onboarded and comfortable and we can open a junior position again.
Because nearly every other industry is credentialed, so proving your 3-8 years of university grind is enough to get hired.
Outside of tech, any other white collar interview is basically a 15-60 minute coffee or lunch with the boss (the culture fit interview basically) and then, unless you showed some red flags, you get hired.
Think of all the marketing, sales, Human Resources jobs etc…
Would you expect MD or CPA to be spelled out every time?
"Most working engineers don’t even bother with a PE."
Engineer is right there, no need for you know the E of PE is for engineer.
https://duckduckgo.com/?t=ffab&q=working+engineers+PE
https://search.brave.com/search?q=working+engineer+PE
https://www.google.com/search?q=working+engineer+PE
You are giving off troll vibes. Maybe you are in the wrong forum? HN has been called all sorts of things but never a "general public" forum. Next time, just ask if you don't know - no shame in that.
Nearly every other industry is credentialed and a degree is enough to get hired? I don't think that matches reality at all.
But outside of medical, legal, and certain financial roles, you're more likely to see formal non-academic licensing for truck drivers and hairdressers than for white collar professionals.
Software engineers making 150K with only a four year degree and some leetcode have nothing to complain about. Take homes? Whiteboards? Cry me a river. Try to pass the bar or get through residency instead.
You maintain a current law license, which requires having passed the bar once, continuing education requirements, and the absence of professional mal-, mis-, or non-feasance sufficient to lose that license. Taken together, the requirements are ongoing, though some individual elements are one-time.
Not getting disbarred hardly counts as an ongoing requirement in all but the most meaningless sense of the word.
https://www.ordemengenheiros.pt/pt/a-ordem/colegios-e-especi...
Problem is people calling themselves engineers without being one.
It would be wonderful to have an equivalente for computer science. You do it right after graduating, study for it once and you're done. Not leetcoding for 6 months every other year for life which is ridiculous waste of time.
In general, the undergrad degree is a filter. Unless someone for, say, a marketing role has a lot of experience, a BS/BA is probably a requirement and a better known school probably gives you a bit of a leg up as may a graduate degree.
You're still going to have some interviews. In my experience, HR passes promising resumes through, there are 5 or so half hour interviews, and then there's a thumbs up/down by the people who did the interviews.
But for most roles there's no specific non-negotiable credential like there is for an accountant or lawyer.
Software engineers don't like the concept of formal credentials. Fine. But you'll have to demonstrate your skills at one of the interview stages.
Or we could create something like the American Academy of Software Engineers, or the Society of Software Engineers, or the Software Engineering Institute, which would be tasked with accrediting schools and programs, and in order to practice as a software engineer in the industry, on top of a university degree, you'll have to pass another examination from the credentialing authority to obtain a professional title like Professional Software Engineer or Chartered Software Engineer.
Other professions gatekeep before you can take the interview.
It appears to have been for Engineers who want to be credentialed in software rather than a credential for a software developer to be able to attest to PE level standards of competency.
Local governments impose licensing requirements on some professions. This is not something the professions just voluntarily impose on themselves. (If they did impose it on themselves, there might even be antitrust problems as a result.) This is mainly (or at least purportedly) for the protection of public health and safety, not to make it easier on potential employers interviewing job candidates.
It actually has nothing to do with job interviews, because every such professional needs a license in order to practice their profession, even if the professional is a self-employed entrepreneur.
So to be clear, when someone asks for software engineer licensing, in effect what they're asking is for local governments to regulate the ability to practice software development, even for self-employed programmers.
And then you can expect a visit from the government software inspector.
Anyway, government isn't a fig leaf, it's a cudgel. It has to be, otherwise the system would be undermined by mass defection.
Formal credentialing generally means you demonstrate "I can do this", in a very drawn-out, thorough manner. It's gatekeeping more, but there's a much larger guarantee for your creds to be transferrable from place to place.
In software engineering, you demonstrate "I can do this", in a several-round bout of interviews. And then have to demonstrate it all over again and again, for every single company. Your interview performance is non-transferrable.
My opinion is, the latter is more wasteful as it involves repeating yourself a whole lot.
The former has a higher upfront cost but you are betting in the long run that it will amortize over time as you continue to go on interviews.
The latter has less upfront cost but a greater chance of creating more cost in the long run, depending on how much you interview and where.
OP spent 6 hours- personally I wouldn't spend that much on any assignment, but the bigger issue here is that 6 hours is only valid for one company. Interview "work" is so proprietary and non-transferrable- like currency that is useless outside that one company. You cannot take what you did there and show it elsewhere to expedite an interview. You will have to run through another several-hour process again (whether interviewing or another assignment), for some other variation of a CRUD based problem that they expect developers to do.
I suppose the closest thing we have to avoiding repetition is- the answer many won't like- Leetcode. You still have to navigate through opinionated views of your answers, but at least there are more patterns that carry over that its benefits compound more easily from interview to interview.
> . . . wasteful as it involves repeating yourself a whole lot.
It's also incredibly wasteful for the companies doing the hiring. Which is why they often try to offload that waste to the candidate.
But those companies would probably have to pay more for programmers who were already vetted as part of their membership in a professional organization; people who were actual engineers. So it's not in the companies' best interests to change anything.
Furthermore I do see a difference between being a person who knows how to write code and being a software engineer. The former can easily be achieved without any formal education, but the latter is more difficult to achieve without formal education. There is lots of stuff not learned in a coding bootcamp and it will take time to close the gap.
I do not think one can generalize like that and claim, that we all do not like formal credentials. I wish formal credentials worked differently though. Not through some university institution, which might or might not teach us important things, but maybe more practical and hands on, more projects, where actual work is done and we all get more opportunity to learn from thise experiences.
Easy. They have a 15-30 minute chat over coffee or lunch, then they get hired.
And called it the HN gestalt. Pretty sure I labelled it with what it is right up front.
Nevertheless, I do think there are quite a few individuals who have an unexamined (in the philosophical sense) assumption that they just shouldn't have to do anything to get hired. I bet most of them, if they examined it, would realize that it's not a sensible assumption. I think it sneaks in through these conversations. But I think it's a modestly popular view.
You can’t take the union of everyone’s food dislikes and say “according to the Earth gestalt, no food is edible.”
I get what they are saying. If you read the comment threads here on interview experiences, you're going to encounter groups that dislike or want to forbid every kind of technical screening technique. It's easy to read comments here and come away thinking nothing is OK and everything is unfair.
The difficulty with such is that these tend to be very time intensive on the employer side often requiring more time per candidate than the candidate spends with the prospective employer.
This can work for one candidate, but fails if one tries to scale it up to even 10 (much less 100) candidates. For that, a filter needs to be applied... and we're back at square 1 of a subset of people not wanting to do take homes (and so on).
THAT is your strawman. I would question that this is a "common" assumption - most everyone, when pressed, understands there is some need for an interview and some form of evaluation of skills. They may push back on specific mentions as one person already mentioned, but almost everyone falls in a middle ground where it's something like "yes, interviewers have to vet people, and that is hard, but it can be done in a manner that is respectful of the interviewee's time and properly reflects the kinds of things a person may be asked to do for the job in question."
I would premise that the problem isn't with (most) interviewees not wanting to do anything to get hired, but that (many) interviewers do not have a lot of experience interviewing and have just googled what other companies do and copied that, many times for the detriment to the entire process. If the interview process were actually thoughtfully designed to both probe the interviewee's ability to work with the team in question and ability to do the specific work the position entails, I don't think you'd see nearly the same level of disdain for the process.
It's weird if a company feels it needs to do more than that.
Companies either have trial periods or can let go of non-performing personnel. Hiring is critical but I've not seen any study that would correlate "high scores" in any type of hiring test to actual work performance.
This implies no matter how elaborate interview performances are orchestrated, their capability to do anything else except create false negatives increases as their complexity increases.
Hiring, in essence, is always a gating process with very little predictive capability beyond. The only thing you can know is the percentage of new hires who fail at their duties, and if that percentage is oddly high, you realize there is a problem in your hiring process.
I suppose you can gauge candidate agreeableness and conformity by subjecting them to all sorts of hazing procedures but in our field I don't see how that translates to good outcomes either - technically speaking. Might be good for creating cult enclaves, though, for lonely gurus.
I was a bit tongue in cheek here but my opinion "less is better" holds as it comes to hiring practices - this does not imply lack of diligence from the part of the hiring manager.
Or in your onboarding process.
There definitely are such studies. I don't have citations here, but it's well known that IQ tests or work-sample tasks are both the most effective at predicting on job performance.
I.e. if you prefer IQ in your job application process, you will get a more performative workforce.
However, with point samples it's not a very good benchmark. I.e. if you are comparing fairly strong candidates A and B, just preferring the one with the higher IQ is not necessarily the best option. So, getting the best hire for a specific position, and getting the most statistically best workforce are not necessarily the same thing.
Corps basically just offload this work on the candidate as this is obviously cheaper and more convenient, instead of spending time and looking into his profile.
I question how true that is.
In any case, GitHub as resume basically tells applicants that if you don't work in open source for your day job and don't have the time/inclination to do coding as a hobby, don't bother applying. Which is going to be pretty discriminatory against a number of demographics.
It's fine if someone wants to show off some project, but it shouldn't be a mandatory filter.
Personally I feel fine about take home coding challenges, because I would be programming something either way, but I can understand people that consider that they have better things to do with their free time.
I think it has to do with people the weird phenomenon of people that are working as "programmers" that don't know how to program. I've seen a few. I've seen people write C on their resume and I gave them an hour for an easy assignment with the language of their choice, and they couldn't even get anything to compile. The guy probably read a book on C a decade ago or took a C class in college. Me on the other hand, I had python on my resume although I never worked in python or any programming language, but I studied the hell out of it in my free time. It was difficult for me to get my first programming job, so I like this model more. Trust but verify
Another reason is that there is a lot of variability in programming jobs. A programmer or developer in one company may consist of changing a few lines of code in a project they don't understand all the way to architecting the infrastructure the whole organization relies on. It's difficult to parse out the two from just an interview.
If you're hiring an actuary, and they had worked as an actuary before, you're probably pretty certain the work is similar enough, and if they hadn't been fired then they're at least somewhat competent.
I like simple programming problems, something slightly harder than foobar, ideally industry specific. Show them you know how to get a programming environment setup, know how to create functions, for loops, classes if appropriate and basic control flow. With the technical stuff out of the way you can review the candidates work experience
- how much would you rate your knowledge of C++ on a scale to 10?
- 9/10
- Can you explain "pointers"?
- blank stare
Serious question, if the person actually had a 9 out of 10 knowledge of C++, how would he answer? Or is this just a teaser question?
I don't know C++ but I would answer pointers are addresses in physical (?) memory. They let you pass around variables by reference where the receiving function has full access and permission to edit its content. Pointers have to be dereferenced with the appropriate type before being able to read it
Any answer is basically fine, as long as it's clear you know what pointers are.
It's basically also a starter question to go further into memory allocation and freeing, smart pointers, etc.
The thing is, it's fine if you answer 3/10 "because it's been a while since I saw it". It's just to probe self knowledge.
This absolutely isn't unique to software. There's plenty of incompetent workers in other fields, like construction workers, bureaucrats, pretty much anything.
What I think is different is that it's not so much deep, like programming can be, as wide -- meaning a skilled interviewer can correlate experience and technicity in a way that gives reasonable assurance.
This is not the case with more junior positions where you need to make sure technicity has actually been acquired by the candidate, lest it be just as you say a "net negative" for the team: for finance positions with less than 10 years of experience, we have a case interview. That case interview is not meant to be selective but just a check of capability. Only candidates that have passed phone screening and hiring manager interview get the case interview, which is also delivered by the hiring manager so gives a chance to see the candidate in another kind of situation.
Junior bankers fresh out of school do the heavy lifting in huge important deals. They get supervision of course and there's a steep learning curve, but the fact is they are in the trenches pretty early on. Senior bankers deal more in relationships and sales. Could you imagine Google hiring junior engineers right out of college to tweak their ad-engine?
Programming on the other hand has taken a lot more time for myself, and pretty much most competent programmers I know. To think about it another way, I don't know anyone in finance that spends hours at home working on side projects learning finance for years. However every competent programmer I know has spent a considerable amount of time outside of the classroom and work honing the craft to reach a basic level of proficiency.
There's a place for junior programmers, but its one of those professions that if you don't truly enjoy it, the payoff in terms of the time you have to spend on it just isn't worth it. And the competition is a lot stiffer. The percentage of finance guys that spend their weekends contributing to open source finance projects, creating educational resources, or answering questions on a stack overflow type platform is much lower when compared to programmers that do this stuff for fun.
I don't know if you're suggesting that interviews should assign actual tickets in an interview, but that seems nearly impossible to do in practice. I've never got back work from an interview or assessment that was anything near production code - nor would I expect to. It's to get a sense of the level of their real world coding skills are since there is such a variance between developers.
Last fall I hired a supposed senior dev with 10+ years of experience and watched them struggle for the next 4 months with the most basic things. At first I though it was our codebase (large, mature and complex), but it became pretty clear they had mis-represented them selves. The fault is ours of course, they should never have been hired in the first place.
As for a portfolio, I would normally agree, but it's too easy to fabricate one. The above developer was put on a PIP and given some basic tasks to prove to me that they could actually do the job they were hired for (or at least the apptitude to learn/figure things out). For 3 of the assignments they submitted repos that indeed implemented what I asked,but they were not their work, they were cloned from Github. I had to let them go.
However it has become a common practice to require from applicants unreasonable amount of time to complete said assignments without any commitmet from the company. That's just toxic.
This would not have been such a big problem if companies provided decent feedback on what went wrong with your assignment.
One problem is that take home assignments are, barring some sort of time-gated online tool, open-ended in terms of time spent. Sure, maybe you can "meet the requirements" in a couple hours. But someone else will handle all the edge cases and optimize the code by spending a weekend on it.
I do a lot of writing and if you ask me to write 1,000 words on $TOPIC, I can probably do so in a couple hours. It will almost certainly be better if I work on it over a couple of days.
ADDED: Having said that, while I tend not to be a fan of take home assignments, if I'm hiring someone whose job is going to be, to any significant degree, writing for external consumption, I'm going to need a writing sample. And, if they don't have anything suitable, they're going to have to create one on the topic of their choosing.
> Chemical engineers must, on average, complete 15 hours of technically focused continuing education courses each year. State requirements vary, and some also require courses on ethics, state engineering laws, or professional behavior. Acceptable formats include online courses, live webinars, seminars, and college courses.
So while the PE doesn't have a personal portfolio, they do have a professional one, the tests that they've passed to reach that and the continuing education completion certificates.
The "do things at home" is a way to get a similar type of value as the PE has for continuing education... but not as any formal way of stating it because software development hasn't come up with a good way to do licensure.
So, for all those other technical, scientific, medical, and yes, business fields (the 'C' part of a CPA) which do have licensure, this is a solved problem.
Continuing education is a bit of a joke. So you attend a conference once a year, maybe watch some webinars. In general, PEs mean you've gotten an undergrad degree, passed a couple of GRE-type tests and have worked in the industry for a few years.
Shirley, you jest.
Doctors are asked to perform "demo surgeries," all the time.
Lawyers are asked to argue "demo cases" in court all the time.
Social workers are expected to help patients through trauma, in "demo therapy sessions," all the time.
CEOs are asked to do "demo IPOs," all the time.
CPAs are asked to do "demo audits," all the time.
In case no one realizes, that was all a joke. Of course, none of these happen, IRL.
I can see people with no track record (like just coming out of school), needing to do this kind of thing, but, once we have some accomplishments under our belts, we can have some track record.
Some of us may even have significant portfolios. Mine is so complete, that you can easily tell what I can (and can't) do, without ever even contacting me.
Designers live and die by their portfolios. Any first-time designer is expected to have a portfolio to show the prospective employer. Once the designer has been working for a while, they can point to their track record.
This is anecdata, but based on talking to my wife and friends who work in other industries most recruitment is pot luck. They interview the candidate for 1 hour, maybe 2, don't have any consistency in what they're asking and no standard criteria for 'good enough'. It's all subjective and down to the interviewer (maybe two of them if you're lucky). You then get stuck with a proportion of your staff who are dead weight.
I'm not sure of the reasons why the technology sector cares more. Possibly because the difference between "good" and "bad" is much bigger - maybe not the mythical 10x, but more than the difference between one accountant and another. Or maybe it's just that these are all newer companies who don't have the older ideas holding them back and have discovered a better way to be profitable.
The mere thought of doing that, in the tech world, though, brings about collective apoplexy. I'm not especially a fan of how licenses can be used to protect incumbents, and bar entry to otherwise highly capable people. They do, however, go a long ways towards ensuring that a building doesn't fall down, the first time a hot rod drives by.
But hiring always has been "pot luck." Been that way, for -literally- thousands of years. That's why hiring good managers has always been important, and there's really no way to test for that. For whatever reason, we seem to think we get to play by different rules.
I presume one reason may be because tech industry may be a bit insular, and full of people who enjoy creating complex models of the world - trying to find a perfect technological solution to everything (when they should just try to find a good hiring manager in stead).
Medicine does move on, but a lot of what a doctor needs to do doesn't. And the bits that do are actually a problem with there not being a good way of ensuring that doctors use up to date techniques.
The difference is that structural engineers and doctors are safeguarding safety, and (most) software engineers aren't. If someone suggested people should need chartered status to work on medical device software I don't think that would be as controversial, and will get progressively less so as the discipline is less brand new and progress slows down a bit.
Many apps that I encounter, are pretty much bald-faced, unapologetic, data harvesters. That’s clearly their main purpose, and their stated purpose is really just the lure on an anglerfish.
“Move fast and break things” sounds good, until you realize that said “things” may be people’s lives.
I presume technology sector pays more attention because the task of programmer is ill defined, and that it's quite hard to define exactly what you are looking for based on external standards.
My fiancee is a high school guidance counselor, and she essentially did a demo session with some current students during her interview.
I don't like the situation we are in, but I think your examples actually help illustrate why things are the way they are: we don't have a gating mechanism into our field. Most of your other examples do.
Teachers also have fairly strict licensing and educational requirements (and get paid squat).
I know a lot of finance people. These are not the most stable or reliable people on Earth, but they have to pass their Series 7s, in order to work. They tend to be kind of a scruffy lot, otherwise, but they make a lot of money.
To the degree that it's a reasonable thing to ask in terms of time, it's hard to argue against asking someone to demonstrate something that they'll actually be doing on the job. Certainly I don't think, in all but the most extreme cases, that anyone would argue that there don't need to be auditions for actors or musicians.
When I was an IT industry analyst, we did ask for writing samples. At least one of the big firms did require at least some candidates to prepare a presentation on a topic and deliver it. (This is an example of a sub-industry that has very few formal requirements and people come from all manner of different backgrounds.)
The whole point of these tests and whatnot is to weed out the bad from the good. It definitely is unfair on those who are good, but I'm not really sure what the alternative is.
I dislike the take home tests as much as the next person, and only do them if I really want the job. Beyond that, it's a neccessary evil until something better comes along.
Is there a name for that fallacy?
I.e., when two different subgroups have incompatible positions, treating each individual in those groups as holding both positions?
As a mechanical engineer with sufficient years of experience that would be considered senior in the software world but intermediate in the mechanical world, I have had interviews where we only talked, interviews where I was given a task and put in front of a computer with one hour to complete it, and interviews where I was sent a problem to work on at home. These interview techniques, while perhaps more prevalent, are by no means limited to the software world.
Within a few minutes I can figure out if a candidate knows SQL better than I do (not that hard to do, basic joins maybe a HAVING clause) or if they took a class and forgot most of it and don’t know what it means to de-normalize data. I can do the same with Excel.
But I still don’t know how to effectively judge whether a candidate is effective at closing deals, what their work ethic is, how they actually support team members, are self-starters, etc. It’s much harder to do.
Technical fields have a huge advantage in being able to assess skills with objective criteria. It’s not perfect and interview anxiety is real, but it seems a lesser challenge than how business fields hire.
Name me one interview method that actually has better results than that? Okay, lets eliminate the ones that are obviously unqualified from the ones we paste up, is your method still better?
Most people interviewing honestly have no reason to believe they have a better method. There has been a fair amount of scientific research about how to select good employees, but none of that has made it to the people interviewing. I don't know much that will help, but HR really should be finding scientific basis for how to interview and training people on that.
Once we admit "eliminate the ones that are obviously unqualified", there's no longer a principled distinction between dartboarding and what we actually have, it's just a matter of degree of filtering.
The optimal level of filtering is almost inherently going to look higher to employers and lower to applicants based on their incentives. Fixing this is a coordinated action problem.
The real issue is that the company ghosted OP rather than provide feedback and closure.
Personally, the more experience I gain, the better the hiring process becomes.
Your resume is basically the Instagram version of your work life (and let's not pretend otherwise). I've worked on a couple of teams that hired solely on the "strength of the résumé," and it was, honestly, a disaster. The two people I know we hired on that basis quickly cycled out (one fired, one quit) because they were pretty terrible. We realized they were just good interviewers who couldn't actually ship very good code. Maybe the issue was on us with a bad set of interview questions, but had we asked them to do even a simple take home, we'd have sussed out the issues really quickly.
Having an interview process exactly like Google won't give your company Google-like engineers.
> We realized they were just good interviewers
It sounds like they didn't hire solely on the strength of the résumé, because then wouldn't interview performance be irrelevant? In contrast, there are people with strong résumés who don't interview well, regardless of the interviewing method.
> The two people I know we hired on that basis quickly cycled out (one fired, one quit) because they were pretty terrible.
This is how it should work, I think. Hiring is extremely difficult. Mistakes will be made. The key is to quickly correct those mistakes and move on.
I'm not sure what you mean by putting families at risk?
Incompetent people put their own families at risk by trying to work in an industry where they're incompetent. It's not the responsibility of employers in that industry to provide for the families of people who can't do the job. It's really a bizarre concern to be worried about the families of fakers. Moreover, how would it be better for the family of a fake for companies to not hire them at all, as opposed to hiring and inevitably firing them? In either case they'll be unemployed ultimately, but at least if they get hired and fired they can pick up some paychecks, so that's actually better for their families, if that's what we care about here.
(1) Competent programmer who isn't a good fit for a particular job.
(2) Incompetent programmer, the "can't code, can't even write a for loop" bogeyman that seems to scare everyone.
Coding interviews — whiteboarding, take-home projects, and the like — are intended to filter out (2). But they aren't good at filtering out (1).
(1) can always occur. There's not a great way of avoiding that until they actually get on the job. Of course it's unpleasant to fire somehow who doesn't work out, but I don't see how take-home assignments are supposed to help here with (1). Whereas the (2) people shouldn't be in the industry at all, and they're lucky to somehow scam someone into giving them a job, but the industry should not be financially supporting scammers or their families.
I'm personally not convinced that (2) is as big a group as many people believe it is.
I have a friend who set the expectations upfront (working from home) and was told it might not be a problem, finished the whole ordeal of take home assignments and technical rounds. The company liked him but he was told WFH is not acceptable (hello Tencent).
In my experience interviewing, a properly designed technical interview exercise can fairly accurately peg an interviewee's skill levels in about an hour, as well as their ability to communicate with the team (taking into account nerves will likely reduce their performance in an interview setting). I think having a take-home exercise could be an option that could be offered, particularly if someone is struggling in an in-person(or video) interview session, or even something that is flexible ahead of time to allow the interviewee to put their best foot forward by choosing the evaluation that best suits their strengths.
I do agree with your point that interviewers need to have an ability to spot "good interviewees" - some people are just really good at interviewing. I don't think that automatically means take-home assignment though, I think having gradually more specific questions to suss skill level out (which you mention, i.e. "better questions") is really important.
Thinking about it, I think this is a good approach. Duration is right, you're left alone to think, and it's pen and paper so while low tech it does show your own reasoning and your own knowledge of the language (a typo here or there is not critical to the exercise).
I actually think that'd have, statistically speaking, similar results as right now for a lot of companies that aren't Google.
There should be some kind of test that would be recognized by the industry, that you could attempt once every six months for example, and that would positively assess your ability to code. You would get a score that you could show to recruiters (if you wished to). Some recruiters would accept only 95/100, others 90/100, etc. There could be language assessments as well.
I don't understand why this doesn't exist. Or maybe it does exist and is simply ignored? What would it take for companies to trust such a system?
Write such a test. Give it to all the engineers whose work you know well, and see if it can accurately predict what they brought to your team. I would be very surprised if you could both spot good talent and dead weights reliably.
This system wouldn't replace the personality interviews, etc., but only measure technical abilities.
The downside is some people have barely scraped by in the interview and turned out to be absolute rockstars and day 1, and some people have aced the process and driven everyone else mad with incompetence. So it could be improved.
And maybe that's just on us not writing the perfect questions yet, but a bigger problem is that once we've used a question enough someone will post it online and suddenly a suspicious number of candidates are able to show up and rattle off all 3 optimal solutions in 15 minutes. Weird! I suspect a standardized test approach to software engineers would get this problem even worse.
This is how language certification works (think TOEFL, IELTS), and that industry still suffers from a massive amount of fraud. If you make a certificate like this too expensive, it serves as a barrier for entry for candidates. If you make it too cheap, the people administering the test will start pushing through candidates as quickly as possible without any regard for academic honesty.
If I go by mainstream, then I guess it would contain some coding task, where you might have to solve algorithm stuff, that you will probably never encounter on the job, or some fake OOP "everything is a class" or "I must use [hot web framework of the month] for trivial things" kind of thing. None of that really tells much about a devs true skill.
I think the problem might be, that jobs are so divers as developer. For some jobs maybe the hot web framework of the month is really all you need to know, while other jobs might require you to think about how a system needs to be redesigned to run on some kind of multi server environment.
I guess there could be multiple such tests, for each area of expertise one. It would for sure be a lot of work to put them together. One would also need rotating problem sets or questions.
Also: Who is going to create these tests? I hope the idea is not big tech or some kind of government participation. Big tech would quickly make it a filter for only themselves, making their tech stacks assumed knowledge in the tests, and government would probably come up with something hopelessly backward like testing your Cobol skills or demanding you write an HTML 4 website, while you have gotten used to more semantic HTML 5 years ago.
I think it is a nice idea, but there are many questions about how it could be usefully implemented.
Here is your homework, we expect someone at your level to spend no more than two days on it, here is your $800, have fun.
I never had to
For example, my wife is an organization development consultant, she's changed jobs a few times over the last 12 years, and never has she had a take-home test. Instead, she has a few rounds of interviews, typically starting with a hiring VP, followed by an SVP or executive, and possibly a panel interview with possible peers. They talk about her previous work in detail, digging into frameworks she's used, papers she's read, etc. And then relate that work to the open position.
Personally, I've had reasonable success talking through a basic system design problem. Depending on the candidate's experience level, I can focus on different aspects of the problem... overall design, performance & algorithms, eventual layout of services/components in AWS, etc.
Meh, in the Midwest plenty of jobs pay enough for that, 6-figure tech jobs put you at 3-4x the median income (31k here in Indiana), but plenty of other jobs absolutely pay enough to buy a house (teaching - my wife makes just shy of double median income as a high school teacher, nursing, law enforcement, fire fighting, all sorts of trades, etc).
Well... ya know. Things have been weird lately.
My wife and I bought 20 months ago with me making 39k and her making upper 40k. We bought this house with zero down at 170k and it's now estimated at 240-250k per the mortgage company and Zillow. We have about 1200sqft on the ground floor then about 1200 in the basement with a little over half of an acre, we're 30 minutes outside of Indy. House is 1960s.
A new construction house in Plainfield, immediately adjacent to Indy, you get 2 bed, 2 bath, 2 car garage, basically no yard with the house at 1504 sqft is starting at 290k right now and can get a 3 bed with similar sqft for closer to 300k. If you picked the neighborhood well, and were ok with not having the latest and greatest features, a single person could still probably pull a mortgage in a safer neighborhood without issue.
>somewhere around 40k ~ 50k? (which is what high school teachers in the area seem to earn
So this varies wildly. My wife was getting the higher end of that at the rural school she was teaching at, she's moved one school district over and took a considerable pay bump - mind you she has like 12~ years of experience, and once she finishes her masters she'll get a 4-5k pay bump at her next (annual) contract renewal automatically.
Perhaps the reason why things are so is that a good chunk of tech revolves around advertising, and it’s easy to think of resumes as ads if you work in that space day in and day out.
Yes, but to be fair, I am not convinced that such a method would yield drastically worse results than what is already done at many places. I am not totally being facetious about that either!
It's a jungle. People get hired or rejected for all kinds of reasons, meritocracy has less to do with it than we would all like.
The best approach is to keep one's professional network well-maintained so that when it comes time to change, your reputation precedes you. That's not always possible. Some jobs really are filled from candidates who drop their resume into an anonymous applicant vat on the web and hope for the best. But I think the best-fit jobs are often obtained when you know someone and they know you.
My post was not meant to imply that home assignments are a wrong tool to filter out candidates.
It's how it is conducted that makes it a toxic practice. Not providing any meaningful feedback or not paying for a work that takes too much time is what makes it not acceptable.
Take-homes exist because companies suck at interviewing tech workers. It should be enough to simply review some code while on a call.
Companies should hire senior engineer consultants to do technical interviews. A staff engineer with 15+ years experience can suss out how good another engineer is in 1.5 hrs.
I don’t understand how this is hard. I’ve hired dozens of engineers in the last 10 years. I’ve never needed to resort to any google level leetcode tactics. It’s never made sense to me if the goal is to hire a quality software engineer that can get the job done. Maybe that’s not the goal for companies that make interviewees do this performative bullshit.
I love it.
Like, "Zeitgeist" seems to be used like I would in German (although far more frequently).
What am I missing? Is there some subtle difference in meaning via Yiddish or similar?
A collection of physical, biological, psychological or symbolic elements that creates a whole, unified concept or pattern which is other than the sum of its parts, due to the relationships between the parts (of a character, personality, entity, or being) quotations
It's also used that way in German, aside from its regular meaning: https://de.m.wikipedia.org/wiki/Gestalt
Nevertheless, I would absolutely not spend 6 hours on a take home assignment without compensation. Capitalists get carried away sometimes...
It could be an interview. Or a series of interviews. Just like every other job.
The important part of contract-to-hire is it shows how the person will communicate while working, which is not really verified in any way with a take-home or whiteboard setup.
Going straight to full employment is like going from strangers to marriage, contracting is a dating phase.
As someone who has been freelancing/contracting for a long while, it is much preferred to full-time employment and essential ownership anyways.
I personally don't mind take home tests, everything you do you can learn from and this gives you somewhat an idea of the type of company you are working with (not necessarily the work). Though they should be paid, as people that are in demand would be more willing to do them as well as the company more invested in reviewing them. However, in no way is the take home test a good indicator of how this person will ship product.
Not when employment is at-will.
It's more like going from one date or few dates (depending on the length of the "interview process") to an exclusive relationship, which happens all the time. And non-marriage relationships are also at-will, so they can end quickly if necessary.
It does suck that when you are on your own you have to either join a group or be rated at the individual/family level. If we fixed grouping in health insurance this would be much better. Grouping like how Medicare does it across all private insurance would be much better.
The great thing about not taking work insurance is no extra cost, can easily go from contract to contract or full time for a while, and with your own company it can be expensed. The only downside is the part work would pay for, however you don't get any say in the type of insurance or what companies. You can ask for a small bump in pay to help cover it.
Companies would prefer not to have to deal with health insurance as it is a competitive weight in the US. Private insurance and public options, both separated from the employer is the way to go.
We don't get our home/auto/life insurance from work, why do we get the most personal type of insurance, health, from employers. Not only that healthcare bound to the job leads to all sorts of problems, ageism, harder to change jobs, harder to compete with other countries and more.
If developers starting doing this more, we alone could change the legacy employer/healthcare coupled system.
The difference is in the presumed defaults.
If you hire me, my assumption is that outside of company/organizational problems/changed of some sort, you'll keep me around so long as I'm reasonably competent.
Contract to hire on the other hand , sends off a vibe of we'll probably hire you if you wow us. Which, if someone already has a job or has other good offers in hand, is not going to sound like a great deal unless it's a company/position they really want and are very confident they can do.
Same with interviewing/take-home/whiteboard. It is just a better realistic view of the company. It not only makes it a "wow us" for the worker to the workplace, it is also the workplace to the worker. It is a two way street with a more realistic sample. It also reduces the friction of locked in employment.
Creative/programming/product/design are all creative fields, most projects/products like that have more flex/contractor style arrangements based on projects/products not necessarily the company such as movies, music, books, etc. Most of my career has been in games/interactive/agency style projects/products so it may make more sense there.
You can't judge creativity and product shipping capabilities with a take-home/whiteboard/interview very well. The sample is much smaller. The better way is the body of work and a real test that is longer or closer to the actual work and timeframes.
And yes there are agencies and consultants for all sorts of specialized work on a project basis. But that's not the majority of software and I'd guess the majority of programmers/software engineers don't want to be constantly looking for work especially if demand lets up.
Obviously, some things like games lend themselves to more of a project-based approach to staffing than other things do. (And certain other engineering projects do as well.)
The recurrence of those complaints indicates that the process could be improved to benefit both parties. Everything you listed can be made better for all. The reasons they're not have to do with a mix of convenience, traditionalism, mimicry and other social factors.
Think of all the good reasons companies had to have workers on site a few years ago, but can somehow still make it work remotely today.
then (or instead) we might do the opposite: I should have a project on github (or similar) that you can look at beforehand, let you pick an issue, etc etc
if the issues are 6 hours issues, the commitment is on both sides: the company has a team member spending those 6 hours too
Not that there's an approved method, but the best one IMO (and which I've been using succesfully for 25+ years) is to carefully read the person's resume and have an in-depth conversation about projects they have worked on. No trick questions, no whiteboard memorizations, just a normal conversation about the work they claim in their resume they have done.
We then walk through their notes (if they made any), and the code sample itself together. I have them make any observations, critiques, suggestions, or other thoughts together with me. This gives me an idea of how they think, what issues they saw, how they would improve things, how familiar they are with the specific language, and coding in general. It also gives an opportunity for some light “conflict” resolution , if I push back on their thoughts.
This has worked really well. Those that don’t do well on the spot can make notes ahead of time (but aren’t expected to spend time coding for hours) and those that are confident can walk through it live.
It was such a good trigger for discussion and we could easily measure a candidate's fit in moments. Some would just go "yeah this code's perfect" and not argue at all, some would spend ages with irrelevant details, and the couple of people who immediately looked at the diagram and spotted the most glaring OOP issues turned out to be great at their job.
The guy also asked me a few relatively simple technical questions, then said "Okay I know you can handle the job, let's see what you really know," asked me some really difficult questions, when I said "I don't know" to some he actually spent a few minutes teaching me the concepts.
That same guy was the lead programmer on some famous arcade games, btw. Did a lot of assembly programming back in the day. He had serious technical chops.
Got a job offer the next day. 1 interview, 1 interviewer, a little over 30 minutes spent.
Best interview experience I ever had. If more interviews were like that I wouldn't feel dread every time I felt the urge to switch jobs.
Also if that's all someone of his caliber needed to determine applicant quality, I don't see why everyone else feels the need to go through all this other bullshit.
Flipping it around, it takes someone of that caliber to be able to determine applicant quality with just that. Can we replicate it without having to spend some of your most valuable employee-hours, let alone in a way that doesn't leave you open to accusations of all sorts of bias?
I've long felt that 60 minutes is sufficient for a competent interviewer. In a past life, I was tasked with interview-slots where I had to both assess the candidate and double-check the signal other interviewers got, and I would spend:
- 5 minutes on intros,
- 15 minutes on a coding exercise,
- 15 minutes on architecture discussion about something else,
- 15 minutes on "behavioral" questions about something other than *those*,
- and 5-10 minutes on "do you have any questions for me."
I love the fact that your interviewer got it down to 30 minutes, and I especially love that part of his interview process was sitting down with the candidate and teaching something new on-the-fly. (If I had to guess, it was your "I don't know" -- followed by how you handled being taught something -- that sealed the deal for your offer.) That sounds way better than a lot of the behavioral-question fluff that I used to try when I was still finding my way as an interviewer. I'll have to give that a try.
One swift interview process I did included a single short page of code and the invitation to (with pen and paper) review it; I was given some time to think about it on my own and then to talk it over with the technical interviewer.
I liked that one and they had what I thought was an above average quality of developer when I joined. Anecdotal but interesting.
The one thing I didn't feel just asking for review/discussion accounted well for was their ability to build/add something new to the code, and so we have followed up the code review/discussion with a very specific/small exercise to evaluate the process (architecture, pseudo-code) they'd use to do that, and then discuss that as well.
I've had 4ish interviews since that job, and all of them gave an on-the-spot code reading assignment to walk through their code to find the bugs. It let me see who has old style coding standards, who takes advantage of modern C++ utilities, and that the interviewers are competent enough to walk through coding examples, too.
Depending on the tweak it can also make the most basic of applicants fail to find matches entirely e.g. ask for a “YcomBinator” instead of a “FizzBuzz”.
That's way faster than writing unpaid working code and shows you're competent AND efficient.
2nd aspect is unpaid take homes. It's complicated. Companies do pay a lot to fly candidates, put them in hotels and feed them for onsite interview. So financially company can afford to pay something. But it adds a moral hazard as you are paying cash irrespective of any expected quality or commitment.
Other thing I would like to highlight is that unlike any other industry, tech. allows people to enter at ease irrespective of the pedigree and credentials. No hospital would hire a physician who has read a lot on the Internet and did some pro-bono help to the neighbourhood kids. So what is the best way to hire isn't very well defined. It could be Leetcode, puzzle problems, take home tests or something else.
I tend to see 1) quick phone call with recruiter 2) please make assignment
in Europe at least
Then they go on to complain that candidates will jump off and that they can't find enough employees.
In one case, Airbus, they didn't even bother with a phone call. I applied via their website's application process. More than a month later, without having heard anything from them at all, I get an email from someone with an exercise that "should not take you more than 12 hours". They also gave me 3 days from the day the email was sent, to send my code. Eh...
Ah, there was a local company which, as part of the application process itself, would present you with a small test. First problem: it was incorrectly set up and I got some totally unrelated test about something like Excel or whatever. I added a comment to let them know that I thought it was the wrong test. Again, never heard from them for 2 or 3 weeks. Then an email addressed to no one in particular, asking to do an assignment. I replied asking for clarification. Saying I hadn't even spoken with anyone yet and the exercise felt weirdly unrelated to the position I had applied to. Never heard back, fortunately.
The take-home assignment is short. It shouldn't take more than an hour if you really really want it to be polished, which we heavily insist we do not care about. The overall process is a bit longer, because we then discuss it together.
For the pair programming session, the exercise is picked by a developer that isn't involved in the process, to put candidates and interviewers on equal footing.
Our goal is to be able to give an offer within 48 hours starting at the first contact, and give a feedback, positive or negative, maximum one hour after each stage.
It's a candidates' market. A long and complicated process only ensures you weed out the candidates who have plenty of options. I'd almost be tempted to give a long assignment and consider everything but a firm "Not even in your dreams" as a failure.
That's fine, pay me.
>pair-programming
Again, that's fine, pay me.
>go through an existing project of theirs, discuss it, and extend it together
The SaaS's I own and operate are closed-source and I'm not looking to take on any contractors at this time, but I'll let you know if anything opens up.
Should I pay you for an hour of your demanded salary? So somewhere in the $100/hr ballpark? Or some nominal fee, the same across all candidates? $100 for the interview?
It even sounds absurd. With this kind of attitude, you'll end up having to do your own roofing. Self-respecting men don't work for free.
For prospective software employees: stop granting hiring managers absurd entitlements - do not work for free. This industry is not special and it needs to stop pretending like it is.
If you come develop for me and do nothing but cry that you can't do a proper slice append in go and you need us to change our stack to elixir because you're used to it, all I get is a fight with HR for 3 months getting a paper trail together to fire you while I pay you $40k.
Not to mention there's a whole sales model of household appliances where they very much come in and demonstrate the value of their vacuum before they sell you on one...
The point of offering pair programming is that we invest the same amount of time. You don't spend five hours coding to my five minutes reviewing.
If we're at this stage, then it means you are interested by the company, and the company is interested by you. An interview is not unidirectional, you review the company as much as they review you.
> The SaaS's I own and operate are closed-source and I'm not looking to take on any contractors at this time, but I'll let you know if anything opens up.
That's fine. I interviewed a very junior candidate this afternoon. She went through a coding bootcamp, only has one year of agency work. She didn't feel confident going into a pair programming session blind (which is perfectly understandable, it's not my favourite exercise either).
She however had a bootcamp project she was quite happy with, and that she chose to explain.
You can maybe get in on reputation or network alone, but for every person like you there's two hundred juniors who do not have that luxury, and we're trying to play as much as we can on their strengths.
| We are wondering if you would like to attempt our code challenge (C/C++ or
| Python). It is a Computer Vision problem which we think can be solved in
| about 8 hours.
I'm afraid I wouldn't, 8 hours is far too much for a interview
problem.
However, thanks for considering my application.I'd skip the shenanigans and look elsewhere. I think "thanks, but no thanks" is a good response in this case.
1. Is it actually giving the employer more confidence or information about your candidacy?
2. Is it within the normal bounds of an interview? If someone asked you to do another 60 minute technical interview, would you?
3. Is the company clearly not trying to use the product of your code challenge?
To be honest, there are a lot of people who can look impressive but turn out to be totally under-skilled at coding. I’m sympathetic to hiring managers who feel like they need to test for that. As long as it’s not a super long exercise, they’re not just hazing you, and they’re not trying to use the results, I think it’s fine.
The technical interview itself drilled down more deeply into my skill level, however I can appreciate that they wanted to have a bare minimum idea that I could actually program before committing more time.
The favor was more than returned when they took me out to lunch in the middle of the technical interview and let me ask as many questions as I liked about the company.
Almost everyone chooses to do the small coding assignment, because they don't have code to bring.
There is a real treasure trove of things to talk about.
Questions like: - Why is it like Y? - What is good and bad about Y? - Why do you think X would be better? - Why did you end up doing Y instead of X? - How would you approach changing it to X?
Discussions like these on both a macro and micro level is very valuable.
I have prepared scenarios like this for applications (with some intentional bugs ranging from off-by-one-errors to big architectural fuck-ups) and I think it overlaps a lot with what a candidate would have to do in a take-home assignment anyway.
Kids on Blind are doing 200+ leetcode problems before starting their interview loops. How many hours is that?
You’re basically just choosing to forget about all the prep needed to ace a DP problem on the spot.
Lucky for me, landed a role that had checked my github profile and answered some questions, which was sufficient to convince them. No waterboarding required :)
The second is they didn’t set expectations (or at least you didn’t tell us if they did). Should you have spent 6 hours on this? How long does it normally take? After the assignment would you be brought in for 6 more hours of interviews or is the assignment the major filter?
At the end of the day, take home assignments are the best filter, the most equitable and the most efficient way to do hiring consideration. Companies (usually the middle managers within them) just don’t have the courage to use them correctly and don’t spend the resources to make them a good filter (they also don’t spend resources on the other job filters so those aren’t good either).
Nonetheless, I can't think of a better solution for this. Hiring someone is a big commitment (for both parts), so I want to be completely sure that I am hiring the right person. How do you that? How do you measure the quality of the other person in an area such as IT where it's so easy to lie about knowledge and experience?
It's much the same vein as that Canonical recruitment questionnaire that was doing the rounds a while ago - if you want to sit with me (virtually or IRL) and ask me questions, that's fine. If you want me to submit an essay that takes me hours to write then you're going to have to be offering me something I cannot possibly get elsewhere!
In the end it's about mutual respect.
Secondly, some candidates get very nervous/anxious when they have to answer code in an interview and can't show they real knowledge, etc. Pair programming hasn't worked for me because you get a lot of false negatives that are hard to distinguish from the real false.
To your second point - any hiring exercise is going to be stressful for the candidate. False negatives are better than false positives (for the hiring party). If you're certain that this is going to cost you too many good candidates, then you're going to have to find some other way to demonstrate good faith up front - offer to pay them, offer to host them on your premises, something like that.
The original fizz buzz type test was to weed out the fakers. It is something most people with a bit of skill can pull off. But as an industry we did what engineers do. We over think it. We blow it way out of the bounds of what it should do. We ended up with enterprise style fizz buzz testing as a filter. Honestly, at this point you really can not expect more. But can they bash out any code, can they read code (and understand it), are they cool with your 'culture'. There are 2 things you do not want on your team, a jerk (they can usually hide it in a few hours of interview), or someone who does not 'get it' (they can become help vampires). But I will take a dozen help vampires over a jerk every day of the week.
It is not like most people are going to 'hit the ground running'. In many places it takes a few weeks just to figure out who to talk to so you can check in code, or get your computer setup correctly. I have yet to come into a job and not have to wait on getting a computer (first day), then a few days of getting access to everything. Let alone beginning to comprehend the problem they are trying to solve.
Let me tell you why that was better than the current hiring meta.
1) I can learn everything I need to know in an hour with someone. 2 will likely not drastically give me a better idea. 8 hours with someone is not 8 times better, it just doesn't scale like that. So the idea that the more work you give a candidate, or the more interviews, or whatever, the better your hiring practice, it's just not true.
2) People can be very nervous in interviews. It's an intimidating, adversarial environment that has no relation at all to the job or work. It's almost the beginning of the way for the company to treat you as lesser, and they will take advantage of that to screw you these days. As you say, they will just ghost you. They didn't used to do this nearly so much. They'd at least send you a letter, or give you a call.
3) Even if 1 and 2 weren't true, the questions asked in interviews have no relationship with the real job. You just can't ramp up in time, etc. There's too much learning. Picture an intern. They do nothing except take everyone's time for the first month, but it's the rest that matters. Will they get something useful done before they leave? Maybe. But it does take a while to ramp up. So interview questions don't judge job performance.
But we still do it, because people like that tinge of power I guess. That feeling of control in that you can say yes or no to someone, they have to placate you for the job, etc. A lot of interviewers I know act very strange in interviews too, and they don't act that normally.
For the interviewee, it's even worse. You waste your time, you spend a lot of emotional energy, you don't know if you like the people you'd be working with, you don't know really what you'll work on, and no matter what you have to do what your boss says anyway, which can change at any time. Oh and it's "right-to-work" so they can fire you any time for any reason.
It's much more useful to talk to their previous boss or friends or references. Even if they are biased, you can usually tell pretty quick again who is telling you nonsense. At least then you can hear real stories about real situations in the past, rather than useless hypotheticals. But we don't do that anymore because of liability from lawsuits in hiring. Because a lot of companies are actively discriminating. Some are passively doing it, some are trying to do better but are hopeless. So everyone is just covering their ass.
Sounds like a great deal!
I will however not accept a take-home coding test before I spoke with the engineering team and saw that they're seriously considering my application (i.e. it looks like I will be a good fit, and they just want to make sure I'm not some fraud who can somehow talk the talk, but can't walk the walk).
What I DO HATE about take homes, is if you complete one, clearly put time into it, and then are refused feedback when you are rejected. If I put 2-10 hours into writing something for you, I at least deserve three sentences on why it wasn't accepted. I know these people are busy, but so am I. Most of us also have jobs, and even if we don't we want to know how to improve.
That being said, I've seen cases where the take home assignment is actual company work, in which case I 100% agree you should be paid.
Actually, the assignment was "take this shitty service and make it production-ready". It had some pitfalls that I avoided, and I introduced monitoring, automated deployment, API documentation, etc.
There was a bug - a minor misunderstanding that was simple to fix if they gave the feedback and allowed to do a quick fix - but the rest was production-ready.
I had fun doing it, that's why I didn't mind taking time. What I didn't expect was the poor binary feedback with zero content.
From now on, I'm only taking these assignments if they agree to have a feedback session. If not, I won't do anything. I'm close to 20 years of experience and it feels like they treat me like an undergrad (although I have a full Github profile).
Then I got grilled in the interview for not pulling in all sorts of bloat and other bullshit for a single endpoint CRUD app.
Full stop.
It was just fixing the buggy implementation, write down a Makefile with everything I thought useful (linting, security scanning, Docker image generation, etc), and integrating with Datadog and Rollbar's SDKs.
So I decided to mail a take home assignment. I took about an hour myself to prepare it, the assignment was not work related, it was a completely fictional task. I think it was creating a HTML page of ~50 lines demonstrating the hole could be filled if the person had access to internet and all relevant docs. The person delivered the next work day, did well, and we hired.
I've thought long and hard about this, and decided the assignment was a mistake and I would not do it again (sorry to the person in question). Why? In practice, when listening an hour to a candidate and let them write a minimum of code, you know if you have a complete weirdo who never touched a computer, or someone who is at least basically OK. In this case, it was a short term job, so the blast radius was pretty small even if candidate was bad. For longer term engagements, there is generally a test period and better vetting available. Meanwhile, I was in a position of power over someone else, and while the whole thing seemed minor to me at the time, I later heard the (junior) candidate was worried sick and spent way to much time on the assignment.
An underlying cause for my mistake was the interview situation, which was very bad. Basically, I was in the middle of something else, and a project lead I had never heard of for a project I had never head of asked me to do a technical interview while the candidate I had never heard of was already sitting in a meeting room. Needless to say, I was completely unprepared. There wasn't even a whiteboard in the meeting room. So I basically adjusted by always being prepared for surprise candidates: A list of questions for most common technologies and a template for technically grading candidates were always stashed away in my locker from then on, and the people starting projects would keep us interviewers in the loop when a job would open.
If the hiring process requires 2 minutes of the company's time for every 1 hour of yours they are pretty much guaranteed to waste your time.
I've made exceptions to this rule before for what I thought were good reasons and deeply regretted it every single time. I no longer believe there are exceptions.
That being said, if you want to avoid having a large imbalance in risk/reward, I would suggest talking compensation before agreeing to submit yourself to a large time investment. At least that way you know why you are putting in the time investment that comes with take home assignments. There's nothing worse that putting in a significant effort into the interview process only to discover your compensation expectations would never be met.
Even then, if something can be done in 1 hour, it can probably be done better in 2-3 hours or more. It takes time to show your best work. That's why many companies that do take-homes claim that their process takes only a couple of hours (but in reality it doesn't).
There’s nothing inherently wrong with unpaid take-home interview work. Don’t do it if you don’t want to do it, but don’t declare that it’s “not OK” in a general sense, as if it’s somehow immoral. If both sides are clear on the arrangement, and it ends with either acceptance or rejection with feedback, there is nothing wrong with it. It’s my favourite format as an interviewee.
I've interviewed maybe 500 engineers in my career. I'm an early engineer of Instacart, 3rd engineer of Eventbrite, founding engineer of Reforge. Started 3 companies myself.
My interview is always the same:
1. Bring code you've written
2. Share your screen
3. Explain what it does and I will casually ask questions about it
You get so much information from this:
- How they think about code
- If they think it could be better
- Who they blame if the code isn't the best
- Personality
- Product dev glimpses
- Comms
- Sentiment
I just detest the practice of not giving anything in return for the time you have invested in doing the assignment. I would not complain about this if candidates received good fedback.
So you think that a 6 hour unpaid assignment is unfair. What do you think the alternative should be?
Many of the posts here point out that 'take-home-tests' aren't common in other professions, but completely miss how many free hours other professionals have to do to prove their suitability for potential employers.
Here are some examples:
- Designers, photographers and writers spent countless hours putting together and updating portfolios.
- Anyone part of a professional association (doctors, nurses, pharmacists, engineers, architects, pilots, etc...) spend many hours studying for and writing exams (for free), and then every other year spend hours documenting all of the professional development they have done to maintain their credentials (also for free).
- Many higher responsibility jobs, especially in public institutions and government, require months-long application processes complete with multiple interviews, essay questions, and personality tests.
- Consultants can spend 40 hours or more crafting proposals for every new contract they go after.
- Academics and researchers require regular publications and often do regular peer reviews for free.
I doubt anyone gets a job of any importance based on a resume and a quick coffee unless of course there's nepotism or greek letters involved.
So how should programmers prove themselves to a potential employer? Would you rather have a professional association? or standardize on coding portfolios? Or perhaps you want to go back to the days of three essay questions and a personality test?
At the end of the day, interviewing - regardless of take home, whiteboarding, pair programming, leetcode - is a time commitment. Yes people with a family or limited spare time are at a disadvantage, but I don't think there is a way around that.
I spent a couple hours on it, but I realized that in order to get it working and tested, it would require many more hours of work. I started peppering the code with comments like /* here I would verify the validity of the data by doing X */ to show them I knew what needed to be done without doing all the grunt work.
I sent the code, but the manager got back to me and said he expected finished code that actually worked and not just pseudo code. I politely told him that I would gladly do the work if he wanted to pay me my fee I usually charge for contract work.
He declined so the company did not get a bunch of free consulting work, which I think was the point of the exercise. Too many companies get away with this.
I think this is a bad place to start. But how do you weed out people misrepresenting their ability? I think it’s a lot easier than most realize: first you need _something_ to talk about, have the candidate bring something they worked on. Then ask a ton of questions. Not just technical questions, questions about motivation, interest, reasoning, and taste.
The argument against this is that not everyone has time to make something on their own time and it just shifts the privilege to those with a greater share of free time. I think there is something to that, BUT: the power to choose the subject is in the hands of the candidate not the interviewer, that to eliminate all the test-like processes seems like a fair trade.
I refuse to do basic projects that make me learn nothing and feel like a wasting of time. Most of the time if the assignment is "not interesting" the future offer would be lower than I expect and the daily job is going to be boring.
I think that my point of view is going to be unique here, maybe is because I need a "push" to improve my skills (I'd rather not to be fair), but although I have been rejected from some processes after a long and hard assignment, I have learn something and I'm happy (almost grateful) about that.
I think we need to stand up and advocate for the expectation that corporations should provide more value to society than just print money. They should be held accountable to creating a strong, and healthy labor force.
For time bounding, I love the time estimates they give you for a decent sized problem. A major company told me that a feature extraction problem + report should only take 4 hours. They provided me with some large CSVs representing DB tables (which contained some malformed data) and wanted a brief report back.
That's unrealistic in 4 hours even for a great performer. Then you spend the weekend on it and they won't give you feedback on it because of the suits in legal.
It's disrespectful.
The feedback is usually "well, but I want to make sure I can take all the time I need!".
Home assignments should be short and sweet, it does not take a huge complicated task to see who can and cannot write programs. There are differences in ability of course even among "good" programmers, but the main difference is between those who truly cannot do anything, and those who can.
I am assuming an active job search where you have more than one potential employers.
If the test task is simple and you have the time: do it. If the test task is interesting and you really want into this company: do it perfectly. If the test task is simple, but you don't have the time: write an email explaining how you have a full time job with lots of interviews, tell them your CV clearly demonstrates you have enough experience, ask to forgo the test task. If the test task is hard and you don't have the time: explain how you would do the task, ask to forgo the task as above, walk away if they don't agree.
You would be surprised how often people will skip steps of the interview process if you ask nicely. This way I skipped tedious home assignments, psych tests and leetcode home assignment (not even hard, just told the company I don't want to rehearse dynamic programming with so many interviews lined up).
In my recent job search I had two good offers and incidentally both were the ones where I did the test tasks 150% because they were interesting. So do them if they are worth it.
A dr would not do a quick surgery and a power plant engineer won’t build “just a quick test reactor” because they have credentials and the hiring bodies trust those creds.
In software is the problem that we don’t trust what the resume implies? Where does the deep distrust come from? Why don’t we have an equivalent to a medical, legal, or engineering accreditation?
(I’ll state the obvious ghosting was wrong)
In software, it would not be enough just to require a degree. We'd have to also have some auditing entity that ensured the software written in practice conformed with the standards of professional engineering. And as the industry stands today, that applies to 0% of software created in a professional setting, never mind that very few corporations would be cool with auditors poking around in their code, at least without strict NDAs.
Not that I'm arguing that we should go this way. I for one embrace the wild west era of software for as long as it lasts. But if we were to give accreditation any weight, it would have to come accompanied with a standardized method to verify the accreditation is actually meaningful.
Auditing and forensics does seem to get applied more frequently in cryptocurrency projects, but as you said, that sort of accreditation still has to be meaningful.
The most reasonable company I have dealt with was Trip Ninja, an Amazon gift card for my time. Regrettably I tripped up, and messed up the test, timed tests are my Achilles heel.
“Home assignments” suck because they can be unbounded and tend to disadvantage people who might not have a lot of time.
“Pair on a feature/bug” sucks because it’s quite hard to many people to demonstrate their skills in that stressful environment with someone they don’t know.
“Whiteboard Code Golf” sucks because it hands an advantage to people people who have the time and energy to grind shit they won’t use in their roles.
“Paid trials” suck because they’re not something that everyone can contractually participate in.
I honestly don’t know what the best approach is, from either side. I have stuck with a single take-home for devs on the basis that we can establish a clear rubric for evaluating them and try to be at least somewhat objective with what we are looking for. Maybe a better approach would be to let the candidate guide it a bit, and offer a selection of different evaluations that they can participate in?
At least I agree it’s important to maintain a clear line of communication and respond to everyone. I know it’s easy to let people fall through the cracks as you scale; I hope I’ve never done it myself but I can definitely advise that a robust ATS helps to make sure every application is tracked.
- initial interview with HR
- technical interview with devs
- home assignment ( you can find it on my GitHub with the name 2048 )
- discussion of the assignment with the same devs of the technical
After every step they said they were very impressed, I did reach the step where they were scheduling the final interview with the CTO then they weren't able to schedule it and they disappeared for months with the excuse of hiring freeze. Apparently they hired someone else and they decided to put me on hold indefinitely
Programming is largely solo work punctuated by a high-bandwidth communication (presenting your solution to the team, review, etc.) A take home assignment with a follow up simulates the actual job environment, especially for remote work. No other coding interview technique comes close - portfolio review doesn't show how you worked only that you've worked, in-person coding challenges and whiteboard interviews are so far removed from the daily experience.
2kloc / 6hrs does seem excessive though. My advice to interviewers - keep the exercise time-boxed to 2 hours (And actually have some developers work through it internally to confirm this!) "How far can this candidate get on the problem in 2 hours" is a better signal than "Can they build the perfect solution with unlimited time". And time boxing the assignment gives you lots to talk about - "If you have more time, what would you add or change?" is a prompt that should yield another high-value signal.
As a company, you can't really provide feedback on why people failed the test. Hell of a lot of people will get defensive and get into arguments with you that you don't have time for.
I agree however that one should at least tell you you are being rejected instead of ghosting you. That's just bad manners.
I find this to be a bullshit cop-out. You can always ghost them after providing initial feedback if you don't want to engage in arguments.
Happened to me once. The best thing is I explicitly told them beforehand I had little time to spare for home assignments. I got rejected after almost a week worth of work total. I should have billed them...
Otherwise I’d write to them and tell them that you have an issue. Chase a bit, presumably you don’t know the reason for the ghosting.
Having said that they could be ghosting you because they are assholes. There are unfortunately a lot of inconsiderate assholes in tech.
Asking someone to do so much work for you and dismiss them is a dick move in any culture.
"Not texting back" would be the equivalent of a company simply not following up after you'd submitted your resume; I'm sure OP wouldn't have complained about that. This is more like, the girl texts you back, said, "I'll consider going on a date with you if you do this 8-hour thing for me", you did the 8-hour thing, and then she didn't contact you after that -- yes, that's basic failure of decency.
They rejected my application before I got too far with the demo, which would have taken me a lot more than six hours to finish. Although it might have been an interesting addition to my Github repo, it was kind of tailored to this one company.
I'm not disagreeing with the OP, because I probably wouldn't find an interview project nearly as fun. I wonder, though, whether anyone has been successful with a one-off, company-specific project like this, as opposed to a general portfolio of projects that you've made over the years.
I then time myself doing it.
I figure that means the applicant time commitment is under 2 hours, which seems not too bad.
Before doing an assignment just work out the “pot odds” and see if you want to raise or fold. Yes it is basically poker!
If you are doing it because you don't believe someone's ability, you should either pay them or be honest and say you don't believe their experience, in which case it is up to you whether you want to do it for free or not.
I find the ghosting unforgiveable though, I can't understand somebody being in contact and you simply not replying even if to say "sorry, we aren't taking you". Are people so insecure, they don't know how to reply without sounding aggressive/opening themselves up to lawsuits or something?
I don't think it's a question of fairness at all. It's certainly inconvenient and the longer the test the more likely I am to dismiss it off hand but a couple hours is ok, more than that needs good justification. If a company is offering me a salary thats 90th percentile or above then I don't see the problem with having to jump through some (reasonable) hoops.
The ghosting part is unquestionably rude though and any such company should be named and shamed so that people can avoid them if they want.
-----
For engineering roles where the person is expected to write copious amounts of code to ship functional features everyday, a 2.5-hour machine coding round to assess coding competency (as part of an interview loop that also has other rounds for problem-solving, data-structures/algorithms, high-level systems design and low-level design) is standard in multiple companies I have seen (and advocated for). While coding round is a strong filter, it is important to assess the interaction of the candidate in these other rounds that are non-coding but designing/problem-solving rounds, especially in high functional domain complexity industry like e-commerce or payments etc.
In the machine coding round, the candidate should be given a well-defined written problem statement (from an internal question bank of well-calibrated problem statements) along with test inputs and a few test outputs, and in the first 5 minutes ask them to read the problem and ask any clarifying questions. Then leave them be with a dev machine for next hour. If they choose to, 10 minutes in, they can discuss the approach they are going to take with you. Then after an hour, you can discuss if they have a working code that passes the test inputs. If yes, you can give them an enhancement/modification to make. and leave them be for another 45 minutes. Then, take the last 30 minutes to discuss the solution they have come up with and understand their thinking style w.r.t coding conventions, style etc. If they couldn't complete the enhancement, discuss a potential solution (give a hint) with them and see if they can tackle it. Write calibrated/standardized evaluation notes for your interaction so that others (hiring panel) can understand and decide. Existing engineers inside the company should have tried the problem statements themselves and calibrated to see if the problem is meaningfully solvable in the allotted time.
I don't understand the practice. Maybe someone here can fill me in on why it is considered OK from the recruiter perspective. Too many folks asking for feedback post-rejection? If that is the case - automate the message with a generic rejection.
a) be 100% absolutely certain that you definitely need this to evaluate the candidate (you most likely don't)
b) if you do then you owe it to them to study their submission, run it, test it and provide clear feedback and honestly if they've broadly met the requirements you should interview them. The submission could form part of the interview process - maybe you pair program some changes together, or discuss part of the implementation. Basically if someone delivers six hours of work, you use it.
c) No matter what, you communicate with the candidate throughout this process. There is absolutely no excuse for ghosting candidates. None
1. set up a good GitHub (or other) profile; convince potential employers to look at that rather than test you (I've done this - it can work) 2. if a company still insists you "do the test", then either your GH is too sparse (spruce it up!) or they're full of shit - but you have another data point to work with 3. even the most trivial "toy project" test could be 1 hr for one person and 6hr for another, so what's unreasonable for you may be totally reasonable for the level they're aiming at
You also get an idea of what company cares about and if its a good match for you.
If you move to next step you did something right, if not then there are others that have done better.
Ghosting might be a problem like someone forgetting to process stuff, vacations, etc, just check with recruiter whats the status, dont be antisocial.
And if it took you 6 hours to do, you chose to do it, and hopefully you learned something. There is no implied obligation on other side to pursue you just because you took on challenge.
In both case, if you are going to do home assignment, my question to you is: How good is your offer ? I refused several HA when interviewing with companies because their offers was just not good enough for me to take several hours, on top of the regular interview, to do it. If you do HA, make sure your candidate have a reason to want to take several hours for you.
It's definitely a strike against a company, but it's something I would put on the scales against how good this job would be. Sometimes it might be worth it.
So far, this approach worked very well for us.
Maybe someone else is more hungry, more ambitious, more game than you. They deserve their shot.
When I have given assignments like this, I said “the result of this work belongs to you. You can show it to any prospective employer. In fact, if you can show me a portfolio of your work, then you do not need to do this assignment.”
(I hire testers, and they rarely have examples of their testing work to show off.)
Combine this with relatively low salaries even for experienced developers, I don't think it is something anyone should want to do, no matter how much you'd love to go there.
My wife is a lawyer, she had to spend 2 years (in the US it's like 4 years at $80k+) and another 40 hours a year for life to maintain the qualifications that allow her to skip the equivalent of coding interviews.
I'd rather live in the world of none credentialed programmers to be honest.
So, what ends up happening is that nearly everyone in the interviewing process ends up half-arsing it - from the recruiters who prefer to contact nearly everyone eligible based on some keyword searches - to the engineers who have no time to be thorough - to the hiring manager who usually has a bunch of fires to extinguish.
A few companies do tend to put quantifiable metrics around their processes but they're usually too reactive and inconsequential.
The interviewers have a vested interest in being more productive by hiring the best people, but at the same time, there is personal friction against hiring someone good/better when your own promotion is hanging barely by the 'meets expectations' thread.
Now the assignments - they're almost always totally useless. Like, build a new app based on a third party API and then make sure to test it. Useless because of multiple reasons - the API is nothing like what you use at work; your current product team has coalesced around some archaic architecture that no Greenfield app would use; it's the take home assignment of the month (like the favorite 'build a movie app using the OMDb API'). A better alternative would be to open up your own API and perhaps a module of your current production app and add something to it, create a PR, or even just fix a bug or two. I actually think asking candidates to document a module is much more useful than asking them to write code you would never use, even if you pay for such an assignment.
Onsite interviews are even more useless - it's usually the most vocal people asking esoteric questions about older technology or algorithms that they've had success with candidates stumbling upon. Again, almost none of the whiteboard exercises or questions are from real-world scenarios.
You could pay candidates for the interview process, but that doesn't even address why interviewing is so fundamentally broken. I bet we would have amazing software products if we just interviewed better, especially when companies have been able to lobby for right to work laws nearly everywhere.
This is where you messed up. Set a hard limit of 1-2 hours, and mention edge cases in your comments and move on. It also depends on the stage of the interview. If you're in the early stages then they probably just want to see that you know what you're doing, and it should be fairly fast with the right frameworks, scaffolding, starters, etc.
Additionally, software development seems to be the only area where the candidates get bent out of shape being expected to prove they know what they're doing. Take home tests are bad and have to be confined to an hour. Whiteboard interviews are bad and don't accurately model the day-to-day. Leetcode is bad because it's 10x harder than anything you'll work on. How exactly do we expect someone to get a sense of our skill at programming if we eschew every opportunity to... program for them to review? The exception might be extremely well-known people who have extensive public histories on GitHub or something similar, but by definition they're in the minority and 99.9% of companies don't need someone of that caliber.
I interviewed with a health tech startup and they had a good way to do it. We set up a time ahead of time when I'd be given access to the repo. 90 minutes later the interviewer was going to clone the repo and whatever was there was what he was going to look at. This was after a previous ~hour long interview where we did the "tell me about your experience" rigamarole that some developers seem to think should be enough to get them any job.
FYI—Graphic Designers have been “bent out of shape” over spec work in hiring for ages.
At least in my experience, people in most fields aren't expected to do free work to show they know what they're doing. In general, past jobs, references, and portfolios tend to be used.
Yeah, let's fail someone who cares about showing their best self for an interview and takes pride in the craftsmanship of their code because we asked them to spend no time on it and probably get something that doesn't even work.
Not to mention there are probably other candidates that are taking just as long but not mentioning how long it takes them when turning it in, so the employer might start having an unreasonable expectation of the quality of work that can be done in 1.5-2 hours based on other applicants, and if you actually do follow instructions and only put that time in, you're likely going to be compared unfavorably against those other applicants.
Only way to properly enforce that limited time is to bring someone in to take a test in person, which is what my last company did (I barely passed and watched hundreds of people get brought in and not pass in the years since...in fact I saw only a handful of people pass that stupid test, part of the reason we never hired anyone, even though we needed to).
Your last point isn't true at all, it doesn't need to be in person it just needs to be coordinated. I mention one real-life example of this that worked out very well at the end of the comment you're replying to.
The only companies that asked me to do a take home are ones with horrible, unprepared interviews where it was clear interviewers were just kinda talking and feeling around for... something.
Happy to do a free code review if it's in one of my native languages.
It seems like it was implied that you were a serious candidate if they asked you to do this. And the implied understanding is that they would at least review your work and provide feedback.
Send them an invoice!
- The work produced is not used in any way by the company other than to assess the candidate.
- The project has a time limit to prevent those with more time to dedicate to it being able to do better (reduces bias).
- The company puts time into reviewing the code that is commensurate with the time spent on it (a 3 hour test warrants at least 45 mins of thorough code review and feedback writing across a couple of engineers).
- The company provides options to take the test at a convenient time, or even on-site but unsupervised if desired by the candidate.
Asking for payment to take part in an interview isn't a great approach assuming the above are true (which they are often not). Companies don't pay for interviews as general rule, but conversely should be clear about the full time commitment so that candidates can make informed decisions about taking part.
Why should we be any different?
Putting an hour limit on them makes sense.
I find it weird that you guys don't want to spend some time to show your skills to the employer, without getting paid. These are not real projects. You're not benefiting the employer when you finish them. They also spend some time evaluating your project submission.
(I'm not talking about the companies trying to make people work free by giving them parts of the real projects in disguise)
That's several hours of your time that could be used for things that will have a more immediate benefit: working your current day job, working on open source projects, or maybe on some other hobbies or even spending time with your family. Or even sleeping.
I could evaluate a project in 5-10 minutes. Or an employer could have a fully automated system for the first round of evaluation.
The problem isn't benefiting the employer, it's that the costs are asymmetrical. The employer has no incentive to act in a way that respects the applicant's time.
There's a similar problem in real estate. If buyers weren't required to place a deposit with their offers, they'd waste the time of countless sellers.
How is it possible to do anything meaningful in just an hour?
As far as ghosting is concerned it's a bit baffling to me, seems far more in a company's interest to promote the notion of not ghosting so that they actually stand some chance at all of applicants letting them know when they've dropped out.
Is a prospective employer supposed to suss this out entirely through verbal questions?
I've also had interviews which just talk about work I've done before. Most industries only need that, and I'm coming around to those being the more valuable questions.
Yes
It's really not that hard to ask questions that evaluate a candidate's ability without devolving into trivia
Programming is merely logical thought converted into a terse form that a computer can execute
Ask how to solve a problem; ask what needs to be considered to solve it; ask what corner cases does the initial answer not cover; ask for something like a confidence interval on how likely you think errors will happen (and how you plan to respond to them); ask how the program needs to change when you convert it from a cgi-bin/ C program to Ruby or PHP or Go; ask how a solution looks different in any two of the languages the candidate knows/likes
Would you only hire a surgeon after they demonstrate an appendectomy? Would you hire a mechanic only after they demonstrate building a transmission from spare parts?
I think these tests are a good way to have people without previous experience or formal education prove themselves. If you have skills, but no credentials, no public Git[la|hu]b, and no experience, you can still prove yourself with one of these programming tests. However, once someone has a year or so of experience or has finished formal education, the decision should be based on their credentials and the interview, not some kind of demonstration.
It's still silly to expect to have to work for you for free. You can discuss technical topics (design experience, programming language preferences, etc.) without doing a whole exercise. Honestly, it's insulting that people will ask you to come in for an interview and basically tell you that they don't believe what you've put on your resume and start quizzing you to confirm that you know what you claim you know.
I invest 6h of my time as a highly paid engineer to do a home assignment and at best I receive a reply "sorry we found a candidate that's a better fit?".
That's what's unacceptable.
If it were a ship building company asking you to do 6 months of ship building business logic to get a job that would be different, for example.