Show HN: I made a job board that doesn't allow HackerRank tests
borderline.biz
borderline.biz
Genuine question here: what's the better way?
I'm not a fan of whiteboard coding challenges, even though they got me my first "real" Bay Area tech job.
On the other hand, take home challenges eat up a lot of time and credentialist filters like going to a name brand school (or even having the right major) filter me out.
My go-to for when I'm in a hiring position again would be recommendations from my personal network, but that has its own flaws and doesn't scale very well.
I think at the core of the problem is that most companies are extremely risk adverse in their hiring and would gladly miss a Jeff Dean type of super-performer in order to reduce their number of bad hires.
I work somewhere in the middle of biology/neuroscience and ML. Interviews for essentially the same job (e.g., building brain-inspired robots) are hilariously different depending on how the org views itself.
In a "science" organization, the interviews mostly involved talking about what I've done in the past and how I'd approach new issues (technical and behavioral). There were certainly a few probing questions about things that seemed designed to check up on claims, but my experience seemed mostly taken at face value. A coding test, for example, was skipped because an interviewer had already seen some of my code and thought it was fine.
"Tech" interviewers seemed to want everything "proved" on-site during an all-day slog. I reversed strings and traversed graphs on a whiteboard. Someone quizzed me on facts and figures about the human eye (which they were reading out of the same book I learned them from). To me, this seems to test immediate recall and stamina/stress tolerance more than anything actually related to my R&D skills.
I think something more akin to a thesis defense would be better than what we're doing now. Prior to the interview have the candidate write up a description of a project they worked on and particularly liked along with a set of slides. Have the candidate send this presentation to the main interviewer who will distribute it to all of the interviewers. The interview then becomes a presentation where interviewers can ask questions about the presentation. I've seen this done in industry, but it was only when we interviewed folks who were PhDs coming from academia - I don't see why we couldn't do this for all kinds of candidates, not just those coming from academia.
1) What did you, personally, work on previously that's relevant to this role? This usually means pretty in-depth discussions about previous roles and projects, accomplishments, tasks, skills gained, etc.
2) How do you approach tasks and problems at work? This is usually a combination of asking about why the candidate did things a certain way previously and how they would approach theoretical situations in the future. The goal is to uncover the mindset the candidate uses to actually get stuff done in a changing business environment.
3) How do you interact with teammates and managers? This is the concrete version of "culture fit" and includes asking about various interpersonal situations likely to have been encountered in the past as well as discussions about how the candidate would handle quirks of the office environment at the new company (ex. some companies love people to make decisions quickly on their own while others hate it and mandate team consensus).
4) Do you have the base-level skills needed for the role? This is often far more than the technical aspect. After some basic questions to establish that the candidate knows the basic processes, software tools, artifacts, and terminology, it's much more important to establish if they have the chops to actually be successful rather than a "task completed" robot. Can the candidate communicate ideas and opinions clearly? Did they previously work in a collaborative way rather than an adversarial way with teammates? Can they present findings and issues to management effectively? Have they previously worked with supporting teams and vendors outside their main role to solve problems? Etc.
In most jobs the technical skill part is a small piece of what the person will actually be doing so the interview is a means to collaboratively establish if the role is a fit for the candidate as a whole and vice versa. If I only looked at technical skill and immediately disqualified people for failing concocted and unrealistic tests, I'd end up with mostly shitty candidates who aren't actually very good at the wider job.
I'd be extremely surprised if Jeff Dean (at any point in his employment) would fail a SWE interview loop at Google. The Google interview is heavily based on data structures and algorithms, and Jeff Dean likely knows them cold.
Sure there's always a degree of luck, and Google errs on the side of false negatives, but there are certain people that are so high above the hiring bar that luck becomes too tiny a factor to be relevant.
That's pretty much what Leetcode is. When I interviewed at Google I was asked tree traversal and hashmap design questions that were very similar to leetcode questions.
If someone has worked at 6-8 places over 15/20 years covering a range of skills required I don't see the risk.
1. It fosters meaningful conversations about code that is not dependent on the candidate solving a toy problem.
2. As you said, it tests a candidate's ability to read and understand code.
3. It's actually representative of the type of work you _will_ do at any given company, i.e. code review.
The major downside is if you give the candidate code in a language they're totally unfamiliar with then that candidate will likely bomb. I think this can be mitigated by sticking with mainstream languages like C/Java/Python, or domain specific languages where not-knowing would be a deal breaker anyway, i.e. must know Swift for iOS dev.
I was filled with anxiety working under the time pressure to understand a document of code which I had very little context of, and no previous relationship with the reviewer so I kept thinking to myself whether or not the question that came into my mind was stupid obvious.
In contrast, when I had take-home tests, I scored exceptionally high on all of them, leading to offers. I recognize that a lot of people here hate them because of the time involved and it's akin to spec work, but IMO it felt much more like my usual day-to-day life as a developer.
One of the take home tests had existing code to work from but again, working at my own pace and having access to online resources to read docs was much less anxiety provoking.
You say:
> One issue was the syntax was so weird compared to the type of code I'm used to writing and reading from others in library code that it was hard for me to comprehend the logic without the benefit of looking it up.
From the point of view of an interviewer, that's feedback I'd be interested in. One problem I've seen in some of our code is an engineer using a completely valid but very obscure method of doing something, and then the rest of us having difficulty maintaining it. I would want to hear, "this looks hard to maintain." But I understand that in an interview, it can be difficult to assess what the interviewer wants to hear and that can be nerve-racking. I personally think take-homes are a reasonable compromise, but not everyone does.
It also works even if the person doesn't know the language because you'll get to see them mapping this new language to something they already know and it can give a good understanding of how well they think of things beyond just the code in front of them.
Pick some interesting problems from there, and without asking the candidate to actually code a solution, discuss the problems with the candidate. Ask for ideas on how to approach the problem, and suggest some yourself if at any time they appear to be getting stuck.
After they, possibly with guidance from you, have reached an approach that if correctly implemented would solve the problem, including meeting time and space requirements, discuss what issues might arise in actually implementing it, such as edge cases that might require special handling. This would be a good place to talk about how they would go about testing an implementation.
Finally, go to the discussion on LeetCode for that problem. There you will find various solutions posted by others. What I have found looking at those is that a lot of them don't meet the time/space requirements, or miss edge cases, or sometimes are just wrong. Go over a few of these with the candidate and have them criticize them for you.
You can save some effort though and just hire anyone who passed Chinese or Korean grueling school system with emphasis on standardized tests. LC-based interviews follow the same tradition.
I have create a git repo specified for the question, with some code working in way, but buggy, such as logical bugs or race conditions...etc. with a lot of not business relevant assumptions. I usually told the candidate what this code supposed to do, and ask him to review the code first, then tell me what was his impression on the code base, and what were the problems, and how he can fix it. By doing this, I can judge the candidate if they are good at general coding, debugging and and comfortable to work on projects which they are new to, as well as their mindset on writing maintainable software. A good candidate will not just fixing the obvious issue, but propose some architectural changes, then I will ask them what are the strategy to do so as the project has existing dependents, this can give me signal on how the candidate coordinate with other teams. Then I will ask him some more questions related new features, and scaling..etc. to measure his design, communication skills and more.
It never failed me for senior level positions hiring, that either the candidates were hired and proven to be a great match, or they got a better offer elsewhere.
One strategy that's been successful for me is skipping the technical screen and jumping to a narrowly-scoped paid assignment. If they do well, I give them a larger paid assignment, and we work our way up to regular work together. If they do poorly, I pay them for their time but tell them it's not a match.
I'll add the caveat that I've only used this to hire junior to mid-level developers for part-time or short-term work. I've never tested it on a full-time hire, and I imagine that would be more difficult.
I've never written in detail about hiring developers, but I've shared similar strategies for hiring writers,[0] illustrators,[1] and editors.[2]
[0] https://mtlynch.io/hiring-content-writers/
It really explains a lot about the C-level people and their lack of engagement with work (for my company at least!); they don't seem to be doing their full jobs as, well, they aren't. I really need to figure out how to get on this carousel.
Why would this exclude people with family obligations? When I say "narrowly-acoped project," I mean 5-10 hours of work. I've never gotten an offer for a dev role where the application interview process took less than 5 hours, and those were unpaid hours.
5-10 hours would be similar to a typical interview loop yes, and if that was the entirety of it then it's not more time consuming than the typical engineering interview.
In other words, some visa holders in the US are very much allowed to work, but not contract work.
Is it really worth it to lose a day in your weekend for a couple hundred bucks and the hope of getting yet another assignment or a job?
And, if we're being really honest here, there's a ton of informal backchannel talking that goes on, so your interview "performance" regardless of medium is affected by that at many places. I think if you aren't actively working to make your interview process objective, it's being strongly affected by many biases and informal relationships.
That's all to say that on the balance, taking up "some" or "all" of a candidates time isn't that interesting of thing to think about because the process is so broken in so many other ways and this one doesn't have a huge impact.
I can't get that from a take home assignment.
I feel like this is the solution that values candidate time the most. They don't have to do an interview, and I'm paying them for their time immediately.
The alternative is to have them take time off of their dayjob and travel on-site for technical interviews.
In the real world, candidates are greatly appreciative of take-home assignments that let them work in the comfort of their own home, on their own equipment, at times that work best for them.
I can understand not wanting to do that if you're a full-time employee, but if you're a freelance developer actively looking for jobs, why would a paid project be worse than an unpaid phone screen?
It always seems strange to me that people won't bat an eye at taking a day (or more) off work to come on-site for technical interviews, yet the idea of giving someone a 6-hour project do at home, at their leisure, is somehow a bridge too far.
In real-world hiring experience: I gave candidates a choice between on-site interviews and take-home projects for a while. Zero people chose on-site interviews when given the option, so I stopped offering it.
The bottom line is that interviewing requires time. If someone isn't willing to lift a finger or allocate a few hours to an interview, how are we supposed to gauge their competency? The "just trust me" model doesn't work at all.
One of the key differences is the company has "skin in the game" at an interview, but not in the 6-hour project.
When you interview, you know the company has included you in a shortlist and they will put time into getting to know you. For the time put in, you have a chance to pitch yourself, to at least get across your general abilities and personality, and some people will pay some attention. Also you can learn more about the company and the people you might work with.
When you get a take-home project, it's like the way spam is cheap: It takes almost no effort for the company to send the same project to a lot more candidates at low cost.
You do the work, but there's no guarantee the company will really assess it. Sometimes the company drops the ball and just "forgets" the application, or discards it the same way résumés are filtered: By skimming though 6-hour project submissions and casually throwing some of them away. It's not uncommon to not hear back anything from a company after sending the result of working on their project.
Some candidates will enjoy doing a project more than an interview, but that's not the only consideration in determining their value to the candidate.
Personal experience: I have never had an interview last more than an hour. I have also been hired twice on a "just trust me" model and it worked reasonably well.
Replacing a day of interviews with a 6 hour project sounds fair. That isn't how most companies use projects.
Kudos to you. You have worked out one of the fairest ways to gauge someone. May your tribe increase!
I have 20 years of experience in X but prove it? Why's the burden on me to prove what I've already told you? Are you saying I'm a liar? If my date of birth is on my cv do you expect a birth certificate to validate that information too?
Anyway, that's always been my philosophy - hire on facts and fit, fire quickly if anything was wrong - and it's been pretty smooth. The great thing that it does is it prevents testing for futile thing that you will never use in your position and that are just elitist criteria or vanity wishes.
Lots of jobs aren't all that demanding.
US hiring practices expect to see something that validates your birthdate, so kind of.
Fast firing can work, but none of the places I've worked were prepared to do it. Similar to what a sibling commenter posted, years of experience is something, but what you got out of it is another. There are plenty of people who have spent a lot of time doing something and not gotten much experience. In my mind, a good interview process figures out where the candidate actually is with regards to experience and capability, if they fit with the company, if the company fits with them, with as minimal a time commitment for both sides as possible.
It's not great for either party if someone is hired and fired in a month.
Indeed. People also underestimate just how quickly you can learn in the right environment.
The reason this philosophy is misguided is not that candidates flat out lie all the time (though that happens too), but more that resumes are crafted to inflate, exaggerate and obfuscate to make the candidate seem more qualified than they might actually be for the position upon closer scrutiny. I say I have 20 years of experience in X, but 4 of those years were just hobby projects in high school, 4 studying in university, another 5 in entry-level positions in a semi-related industry, etc etc. It would be a huge waste of time in obviously unnecessary hirings+firings to blindly trust stated experience on a resume without verifying its accuracy first.
Engineers who are 100% honest, overly humble, and fortunate enough to have only worked at top-quality companies tend to not understand the need for verifying the skills that people put on their resume.
In the real world, some of the best candidates have the worst resumes. Some of the worst candidates have the best resumes. And perhaps most dangerously, some of the worst hires tend to be the best at lying and exaggerating because they've used their charisma to get them through life.
Making bad hires is a very costly mistake for the team. Not only do you lose out on months of team-building and project progress, but it's bad for morale to see people hired and fired quickly. In reality, bad hires are going to stick around for 6-12 months or more at any company of moderate size company.
Also, not all experience is created equal. It's shockingly common for applicants to claim 5+ years of experience in something, only to discover that they only touched that topic a couple times per year for 5 straight years, for example.
Yes, and the way that I know this is by performing the tests you're arguing against. I've interviewed plenty of people that could converse fluently in buzzwords, but could not write code. In the actual job, it's necessary to be able to write code.
Imagine you're a team of 2 top performers. Your workload is increasing, so your manager gets approval to hire a 3rd person so you can take on more work. Unfortunately, your manager is fooled by a charismatic person who greatly exaggerated their experience on their resume, to the point that they work about 1/4 as fast as the rest of the team and tend to code by trial-and-error.
Suddenly, the expectations for your team's output have increased by 50%, but the 3rd person can only really contribute another 10-20% at most. By extension, your job just got harder.
Worse yet, the average talent of the team has now gone down significantly. You spend more time fixing bugs and chasing bad work. You're also faced with the fact that you're probably getting paid the same as the over-leveled person next to you. This breeds resentment, wastes time, and ultimately leads to good people quitting for other jobs.
Alternatively, imagine if the company hired quickly and fired quickly. Every time a new person showed up, you had no idea if they were actually good or if they were just rushed through the interview process. You also don't know if they're going to be around after the 3 month probationary period, so why bother building a relationship? High turnover like this tends to destroy morale.
You can have code challenges that aren’t HackerRank or Leetcode.
Using problems from those sites is a signal the employer or team is lazy.
This is the fundamental part that is broken in my experience.
They are often challenges/coding puzzles, when rather white board exercises should focus on collaboration, and solving problems with code.
Having a "gotcha" that shows rudimentary CS understanding is fine (such as understanding why the obvious O(N^2) is a bad solution). But having a gotcha that requires some crazy leap of faith guess (or having seen the problem before) is not.
I admit, my fondness for a candidate increases with the level of complexity of coding problem they can solve; however needing hints on the harder problems doesn't mean they are not qualified.
Questions are mostly about technologies that hiring team needs, the more the better, and the deeper the better. I never had a problem getting the extra experience not found on previous workplace from my pet projects and blogs before moving to US.
Funnily, I've seen people who moved back from SF Bay Area, and struggled to find a job there, despite being very good at whiteboard and CS problems. Turned out eastern european companies don't care about CS problems really.
I think the candidate should be given a choice - whether they want to solve one of these tricky puzzles or work on a domain problem.
I did whiteboards for Architect positions, but once again, I just asked the candidates to draw architectures to have a friendly discussion.
Do they quiz them like they did to the residents on Scrubs? Do they pull up a list of symptoms and ask what they would test/try next? Do they show them x-rays and ask them to spot the coronavirus?
Seriously? Or do they look at your degree and experience and conclude that if you worked at General Hospital in Atlanta for 5 years you can work at Metro Hospital in Philadelphia?
I honestly don't know how other professional industries hire industry-hires (not college grads/interns) but I only hear about quizzes and riddles and online tests and shit like that in programming.
Hell, do they do the same shit for electrical engineers? Chemical engineers?
I really want to know if they do and if not, how is it Boeing can hire a EE without a whiteboard puzzle but can't hire a CE without that bullshit?
I'm not that opposed in general to whiteboarding if you feel like you can't assess the candidate. However I do feel that it sometimes feels like credential and experience similar to what you describe in the medical field: if a doctor had 5 years at an inner city hospital, I don't think they would doubt his credentials to the point the want to make him put cell structures on a whiteboard?
Do you really want the same for programmers? "only people with Ms or PhD from accredited universities can apply, and diploma is invalidated on some mistakes"
I think it would be a shame to bring this sort of thing to software development, but it would allow us to get rid of some of the hoops people have to jump through to ensure they can actually do the job being hired for.
Physicians are similar in that you have board certification and residency which provides a baseline level of implied compatency. As I understand it, your residency can essentially set your career. You won't be able to change speciality, and moving up in terms of institution is difficult.
In practice this might be "you can only get hired at faang with faang on your resume", so your options if you don't get it right out of college are to do masters or hope to get a job at Amazon and possibly pivot that into a position at a different firm.
In some industries an engineer might need to be licensed. That requires passing multiple tough exams and working under another professional engineer to get their license.
As have already been reiterated in each and everyone of these threads (about leetcode/hackerrank/whiteboard coding) this is what the companies seem to think is the most proven, and produces most consistent results. But I would love to understand how other industries does this as well; does a propulsion engineer start to draw up a de Laval nozzle and calculate the exhaust gas velocity on a whiteboard? (again genuine question)
Other industries have higher and more consistent standards, and certification processes. The software industry is the wild west, and that is partly why it has succeeded and why so many self-taught programmers have found success in it. The other side of the coin is that the quality of candidates varies more widely than in other industries.
He's previously said that if he could only decide based on one or the other, he'd hire the candidate referred by a trusted source and skip the interview. In practice, this choice never comes up, but you can weight referrals much more strongly than interviews.
You tell them what the job is like, what is expected of them, and what the best way to work with you is, then you you ask if that's what they want, and how much it's going to take to get them in the door. That's it.
I wouldn't ask an accountant what the Corrodium 3 Deduction is; I'd tell them I need them to file my taxes, that I need them to own the process and chase whoever they need (even if that's me).
> I think at the core of the problem is that most companies are extremely risk adverse in their hiring and would gladly miss a Jeff Dean type of super-performer in order to reduce their number of bad hires.
I think you're probably right, but if someone lies to you and says they can do a job that they clearly can't, you should fire them.
This should be unusual because getting a new job can be stressful, and nobody wants to put themselves into a job they can't do, so it's rare that you even have to consider this -- but it does happen, and unless you're a sociopath, it's going to make you shy about ever having to go through it again, because you're not going to want to fire anybody. It sucks for you, and it sucks for them, and it sucks for the team who see the revolving door, but nobody benefits from the wrong fit in the long run, so you just need to suck it up and do it.
So be firm, and if it turns out you're firing a lot of people straight in the door, look inwards: you're probably not being honest with them (or perhaps yourself) what the job actually is. Figuring that out instead of some new programming puzzle to torture grads with is going to pay dividends.
This is absolutely why whiteboarding exists, and persists.
I have interviewed a lot of people at this point in my career, and it's just not possible to tell if candidates are good by looking at their credentials or job histories.
I've seen people with impressive credentials, or impressive job histories, end up being totally unable to produce working solutions to FizzBuzz-type questions. I'm not talking about Leetcode-style brain teasers; I'm talking about being unable to construct a working for loop and do basic array operations in a language of their choice.
I'm going to continue to do whiteboarding for candidates for this reason.
That honestly sounds to me like you found some people who lied like crazy about their credentials and job histories.
Side note: Why is everyone into job boards in 2020/2021? Was there some event or article to push this as an untapped market? Seems like so many people recently came up with the ever same variation of job listing services.
Job boards are a major part of the zeitgeist of collapsing middle class America.
Why now? And I am seeing this in Germany as well, which of course suffers a social divide, but much less extreme, or existentially threatening, than in the US. I assume the "job exchange" market is a natural monopoly, at least for its respective sections, and now there seems to be a race for it. But weirdly it's for the most part all the same: Listings. There doesn't seem to be a sustainable model, which avoids the problems of the past's iterations. Therefore I assume it's driven by business folks, not people with ideas.
I guess it's still a tedious experience for all parties involved.
I wonder why there isn't an OkCupid for jobs, but I think the problem is an asymmetrical interest in a "good match", unlike dating. As usual people fail to come up with good metrics and incentives and it's Goodhart's law all the way. Some problems just can't be solved by technology. But that never stopped anyone from trying over and over again... hyping their overfit tech to non-tech people.
I think the actual problem is with management on the employer side, not the job boards. They need to bite the bullet and accept they need real human interactions to measure productivity and reward it case to case.
There is/was a global pandemic making a lot of people unemployed.
If you're a coder and unemployed, what better way to get a job than make a job board? Potential employers will come to you and tell you about their jobs, and then you have a great leg up by saying, "I built the job board you posted to".
It's a great way to find a new job.
I recall Google rejecting Max Howell despite creating Homebrew for not having as solid of a comp-sci knowledge as they were looking for. Despite flaws of the product that he'd be the first to acknowledge, creating it should've been evidence that he has strong potential to grow into a person that could benefit the company.
And I guess I'm not surprised. Some of the interview questions I've seen people post online from their experiences with FAANG are so detached from the code programmers write on a day-to-day basis.
That said, there are lots of good sources for backfill (job posts) and there are still lots of really great ways to monetize users and applicants via direct sales to employers and pay per click or pay per application programmatic networks.
Instead it's an indicator of being able to memorize, recognize patterns and regurgitate tactics.
Ask why something should or shouldn't be strategically used can be a realer indicator if there is someone behind the steering wheel.
All the leverage is on the happily employed senior engineer side. If they're not happy or unemployed, then it's different. But for the happily employed, it's the recruiter/company trying to convince them to leave. And it's very reasonable to reject these interviews. An employed senior engineer already has a demanding job that requires at least 40 hours a week. And since they're older, they're likely to have other responsibilities like family. There's simply no time to grind for months on pointless leetcode problems.
If more senior engineers start immediately rejecting any recruiter messages with a simple "No, I don't do leetcode", the signal will be sent. Recruiters will start getting frustrated and tell hiring managers, hey many of the great engineers I contact keep saying no because of this leetcode thing, can you change that part of the interview? I can't find any good candidates because of this. That will force hiring managers to really evaluate if this leetcode interview is a good idea.
Don't be fooled if they say it's not about getting the problem right or they just want to see how you think about a problem. That was somewhat true a long long time ago when these types of interviews started and leetcode didn't even exist. And it's what they still say to make it seem reasonable. But the reality today is that the interviewer only cares that you can write out the optimal solution within the interview time. That's it, that's all the interviewer will care about. If you don't have the right answer, it's an immediate zero. They might care about other factors (like are you friendly, code quality, etc.) after you successfully write the optimal answer on the board. But writing out the optimal answer is the baseline, they don't care to evaluate any other factor until that baseline is satisfied.
There might be a few interviewers who still try to stick to the old way of these interviews, the original intent. But statistically the odds are against you. These interviewers are generally older and becoming fewer and fewer. The new generation of junior engineers are becoming senior and they only know the leetcode grind. In a typical interview loop, it will become more and more probable that there's more and more of these interviewers who expect the candidates to have done the leetcode grind and base their evaluation from that assumption.
It is 2021 though, and there really is no excuse for not using TLS.
How does a prospective employer evaluate someone’s problem-solving and programming chops?
But hackerrank/leetcode don't test for those skills. They test your ability to memorize solutions to trivia questions, typically things you'd never be asked to do in the workplace.
There's enough to learn to be good at software without having to also memorize arbitrary trivia.
I understand that at many companies, they just use pre-canned questions and all the above criticisms apply. But I don’t think this is the fault of the platform; it’s a conscious choice of the company to do so.
That's not really true. Most of them are variations on a relatively small set of techniques. Dynamic programming, back tracking, tree traversal, sliding window, greedy algorithms, and minimal connected tree will solve 90% of them. And those are all techniques that are occasionally useful.
Sure, some of them will have some esoteric math or algorithm that makes the solution trivial if you know it. Or sometimes there will be some confusing wording. But so long as they're not like that it's a pretty fair test.
Shouldn't we be testing for skills that you will be using 90% of the time in your day job rather than skills that are occasionally useful (and thus could be looked up / learnt on demand if necessary).
I know developers who have written books on subjects related to development and are 100x the developer I am, but have no knowledge of DP beyond what was taught to them decades ago because they never used it when helping to build systems used by millions of customers.
Because it doesn't necessarily prove one's abilities. Each of those items have pretty obvious flaws:
- Whiteboard: Usually idealized problems requiring specific knowledge that is easily looked up, but not necessarily in one's short-term memory. Whiteboard is generally not the tool one uses to write code. It's very easy to bomb these sessions despite being competent.
- Hackerrank: Easy to cheat. Usually toy problems with idealized properties, i.e. they don't have the dirty edge cases you see in real world development. This also tends to be just a first-pass filter for getting an phone interview or maybe on-site.
- Take Home: Easy to cheat. May run afoul of existing employment agreements. Requires a pretty large up-front time commitment, and IME does not actually avoid the on-site whiteboarding sessions anyway. It's more of a first-pass filter.
> How does a prospective employer evaluate someone’s problem-solving and programming chops?
That's the million dollar question. As an industry we just don't have a good system for identifying competent developers. My personal approach is to stick with folks I've already worked with, who have a long track record of delivering. That doesn't scale at all and is not very good for folks just entering the industry.
I truly wish we had better answers and I support anyone trying to build something that explores the problem space. It's nice to have diversity in hiring practices.
I'm slow and methodical. I'm proud of it. My code runs for years in demanding environments without breaking and I'm famous for it, at least within my own network. My code is boring. The tech I choose is boring. The solutions I deliver are predictable.
Hiring an experienced engineer is not difficult. Can the person do the work? Can we see some examples? Will they get on with the team? How quickly can they get up to speed? Will they deliver value to you?
So when hiring, I don't care if you're slow and methodical, or super fast. What I care about is that you understand basic algorithms and data structure (lists, sets, maps, trees) and can apply them to simple problems without me telling you that using a hash map here will speed up the solution tremendously, or using a heap will simplify the code. Those simple problems are also taken from most common patterns we see in our codebase, which consists of lots of geospatial data processing and analysis.
And if you need more than an hour to solve a couple of those simple problems (let's say, find duplicates between two lists), you'll definitely have a problem fitting in with the rest of the team - since everyone on the team has solved those problems and agrees that we need people with at least this basic algorithm knowledge to be able to make a significant contribution here.
These questions also a first hour of the interview, the rest goes deeply into discussion of previous projects you made, different architectural patterns, etc.
result = set(l1) - set(l2)
Time taken: 20 seconds. I take your point that you need people with at least a certain level of competence. I found I saved those who were hiring an awful lot of time by sending them example code or asking them to give me a challenging weekend project as a test, perhaps even something they needed doing for real. But I dread these exam-like online tests where I have to have a right answer in 20 minutes. I don't work like that. Few people over 40 do any more.
And there are many developer position applicants that wouldn't be able to do it. If you actually solve it in 20 seconds, I presume you passed the test.
Is this a flaw or a feature? What I'm really asking is: is it a coincidence that some of these tests may tend to filter out an older demographic? It may be yet another one of those ways that some people can be filed away as not being a "culture fit" or whatnot.
To be sure, there are companies that do care about domain experience - and I personally value it as well. But that experience often carries a price that some companies will jump through hoops in order to avoid paying. I've seen that first hand, unfortunately.
I am ok with leetcode screens ( actually enjoy them) but I hate take home assignments. Worse is now companies asking you to do both.
Writing algos from memory on a whiteboard is not what I do in my job. It isn’t a subset of what I do in my job. It isn’t something I have ever done in a job at all. You may as well be giving me a first year chemistry final.
So it is just something I drill, forget, and then re-drill when I want a new job.
I am fine with take homes though.
Some people complain that the questions and exercises being asked are silly.
And most people are bitter about a rejection and want to vent.
The same problem happens with take home tests. It pretty much optimises for people with more time on their hands (i.e. people who don't currently have a job, people without kids etc).
In both cases a lot of good people will be filtered out. Some people think it's good to avoid bad hires at all costs but I don't think wasting people's time is particularly ethical.
there's market leaders breaking new grounds every day, and for those I'd say it might be appropriate.
then there's your average bookkeeping application that is at most transforming existing data based on problem already solved by the customers, and these are like the majority of jobs on the market, but still pretend to hire the best 10th percentile - except they don't provide a 10th percentile pay and the job is boring and soulless so they probably won't even have a 10th candidate apply to begin with
> How does a prospective employer evaluate someone’s problem-solving and programming chops?
how do factory evaluate the capacity to tighten bolts?
this problem was solved by Ford a century ago.
if one's building run of the mill business software with spaceship grade engineering, you're trying to solve a hiring problem you shouldn't be having to begin with.
Tell the interviewer that if you work on the L33T code problem, that you will bill them $250/hour, and 8 hours minimum. Since that is your standard consulting rate.
This will make obviously make them balk. But likely the company isn’t worth working for.
Provide work samples, and discuss them. If the interviewer cannot put in the time to investigate your work product, then they’re just being lazy, or they’re really just stupid. And they need to get off their high horse.