Hacker News Reader

Comment by kayo_20211030 | Hacker News Reader

kayo_20211030

Your experience sounds horrible, and unfortunately those sort of "tests" are just a horrible thing. Generally, do NOT do take-home exercises. That's your time, which has value; the company doing the hiring contributes nothing. At least, in a live-coding exercise both you and they have some skin in the game. As with all tests, good or bad, part of the test is actually doing the test. Often it's less what you know, or could do; and more about "can you do the test?", and mostly "doing the test" is a poor proxy, and the whole exercise becomes a waste of everyone's time.

View on HN

There's also the issue of scalability.

Grinding leetcode at least scales horizontally to a huge number of companies.

Homework is typically useless outside of the single company you're doing it for.

I generally refuse all takehome assignments unless:

1.) It sounds uniquely interesting and fun to do. 2.) The company is prestigious enough, or pays well enough, that making any effort to try to get the job worth it.

Completely agreed. Also in my experience the takehomes are even more susceptible to being rejected for absolute bullshit reasons. Someone told me the thing I coded up in less than an hour to solve their problem was “overengineered” so they wouldn’t be moving forward.

Also... I think at 100% of the companies that have given me a homework assignment, it has always been in addition to, not in place of, a standard leetcode whiteboard round.

Same here. Though it’s hard to blame them since some people quite obviously didn’t do the takehomes themselves.

Why would you be interviewing at all if the criteria for (2) isn’t met? Unless you’re just trying to get interview practice or keeping them as a safety option.

Sometimes people just need a job (especially in the US, where jobs are so coupled to health care). I've been in that position, and I'm very glad I'm not now.

I wouldn't dismiss an employer out of hand just because I wasn't already familiar with them but I'd be quicker to back out if I saw something that didn't inspire confidence.

The last time I did a take home exercise for a position I wrote something that was beyond the team's skillset. That was awkward.

That's a great reason to do the take home test. You learn a huge amount about the team by how they react to your code.

Or they just evaluate among themselves and never tell you what they thought beyond thumbs up or thumbs down.

Or just completely drop the ball and never get back to you.

> Often it's less what you know, or could do; and more about "can you do the test?"

It really depends on how you create the test and how you review it. You can prepare one where the actual solution is 10 lines of code and anything else is exactly for showing what the candidate knows / could do.

I used something like "read data from CSV, write it to sqlite, treat it like a mature production app, feel free to use placeholders (usage doc goes here), go nuts". Then the test was really about knowing about error recovery, encoding issues, documentation, error reporting, monitoring, ci pipelines, etc. Many badly organised tests don't make the whole concept of a take home test bad.

Your test sounds like what people complain about. You give them instructions but it's "really about ... ". Just ask them what you want them to know. If you hired this person and they turned in code without error recovery or encoding, you'd tell them they need to add it and they would. No need to obfuscate your intentions and have them read your mind in what you're looking for.

No take home test is even needed. A simple conversation would let you know if they understand the importance of those things.

> Just ask them what you want them to know.

You can't really ask about those things directly. If I ask "(how) would you add monitoring to this?" you can make something up on the spot. If I ask "what else would a mature production app contain?" that will give me a better idea of what you're familiar with.

> you'd tell them they need to add it and they would

Different levels. We were not after juniors in that case, but people who can do independent work. The question would be way more specific otherwise.

> No need to obfuscate your intentions and have them read your mind in what you're looking for.

It's not about mind reading. For that role, you either know how to create/manage serious applications or not. Your specific ideas may be different than mine and that's fine, but the general areas of interest will match. E.g. whether you register some event notifications, do text logging, persist decision journal, or do something else — you'll likely mention that. And you may forget/miss one or two things - that happens. But you don't miss them all if you're a good candidate for that position.

When you say make it up on the spot, I'm hearing they can answer questions when you ask. If you ask them to describe how they'd process input and send it to a database they can just as easily tell you what they generally do. If they don't mention any of the things you're looking for, then you have your answer. If they answer some then you can decide to probe about what's missing, or thank them for their time. You haven't explained why a take home test is needed for that.

> When you say make it up on the spot, I'm hearing they can answer questions when you ask.

Or they have enough surface knowledge to risk an answer that could potentially work. It depends if that's the threshold acceptable to you.

> You haven't explained why a take home test is needed for that.

We're commenting on the blog post about this very question. Making the initial stage of this test take-home is at least partially solving points 2,5,6,8 from the list.

It sounds like we're now back to mind reading the answer you'd like them to provide based on overly broad questions. That is a poor way to assess whether they know about a topic. Ask them what you'd like to know. You don't have a relationship with them and they aren't familiar with your version of "normal" or "mature" processes.

It doesn't matter if they're familiar with my specific concept of a mature app. (Although just reading the job description would cover 90% of it...) There's always going to be a lot of overlap of concepts anyway. The question was broad exactly because there are many good answers.

(FTR, the answers were pretty consistent in what elements they included, so it wasn't confusing for the candidates)

That's like a trick question given that interview questions typically want the opposite.

The instructions were longer and clear - nobody tried to second-guess them. So not an issue in practice.

That’s not always accurate that the company doing the hiring contributes nothing. (Also good companies could offer to pay for that take-home test time.)

Having done take home before from the hiring side, it was incredibly time-consuming for us. We had someone anonymize the three finalist submissions, and then we had three people each individually review and comment on each one, and then we got together to discuss and choose the final candidate. Once we agreed on one, only then was it de-anonymized.

All-in, it took way more time than a single developer doing three live coding interviews. But my guess would be that most companies wouldn’t be willing to be that deliberate with take home.

There’s no guarantee the employer isn’t in final phases with someone else that will lead to them never even evaluating your work.

One of the better laid out interviews I had consisted of them pitching a problem prompt via email, followed by an hour and a half to complete the problem, followed by a ~30 minute code review. Realistically, the task was like a 20 minute job, so plenty of margin for stress-related slowdown. I think that's a happy medium between unbounded take home exams that monopolize your time, versus in-call leetcode whiteboard sessions.

The author did write "better yet, let applicants do a take-home, and pay them for their time. ... think of it as another chance to be selling them on joining your team over some other team."

I added that after-the-fact in response to this comment. So that's on me for not including it originally. It's a great idea and one I wholeheartedly endorse because it forces the company to put some skin in the game and helps limit the number of applicants that they would request perform the take home test while also recognizing that applicants' time is in limited supply.

I think we're going to see a pretty large industry shift away from take-home tests given how easily 90% of these tests can be gamed using chat GPT.

I'd think we'd have already seen this with GitHub Copilot. There was an interview I was part of late last year where the candidate had Copilot turned on during the live coding, and he didn't turn it off even after it was obvious that's what he was using. What I was more surprised by is how I thought this was a bigger deal than everyone else. Like, why come up with these elaborate tests when the candidate is just going to use autocomplete the whole time?

Maybe programming in another few years will just be glorified autocomplete and little more.

And perhaps testing people on how to write code was a mistake to begin with. It's one thing to write code, but reading code is another.

If they are effective at using it to solve problems, why not?

The real challenge is to your testing mechanism and not the candidate

> Maybe programming in another few years will just be glorified autocomplete and little more.

It won't. For the same reason code generation from UML diagrams hasn't replaced programmers [1]. Or why business owner/analysts don't write acceptance tests in Gherkin (Given-When-Then).

Though some tasks that programmers had to do manually in the past may be automated (not the first time).

[1]: https://news.ycombinator.com/item?id=26934795

Take home tests are often paid and those are the only ones I will do anymore after being burned a few times.

That solves the skin in the game problem.

If they're not paid and you still wanna do it, make it into an interesting problem you wanna work on and have it be presentable enough to include in your portfolio.

At least, that way you might get something out of it instead of nothing, although I definitely do agree that being paid for it is the only way to solve the skin in the game problem.

Reply on news.ycombinator.com