Ask HN: In the day of AI what's best for coding tests/takeaway projects?
In the past I’ve set developers a small take home project/task.
However in the age of AI this can be easily completed with ChatGPT… what’s everyone’s go to for interview tasks?
However in the age of AI this can be easily completed with ChatGPT… what’s everyone’s go to for interview tasks?
You could also show them another solution (maybe one with a bug?) and ask them to explain it and compare with their solution.
If they are leaning heavily on an LLM it should show up during the follow-on discussion.
IMO guided reviews seem like a great option in general. You have complete control over what goes into the code to review and it wouldn't even have to be code in the first place. Could also be documentation, processes or architectures.
And that skill remains valueable even when LLMs become the most effective way to write code at some point.
But to be honest, I don't think LLMs will ever replace "coders". Language theory has always been an important part of CS degrees, and that's because in programming you need an efficient and precise way to express what exactly you want the computer to do.
In that way I don't see why whatever we end up feeding into LLMs shouldn't be considered "code". I guess that in the end we'll get smaller LLMs put into compilers to enable better optimizations.
There is nothing wrong with testing people on simple tasks as part of an interview. This isn’t a long task, likely 30 minutes work!
It’s to ensure what they say they can do is correct. Using AI tools is fine but we need to see an understanding too.
I do have an idea I suppose. Present them with a problem and a solution and have them lead a discussion of the merits of the solution, what alternatives they might propose and so on. There’s some issues to avoid with this approach tho, like presenting a pair with a correct response in mind and then judging people against that standard rather than the standard of “how are they thinking through this problem, the drawbacks and benefits, are they presenting their ideas effectively and respectfully, etc”
It also primes them to expect a certain sort of environment coming into the job, participating in meetings and collaborative solutions instead of just being a code cowboy. Adapt your specific discussion to how you expect them to conduct themselves, probably present your company collaboration guidelines and meeting CoC as a precursor to the meeting, idk I’m just spitballing here. I’ve struggled with this problem a lot hiring people.
No, it means they know how to use tools to solve certain small programming tasks.
You can tell in an interview if they are focused on the interview or the job. If there is never any talk about the job, if it's all about some insanely drawn out interview process you know that's all they care about.
I've seen startups (even some here on YC) who have been hiring the same roles for literally years, they don't know anything else.