Hacker News Reader

Comment by angarg12 | Hacker News Reader

I'm an active interviewer for a FAANG company. I've run some of my interview questions through ChatGPT with mixed results. I'd say that on it's current form it could easily help a Junior candidate to pass the interview, not so much for a mid or senior (yet).

In an internal slack channel I proposed to generate (incorrect) solutions for coding problems with ChatGPT and have candidate do code reviews as an alternative interview format. I was almost chastised for proposing this. Someone proposed that, if I wanted to go down that path, I could manually generate a wrong solution so that it is curated. I think he is missing the point.

View on HN

Curating it would give a level playing field to candidates and allow you to decide what you are actually testing for, but maybe I'm also missing the point

You can do both. Generate 10 solutions with ChatGPT and pick the ones that work best. Do this for each candidate.

That is curating

Wouldn't you want to curate? Maybe I'm missing the point too.

OP might be implying that from now on many engineers will use AI tools to write the first draft of their code so it’s smart to hire people skilled at correcting AI-generated code.

Or that the inherent variability in ChatGPT output could make it harder to game examinations based on the same set of prompts

It's also good to hire people who are skilled at correcting bad code in general. This is also a good litmus(?) test to see the interesting ways people come to solutions to other's faults.

"many engineers will use AI tools to write the first draft of their code"

I fail to see how ChatGPT would be useful for capable engineers. Is boilerplate code actually a time suck for anyone?

Yes, and also having to scan through search engine results and documentation just to find out how to do something like this:

  const ts = require("typescript");
  
  const options = {
    target: ts.ScriptTarget.ES2017,
    module: ts.ModuleKind.CommonJS,
  };
  
  const someJavaScriptCode = ts.transpileModule(someTypeScriptCode, { compilerOptions: options });
  
  console.log(someJavaScriptCode.outputText);

(Actual ChatGPT response when asked to convert TS to JS)

That saved me a good five minutes with minimal disruption to my flow. Those little things add up to a significant amount of time and energy. I have two young children, ages 1 and 4, and these tools make it possible for me to program in little 10 minutes bursts between basically endless cooking and cleaning chores.

I actually take offense to the notion that you would consider people who find these tools useful to be incapable engineers.

"I actually take offense to the notion that you would consider people who find these tools useful to be incapable engineers."

I apologize for offending you. I can't imagine parsing code for errors being faster than typing it out. I suppose I assumed most have boilerplate memorized. I'll try to avoid being so presumptuous in the future.

There's two ways to check if something is correct: run it or read it. In this case I just copy and pasted the results and it worked as expected. YMMV.

I will say that you seem to be operating under the assumption that these tools are less reliable at producing the correct code than they really are!

Ironically, the latest GPT-3 models trained before OpenAI made changes to their API resulting in basically useless assistance for things like the openai npm library!

"I will say that you seem to be operating under the assumption that these tools are less reliable at producing the correct code than they really are!"

Its output is generic. Leverage is not found in convention. For that reason, I find it useless.

Plus, I typically retain my codebase. For exploratory coding, I could see some utility.

Gotcha. You want code. I want tools.

So a tool that helps me make tools is a lever.

But a tool that helps you make code is not a lever, because “good code” is your goal.

This explains it perfectly! Thank you!

"because “good code” is your goal."

Original thought is my goal.

Otherwise, I think you're spot on. To each their own as they say!

You get the point.

He can still curate right?

The eternal loop in job interview questions.

Should you ask very similar questions, to keep candidates on a level playing field by always measuring to the same standard?

Or should you vary the questions a lot, to keep candidates on a level playing field by not advantaging those who've seen leaked questions on glassdoor?

I must have expressed myself incorrectly. Generating coding problems automatically with ChatGPT doesn't mean you just throw whatever to the candidates. You can both auto generate AND curate. The part I'm against it is manually creating problems.

There are several reasons for this. One of the biggest problems with FAANG interviewing is that questions get leaked almost instantly. I spent days creating a brand new original problem, and it got leaked by the second time I asked it. ChatGPT gives you a unique problem for each candidate.

There is also the angle of leaning in the tech. I would encourage candidates to use any means to review the code as they would in the workplace, including IDE, web search, and LLMs. If these tools are going to revolutionize the way we work, I want to see how well people can use them.

Lastly I'm also intrigued by the dynamics of this kind of interview. As an interviewer I have asked the same question 50 times and I have seen all variations of answers. But what if the problem is relatively new to me? This seems closer to a pair programming exercise with two colleagues working together to solve a problem.

Reply on news.ycombinator.com