GPT-3 can nearly pass a phone screen

twitter.com

39 pointsGormisdomai4 comments

I'd like to believe it but since I have previously seen GPT 1 and 2 produce a lot of nonsense, I am skeptical.

I mean my first guess is that one, there are some edits in the tweets, and two, GPT-3 actually encodes the full text of an extremely similar interview.

There are no edits here. After the first couple lines which are prompt, I typed the “human” lines and GPT-3 generated the “AI” ones.

But I also wouldn’t say that it’s very close to passing the phone screen. It isn’t actually getting a correct solution to the problem, despite me giving it several opportunities to correct itself. And this is pretty easy for a phone screen problem. More of a warmup. The answer is just one line of code, and for a full phone screen problem you might expect someone to write 10-15 lines of code.

Still, IMO it is impressive how close it comes.

I wonder how the approach of writing test cases and allowing gpt 3 to do the coding until all the test cases pass will work.

Forcing an AI to do Leetcode style questions should violate some sort of Robot Geneva Convention

Reply on news.ycombinator.com