Hacker News Reader

They are vulnerable to reproducing poor quality training data

They are good locally, but can have trouble keeping the focus all the way through a problem

They can hallucinate incorrect statements

does not generate documentation or tests that would build trust in the code

These observations are about human programmers, right?

You may be saying it tongue in cheek, but I think you have a point. After interviewing hundreds of applicants for developer positions, I'm sure the AlphaCode performance exhibited here is similar to at least the 50th percentile.

If you're working in a large organisation where 50 % of developers perform roughly on par with AlphaCode, you'll see it as a potentially useful companion. If you're Peter Norvig, or working in a small organisation trying to hire the top 20 %, you'll just laugh at AlphaCode.

Do you often hallucinate on the job?

Tends to happen when writing self-assessments.

ha

Reply on news.ycombinator.com