Hacker News Reader

Comment by ajafari1 | Hacker News Reader

OpenAI estimates that Codex (what powers Github’s Copilot) can already complete 37% of coding tasks. DeepMind’s non-commercial AlphaCode is estimated to be 59% better at coding competitions than Codex. But how close are we to automation? There is a huge demand for developer jobs currently. In light of OpenAI’s DALLE-2 commercialization announcement yesterday, here is a new cross-disciplinary approach to determine how close we are to automation.

After reading the article how close, or far, do you think we are from software development jobs getting automated?

View on HN

> OpenAI estimates that Codex (what powers Github’s Copilot) can already complete 37% of coding tasks.

This makes me a bit suspicious of the article.

Obviously this is just one developers experience now. I've been using Copilot for the last month, and it's an awesome bit of tech. But generally speaking, when I let it write a blob of code:

* There's a 50/50 chance it'll even compile * It's always, _always_ wrong in some way, usually in ways tricky to spot * And the tests it wrote _for it's own code_ are generally wrong too

As always, I think the bottleneck is in describing exactly what you want in a way an AI can successfully interpret it.

It’s not expensive to check for syntax correctness, so it’s matter of time till they introduce it.

My experience with DALLE-2 and Copilot makes me think we are quite far off from AI being able to perform software dev on it's own. Both tools need a lot of oversight to pick out usable pieces from the gibberish that gets spit out most of the time.

That said, copilot is a great tool for generating boilerplate code, like writing a model class for a parsed JWT token or adding tedious annotations somewhere. I've used GPT-3 to generate javadoc headers for a codebase before with decent success. It gets way more fragile when there's an actual problem to be solved though, and is basically unusable about 100% of the time on complex areas of code.

From my experience, the real result will be that AI might replace interns and some people who are only doing the tedious data modeling parts of coding, but there's a very, very long way to go before it will be able to be more than just a tool for developers to code slightly faster.

I expect that in 2030, you will need 3 ppl to write code that now you need around 10. I think more devs will switch to more product owner roles or analysts that talk to ppl and spec but more high level - constraints etc. So basically these entry level coding that’s boring should get automated. Skilled individuals will get up to 10x leverage and be real rockstars.

That’s extrapolating just from current capabilities. I think there’s chance we will figure real reasoning in AI in next 10 years given interest/investment in field.

currently AI can only base its solutions on what's been done or shown as example before, especially in coding, so if you replace all software coders with AI then you have limits to what it can do

Reply on news.ycombinator.com