Comment by My_Name | Hacker News Reader

Comment by My_Name | Hacker News Reader

With the improvements that AI has made in just the last year, it should be obvious to anyone that code written by an AI will at some point stop being "AI slop" and be better than the majority of coders are able to put out. Reduced to its basics, all code is just characters put into a sequence. Similar to chess or Go, both of which, it was claimed at one point or another, would be impossible for a computer to beat a human, until they did (chess 1997, go 2016), so computers will eventually produce better quality code than even a team of humans is capable of.

A genuine question : If an AI can reliably write code better than most coders, do it quicker, and produce code that runs efficiently which has less, or at least no more, bugs than human written code, why on earth would a company not use an AI to write all their code for most purposes?

And if they did, why is it important for that code to be 'elegant' or even human readable if the bug checking is also done by AI? (as seems to be the direction we are moving in)

This is exactly right, but a lot of people have motivated reasoning about it. I can’t really blame them, the kind of shifts that AI is looking like it will bring are unprecedented, and usually when people claim big, world changing things will happen, they don’t, so most people are primed not to believe it.

However as you say, we already have the evidence about this one, and it would require some unknown wall to exist where AI could suddenly not improve further, and I’m just not seeing it. Most likely it will get more capable and cheaper as time goes on, and then every industry will be impacted the way software currently is.

I think for some people, it's partly that they think that, unlike physical work, work that involves thinking is somehow special, and humans are the only beings capable of it.

In the 40s, the argument was that computers will never think because they are made of matter, Turing rebutted this by saying that brains are made of matter and introduced The Turing Test, which was then replied to by Searle with his famous 'Chinese Room', which to my mind just made all consciousness suspect, even humans.

At every stage, though, computers have outpaced the predictions for their capabilities, and as you say, the unknown wall that blocks this march is yet to be seen.

Yes, also the whole motivated reasoning part, where people are quite reasonably terrified of losing their jobs and having no income and so deny the possibility.

I think we tend to downplay the potential positives though. We may come to a point where we look back at the concept of work and see it as some kind of indentured servitude; the idea that you have to spent the majority of your waking hours serving someone else just to be able to have food and shelter is naturally repugnant.

Sure, many of us are fortunate enough to have jobs we enjoy, but everyone I’m sure has memories of jobs they hated and perhaps couldn’t leave because they needed the money. I hope we end up in a world where this is seen as a relic of a barbaric past.

Such a utopian future is unlikely to materialize however. The salary that one gets paid is backed by the generated (and perceived) value of the work they did, and if people do not have anything to offer in form of labor, then it will create pressures to reduce any UBI style wages paid out, as well as provoke internal conflict as people try to come up with justifications for why they should have a larger share of said basic wages, etc.

You seem to not know what UBI is. The U stands for Universal, meaning everyone gets the same. Universal Basic Income, a fixed amount of money everyone gets whether they work or not. Nobody gets 'a larger share'

Unlike work, where your wages are theoretically matched to your output, UBI is not.

LLMs are statistical prediction machines, they're always going to produce the most "average" code because they reproduce what they're trained on. Average code is bad, and it's going to get a lot worse as people's skills degrade and the training set degrades.

If the code isn't readable to humans, there's no particular reason to think it's going to be magically readable by LLMs either.

Until this year, you would have been right, but in May, the Erdős unit-distance conjecture was disproven by an AI. This required novel thought and reasoning, not just recombining already known facts.

The view that they are only statistical prediction machines is becoming increasingly disconnected from their current abilities.

I probably should have put the word 'easily' before readable. After all, if it is valid code, it can be read.

None of that changes that it's a statistical prediction machine. This isn't me speculating, it's just literally what it is. You can buy a book on how to implement one (I have one!) The fact that it solved a math problem doesn't change that, it just means it was trained on enough math to be able to apply a known technique. These things aren't magic.

Also, I'm really suspicious of any result that comes from OpenAI. AFAIK frontier labs hire people to solve problems for the AI to train the AI, not to mention it scraping the internet. I think it's easily possible that a human might have already provided a 90% of the solution and AI filled in the gaps. I don't know that for sure, but I do not trust these companies published results at all. I'm only interested in 3rd party independent results.

You said "they reproduce what they're trained on"

The problem was unsolved, it solved it when it was not possible for it to be trained on the solution, therefore your other claim that "they're always going to produce the most "average" code" does not hold up, they can come up with better code than they were trained on precisely because they can apply known techniques to write better code than existed in their training data. The book you have is out of date and does not apply to recent improvements in the field.

The unit distance problem had no known answer and no roadmap. The model identified the problem, chose its own approach, and produced the proof.

I am not convinced by either your logic or your argument from incredulity. Neither prove your position.

They predict the "most likely" token given the context. That's a huge caveat. Just putting "this is excellent code" in the context makes a vanilla LLM do better. Doesn't that make you pause for a moment before asserting hard limits on what they're capable of?

You might argue they're still capped at the "best" quality seen in the input. Not so. Take typos. Human text has a certain base rate of typographical errors. LLM output contains almost none. Why? Because there are many more ways to be wrong than right. LLMs are not just averaging machines, they also denoise. That should also give you pause.

I do find it rather curious that on a site where people value discussion, instead of answering my genuine question, people just downvoted the fact that it was asked in the first place.

Reply on news.ycombinator.com