Hacker News Reader

Comment by merelydev | Hacker News Reader

LLMs dont create anything new, if programmers stop reading the code technology will be forever frozen to 2022, no new programming languages, operating systems, concurrency primitives, databases, networking protocols, UI frameworks everything will be based on the training data and future generations will forget about all the primitives we now take for granted.

If someone creates a new programming language/ framework or new better way to do async or whatever, no one will use it because it is not in the training data and it wont take off because everyone is using LLMs. It will be like using the same Lego pieces over and over.

View on HN

What if programming languages, operating systems, concurrency primitives, databases, networking protocols, UI frameworks are already good enough, and the innovation lies elsewhere?

You can do a lot of cool stuff with the same lego pieces.

That is like saying what if music is already good enough.

Totally OT. Any advance in music in the last 100 years?

Electric guitar was invented in 1931

Metal vocals are still advancing today. Check out Will Ramos doing harsh overtone screaming.

What does an advance in music even look like? Shifting tastes for pop music? Or new techniques? New music theory? Or just experimentation?

Considering all music is subjectively influenced by the culture in which it's born (see the difference between Asian traditions of music, European traditions of music, African traditions, and traditions of the Americas) not even all of those have a given structure that is present today like the typical 4/4 and have polyrhythmic and multitonal structures by design. The fact that everything on the radio has converged towards 4/4 165bpm major chord progressions is evidence of that cultural phenomenon.

But what if the fundaments of all these, in the human produced literature, actually contain hidden circularities and holes which make very hard the progress?

IMO for the moment the greatest value from these AI tools is that we can start an audit and hopefully proceed on a saner foundation, after we use the tools and think about it.

This is different than too many AI generated proofs or panic reactions from the academic system with its stupid incentives.

what if our horse carriages were already good enough

This is obviously false, and the same silly arguments were made back in the day with Deep Blue and AlphaZero.

False dichotomy. Chess/Go can still be played between two humans and there is allot of value in that because humans compare each other to other humans, when you see a skillful Grandmaster play you know they are good compared to yourself or the average human, that is why people still watch, play chess/go and train hard to get good. Programming is different because you are creating something not necessarily trying to win a game.

Most programming tasks are exactly like that. Is this agent able to complete this task? Is this agent able to optimize a kernel beyond previous attempts?

Of course some are subjective and that's where progress is harder, like "Is this website pretty?". But for tasks that can be objectively measured, LLMs will go beyond human level, just like with Chess and Go.

That's why RL is so important when training LLMs.

My point is that LLMs depend on training data so the code they produce will be stuck in 2022, no new languages, techniques beyond that because new techniques are not in the training data (at least not enough of it for training because most coders are now using LLMs).

Chess/Go continues to progress because it is primarily a human vs human activity, people will always be learning to play chess and chess will continue to develop.

Pre training data is in large part synthetic these days, and RL data is almost all synthetic.

Computer Chess progress has nothing to do with human vs human activity. AlphaGo Zero used no human game data at all.

> Pre training data is in large part synthetic these days

How much of that data can lead to innovation? Can you predict all innovation map it out on paper.

> Computer Chess progress has nothing to do with human vs human activity.

The point is that humans will always be learning chess because it primarily a human vs human activity they will be contributing games to the chess database, unlike with programmers who are stopping to code and only prompting, generating code stuck in 2022.

> AlphaGo Zero used no human game data at all.

Sure, but that instance of AlphaGo is still dependent on its training, its intelligence, so it is a question of is that the best and only way to win a game of Go. Just a few weeks ago, a Go Grandmaster found a way to beat one of the strongest Go AIs.

So a specific instance of an LLM might be the smartest based on what we know and need today but that is not the limit of how far we can go, this is why it is important for humans to always have an intimate connection with the code, math, science, chess etc for progress to continue.

> Sure, but that instance of AlphaGo is still dependent on its training, its intelligence, so it is a question of is that the best and only way to win a game of Go.

If this were true then it would be impossible for these models to ever exceed the top human level as there would exist no training data that allows them to exceed the top human level.

However, despite there being no training data on ability to beat the top humans these models have achieved it.

> this is why it is important for humans to always have an intimate connection with the code, math, science, chess etc for progress to continue.

This is just you wanting to remain relevant rather than actually based on evidence.

> If this were true then it would be impossible for these models to ever exceed the top human level as there would exist no training data that allows them to exceed the top human level.

Of course AI exceeds humans at chess, I never denied that. I am saying because chess is primarily a human vs human game, humans will always be learning and playing chess, their games will add to the chess knowledge base, AI also adds to this knowledge base. But programming is not primarily a human vs human activity so there is a risk programmers will forget how to code and all software will be stuck in 2022 because of the training data, this stifles innovation.

> I am saying because chess is primarily a human vs human game, humans will always be learning and playing chess, their games will add to the chess knowledge base

I guess I'm contesting that idea you are putting forward that the data from the games these humans are playing, which are at a vastly lower level that the top AIs are meaningfully important for helping the AIs to improve at the top level.

Would more people learning their times tables be helpful for top mathematicians in their fields to get better at the frontier of maths? Probably not right. Same applies here.

Would AIs advance at the same rate for the top level of chess in a world where humans completely stopped playing chess vs the world we have today. I would say they would as the human level data is of limited value to the frontier which is dominated by AI and AI game data, you are claiming that it does.

> But programming is not primarily a human vs human activity so there is a risk programmers will forget how to code and all software will be stuck in 2022 because of the training data, this stifles innovation.

Does it? Or will AI be able to run its own experiments and find better/more efficient abstractions that propagate because they are better and this will find its way into training data for future AI.

> I guess I'm contesting that idea you are putting forward that the data from the games these humans are playing, which are at a vastly lower level that the top AIs are meaningfully important for helping the AIs to improve at the top level.

I never said human games are meaningfully important for training AI. Human games are still important for the advancement of chess, maybe Magnus Carlson can learn from games between two Super AIs but most humans still learn from games by humans, Grandmasters are continuously developing the opening, middle-game and end-game systems, adding to the chess knowledge base. Every serious chess player still reviews and study games by prominent Grandmasters, every serious chess player documents their own games, writing down every move. All rated games are recorded and added to the chess database that every player can review and study.

>Does it? Or will AI be able to run its own experiments and find better/more efficient abstractions...

Only if it is in the training data.

> Human games are still important for the advancement of chess

Are they? Why?

For a human vs human game sure but at the very top level? No of course not because it's all done by AI.

> Only if it is in the training data.

This is trivially not true, as how has AI managed to become better than humans if the knowledge of how to do so never existed in the training data.

We are well past AI can't do X unless X is in the training data. If your claim were true then AI could never surpass human expertise in any field because by definition all the available training data will at best be at the current human frontier and not beyond.

> For a human vs human game sure but at the very top level? No of course not because it's all done by AI.

Glad that you finally agree,this is what I was saying the whole time.

> This is trivially not true, as how has AI managed to become better than humans if the knowledge of how to do so never existed in the training data.

AI can do more work, faster, AI it only needs sufficient compute and data. But that does not mean it is more intelligent than humans, it still uses the same code, algorithms, frameworks, protocols etc etc that are in the training data, sourced from human open source code on the web.

Human games databases are completely irrelevant to the strongest chess engines. We are ants in comparison. The Go thing you mention is playing against handicap. Sorry I won't go into more detail explaining why your premises are wrong, I'm tired of this discussion.

Human games databases might not contribute to AI training but they do contribute to the chess database itself like it has always been, AI doesn't change that fact.

> Chess/Go continues to progress because it is primarily a human vs human activity, people will always be learning to play chess and chess will continue to develop.

AIs are not continuing to get better at chess/go because humans continue to play at levels far below themselves who discover new techniques. They get better because they play against other AIs and discover new techniques that have a higher win rate that way.

I would bet that even if humans stopped playing chess/go and people were still willing to run these AI models against each other they would continue to get better.

I am not talking about the advancement of AI, I am talking about the advancement of chess.

Two things can be true AI drastically contribute to the advancement of chess and humans playing against each other also contribute (even if slowly) to the advancement of chess as it has always been since the invention of the game. The point is that because chess is primarily a human vs human game humans will always have the knowledge of chess, unlike with programmers who are giving it up to prompting, and programming being much more complex than chess (checkmate and win) will be stuck in 2022 because of the training data.

Everything is a game

LLMs receive new data via input context, not just training data.

Thought experiment: How effective will 2026 LLMs be for humans in 2526?

It's not game over just because 500 years are missing from the training data. The important question is how well can 2526 humans make culture and knowledge navigable to LLMs via tool calls.

Today's LLMs might need for example sub agents to translate to 2526 English, sub agents to read 2526 docs.

It's _really not clear_ whether 2026 LLMs will be useless. To believe that reflects an enormous misunderstanding.

> LLMs receive new data via input context, not just training data.

Be more specific about the "new data". If everyone is using LLMs for work (generating code), especially the juniors who won't get the chance to learn from first principles, LLMs will be training on the data they generated. How will new code enter the system at large enough quantity that it can be used for training?

> It's _really not clear_ whether 2026 LLMs will be useless. To believe that reflects an enormous misunderstanding.

They won't be useless, they will just be frozen knowing only whats in their training data. No new programming languages will emerge, in 2526 they'll still be using Rust and javascript, same exact code from 2022 which dominates the training data.

The "new data" is: person A prompts an LLM to create or modify a tool, person A distributes code person B, person B's LLM uses the tool via docs/help/error. That is a direct path for an LLM to "know more" from a human than what's in its training data.

If we get a new programming language not in the training dataset, we could give an LLM a decent compiler with compile errors, and some sample code and it would be able to write code in the new language without training.

Reply on news.ycombinator.com