Hacker News Reader

Comment by jrjeksjd8d | Hacker News Reader

Like so many things in American life these days, we've arrived at a "better solution" that extracts value from the past without producing a future. The decades of Stack Overflow answers fed into LLM training produce plausible answers for today. In 10 years what will the LLMs train on?

So much of life these days is purely extractive, trying to squeeze more money out of less productive activity. It's no wonder young people feel disillusioned and are increasingly focused on gambling and "investing" in meme stocks.

View on HN

I think this is seeing the past with rose tinted glasses, it’s not like SO was on the cutting edge of computer science. The world is probably better off that we don’t need another 12 ways to develop a CRUD app or learn the framework of the month from gatekeepers with a bad attitude.

In fact, I think this is part of what lead to the downfall of SO.

The moderation could be very aggressive with "duplicate" posts getting closed fast. The problem is sometimes the "solution" in the duplicate was either irrelevant or dated. Things like telling someone to us jQuery in 2020.

People keep saying this specifically about jQuery, but without any object examples.

If you lookup how to do any sort of standard front end operation there's a good chance the top SO answer will reference jQuery or some other outdated approach. I don't typically save these cases for future reference but have seen them many times and expect most people who have spent much time searching these topics have as well.

You must have been using SO different to me then. For me it was more like a Wikipedia for specific language errors, compiler/IDE issues etc.

Never once saw anyone discussing how to implement CRUD or claiming one framework was better than another. That was the point - concrete answers not opinions.

> The world is probably better off that we don’t need another 12 ways to develop a CRUD app or learn the framework of the month from gatekeepers with a bad attitude

Are you saying that because you don't like web apps or the frameworks that people use to make them, there shouldn't be a way for people to publicly ask questions about programming?

Most people's problems aren't the cutting edge of computer science. They're "what does this log mean" or "how do I do X in Y framework". These are answers LLMs are great at regurgitating based on a big corpus like Stack Overflow.

As the underlying software evolves the log messages will change and the APIs will change and the answers won't make sense anymore.

Why, us of course.

The results of your claude code session, for example, make fine training data.

Did the user commit the final answer? What changes were made before they did?

> The results of your claude code session, for example, make fine training data.

Does Claude code copy your repository onto its server?

I do not know whether I'm misinterpreting this comment but:

Yes, the context from working sessions moves over the wire - claude "the model" doesn't work inside the CLI on your machine - it's an API service that the cli wraps.

No -- but your follow-up interactions would, and that's something that could be cleaned and filtered upon.

Edit: I also mean to imply that maybe this could be more observable in the future. Opt-in, of course.

If you truly believe that the ai companies, who have used essentially all of the worlds ip without asking for permission won’t use yours because you gave them $20 a month, I have some magic beans to sell you.

Is anyone actually doing that level of fine-tuning? My understanding of LLMs is that they shovel in all the code they can find regardless of quality and let the Lord sort it out.

> . In 10 years what will the LLMs train on?

Hopefully Reasoning is much better that training on original code and API docs are sufficient

> In 10 years what will the LLMs train on?

Why do people think this is necessary? When you learn new things, like bicycling, you don't start with relearning how to walk.

Walking doesn't change as often as C++ standard.

The issue here is borrowing from the future to pay for the present. The bicycle analogy (unless I'm missing something huge here) does not seem relevant at all.

Why do you think source code and documentation are not enough for an LLM to train on?

I think they are positing that LLMs do not produce new thought. If a new framework (super magic new framework) is released, current LLMs will not be able to help.

Why wouldn't the LLM just read the source of the framework to answer questions directly? That's how I do things as a human. Given the appropriate background knowledge (which current LLMs are already extremely capable with), it should be pretty easy to understand what it's doing, and if it's not easy to understand the source, it's probably a bad framework.

I don't expect an LLM to have deep inbuilt knowledge of libraries. I expect it to be able to use a language server to find the right definitions and load them into context as needed. I expect it to have very deep inbuilt knowledge of computer science and architecture to make sense of everything it sees.

Because LLMs do not work like that - there's no "understanding" the source and answering questions, it simply "finds" similar results in its training data (matching it with the context) and regurgitates (some part of) it (+ other "noise").

Meaning as technology evolves and does things in novel ways, without explainers annotating it the LLM won't have anything to draw on - reducing the quality of answers. Which brings us full circle, what will companies use as training data without answers in places like SO?

You’re talking about ChatGPT 2.0 and severely underestimate the capabilities of today’s models.

It's well known that even current LLMs do not perform well on logic games when you change the names / language used.

i.e. try asking it to swap the meanings of the words red and green and ask it to describe the colors in a painting and analyse it with color theory - notice how quickly the results degrade, often attributing "green" qualities to "red" since it's now calling it "green".

What this shows us is that training data (where the associations are made) plays a significant role in the level of answer an LLM can give, no matter how good your context is (at overriding the associations / training data). This demonstrates that training data is more important (for "novel" work) than context is.

Write "This sentence is green." in red sharpie and "This sentence is red" in green sharpie on a piece of paper. Show it to someone briefly and then hide it. Ask them what color the first sentence said it was and what color the second sentence was written in.

Another one: ask a person to say 'silk' 5 times, then ask them what cows drink.

Exploiting such quirks only tells you that you can trick people, not what their capabilities are.

The point isn't that you can trick an LLM, but that their capabilities are more strongly tied to training data than context. That's to say, when context and training disagree, training "wins". ("wins" isn't the correct wording, but hopefully you understand the point)

This poses a problem for new frameworks/languages/whatever that do things in a wholly different way since we'll be forced to rely on context that will contradict the training data that's available.

What is an example of a framework that does things in a wholly different way? Everything I'm familiar with is a variation on well explored ideas from the 60s-70s.

If you had someone familiar with every computer science concept, every textbook, every paper, etc. up to say 2010 (or even 2000 or earlier), along with deep experience using dozens of programming languages, and you sat them down to look at a codebase, what could you put in front of them that they couldn't describe to you with words they already know?

Even the differences between React and Svelte are big enough for this to be noticeable. And Svelte is actually present in the training data. Given the large amount of react training data, svelte performs significantly worse (yes, even when given the full official svelte llms.txt in the context)

But it doesn't pose a problem. You are extrapolating things that are not even correlated.

You started with 'they can't understand anything new' and then followed it up with 'because I can trick it with logic problems' which doesn't prove that.

Have you even tried doing what you say won't work?

If I make up a riddle and ask an LLM to solve it, it will perform worse than a riddle that is well known and whose solution will be found in the dataset. That's just a foundational component of how they work.

Yes you can trick it.

But it’s almost trivial for an LLM to generate every question and answer combo you could every come up with based on new documentation and new source code for a new framework. It doesn’t need StackOverflow anymore. It’s already miles ahead.

I just downloaded "Degeneration in discriminantal arrangements", by Saito, Takuya from the journal "Advances in applied mathematics" dated November 2025 and fed it to Claude.

It not only explained the math but created a react app to demonstrate it. I'm not that can be explained by regurgitating part of it with noise.

I encourage you to try it with something of your own.

Abstract:

Discriminantal arrangements are hyperplane arrangements that are generalization of braid arrangements. They are con- structed from given hyperplane arrangements, but their com- binatorics are not invariant under combinatorial equivalence. However, it is known that the combinatorics of the discrimi- nantal arrangements are constant on a Zariski open set of the space of hyperplane arrangements. In the present paper, we introduce (T, r)-singularity varieties in the space of hyper- plane arrangements to classify discriminantal arrangements and show that the Zariski open set is the complement of (T, r)-singularity varieties. We study their basic properties and operations and provide examples, including infinite fami- lies of (T, r)-singularity varieties. In particular, the operation that we call degeneration is a powerful tool for constructing (T, r)-singularity varieties. As an application, we provide a list of (T, r)-singularity varieties for spaces of small line ar- rangements.

* https://doi.org/10.1016/j.aam.2025.103001

* https://imgur.com/jjfFNMI.png

My recent experience with codex is that they absolutely do work that way today (this may be recent as in within the last couple months), and will autonomously decide to grep for things in your codebase to get context on changes you've asked for or questions you've asked. I've been pretty open to my manager about calling my upper management delusional with this stuff until very recently (in the sense that 6 months ago everything I tried was still a toy), but it's actually now reaching a tipping point that's drastically changing how I work.

LLM’s can already do this without training. I recently uploaded a manual for an internal system and it can perfectly answer questions from the context window.

So adding a new framework already doesn’t need human input. It’s artificial intelligence now, not a glorified search engine or autocomplete engine.

Which particular LLM was that?

Who wrote the manual?

Not sure I understand...

How will CharGPT/CoPilot/whatever learn about the next great front-end framework? The LLMs know about existing frameworks by learning on existing content (from StackOverflow and elsewhere). If StackOverflow (and elsewhere) go away, there's nothing to provide a training material.

Reply on news.ycombinator.com