> In 10 years what will the LLMs train on?
Why do people think this is necessary? When you learn new things, like bicycling, you don't start with relearning how to walk.
Why do people think this is necessary? When you learn new things, like bicycling, you don't start with relearning how to walk.
I don't expect an LLM to have deep inbuilt knowledge of libraries. I expect it to be able to use a language server to find the right definitions and load them into context as needed. I expect it to have very deep inbuilt knowledge of computer science and architecture to make sense of everything it sees.
Meaning as technology evolves and does things in novel ways, without explainers annotating it the LLM won't have anything to draw on - reducing the quality of answers. Which brings us full circle, what will companies use as training data without answers in places like SO?
It not only explained the math but created a react app to demonstrate it. I'm not that can be explained by regurgitating part of it with noise.
I encourage you to try it with something of your own.
Abstract:
Discriminantal arrangements are hyperplane arrangements that are generalization of braid arrangements. They are con- structed from given hyperplane arrangements, but their com- binatorics are not invariant under combinatorial equivalence. However, it is known that the combinatorics of the discrimi- nantal arrangements are constant on a Zariski open set of the space of hyperplane arrangements. In the present paper, we introduce (T, r)-singularity varieties in the space of hyper- plane arrangements to classify discriminantal arrangements and show that the Zariski open set is the complement of (T, r)-singularity varieties. We study their basic properties and operations and provide examples, including infinite fami- lies of (T, r)-singularity varieties. In particular, the operation that we call degeneration is a powerful tool for constructing (T, r)-singularity varieties. As an application, we provide a list of (T, r)-singularity varieties for spaces of small line ar- rangements.
i.e. try asking it to swap the meanings of the words red and green and ask it to describe the colors in a painting and analyse it with color theory - notice how quickly the results degrade, often attributing "green" qualities to "red" since it's now calling it "green".
What this shows us is that training data (where the associations are made) plays a significant role in the level of answer an LLM can give, no matter how good your context is (at overriding the associations / training data). This demonstrates that training data is more important (for "novel" work) than context is.
Another one: ask a person to say 'silk' 5 times, then ask them what cows drink.
Exploiting such quirks only tells you that you can trick people, not what their capabilities are.
This poses a problem for new frameworks/languages/whatever that do things in a wholly different way since we'll be forced to rely on context that will contradict the training data that's available.
If you had someone familiar with every computer science concept, every textbook, every paper, etc. up to say 2010 (or even 2000 or earlier), along with deep experience using dozens of programming languages, and you sat them down to look at a codebase, what could you put in front of them that they couldn't describe to you with words they already know?
You started with 'they can't understand anything new' and then followed it up with 'because I can trick it with logic problems' which doesn't prove that.
Have you even tried doing what you say won't work?
But it’s almost trivial for an LLM to generate every question and answer combo you could every come up with based on new documentation and new source code for a new framework. It doesn’t need StackOverflow anymore. It’s already miles ahead.
So adding a new framework already doesn’t need human input. It’s artificial intelligence now, not a glorified search engine or autocomplete engine.
How will CharGPT/CoPilot/whatever learn about the next great front-end framework? The LLMs know about existing frameworks by learning on existing content (from StackOverflow and elsewhere). If StackOverflow (and elsewhere) go away, there's nothing to provide a training material.