How is it solved? LLMs cannot think new things, they can only cobble something together if it's in their training set.
The training set is about patterns, not facts or specific configurations. Yes, it’s possible to extract (some) of the training set verbatim, but that doesn’t mean it’s all you can do.
Humans rarely think of new things. We're a weak hivemind species. One or two individuals figure something out, and the rest of the troop of monkeys imitates. Brains are too fuel hungry for every brain to be innovating, "innovate and copy to the other brains" is the norm.
Modern man has grown quite dumb. He only seems to be able to "invent" by massive scaling things that are decades or centuries old..
Fire? The wheel? Archimedes screw, maaaaybe?
Electricity runs from simple batteries (600 BCE) to today’s power grids.
RF was predicted but not demonstrated by Maxwell in the 1860’s. His work built on Faraday’s (1840’s) and Coulomb’s (1780’s). Coulomb built on Franklin and Newton, among others. Or do you mean Marconi and Tesla, who merely implemented what Maxwell predicted?
The same is true for lasers and transistors but it’s tedious. There was no single “back in the day people invented things from whole cloth” moment.
In what way is electricity an invention? Electricity is a physical phenomenon. Various machines for doing work with electrical energy, storing electrical energy, converting other types of energy to electrical energy, etc. are certainly inventions... Heck, rubbing an amber rod with a fur is an invention. The static charge transferred is not.
Tell me the date when humanity went from oblivious to electricity to generating and using it.
When LLMs were first introduced, they didn't have much of a feedback loop. They wrote code, but they couldn't compile it. Not surprisingly, the code had bugs.
Now, they run with harnesses that allow them to compile the code, and react to the issues they observe. They can fix their own bugs and solve problems that they create, just like humans.
Give an agent access to the physical world, and it seems highly likely that they will be able to "invent" things based on feedback they receive while working towards goals.
Of course, there are some well-known limitations of LLMs, one of the biggest being that they're pretrained. So there may be some things where they're not as good. Just like how some humans aren't as good at certain tasks, depending on their genetics and/or how they've been trained.
I don't think so. Imagine a model trained on data from an Internet that believes in hypothesis that earth is the center of the world. Even if you feed all the physical data, I don't think it can come up with the idea that all of its training data was wrong.
This might be also a good argument for why this LLMs are not "intelligent". You can feed contradicting training data all day and it will accept it without bating an eye. But that won't work with an entity that is truly intelligent.
Ah yes, that's why it only took 50 years instead of 100,000 years for homosapients to reach the space age....
Dude, there was no glorious past, we've always sucked.
Hell, looking back at the brilliant individuals, they had many, many grey areas.
That is exactly the point. Back then that could exist. Now, we would just cancel them for one of the grey areas.
I was pointing out two things: first, your understanding of LLM capabilities is very outdated; and second, that in this respect, you're behaving much like an LLM with a training cutoff.
That further touches on the idea that the differences between you and an LLM may not be as large as you imagine. In particular, "cobbling something together if it's in their training set" is pretty much what all humans do.