Would it?
Why would "creating a novel" by a human not itself be text generation based on prediction on what are the next good choices (of themes, words, etc) based on a training data set of lived experience stream and reading other literature?
Their next action - word put on page, and so on.
>Why would it need to be a prediction task at all?
What else would it be?
Note that prediction in LLM terminology doesn't mean "what is going to happen in the future" like Nostradamus. It means "what is a good next word given the input I was given and the words I've answered so far".
>How about a dada-ist poem? Made-up words and syntax?
How about it? People have their training (sensory input, stuff they're read, school, discussions) and sit to predict (come up with, based on what they know) a made-up word and then another.
What does "has an ever changing definition" mean?
And why "everything would fulfill that definition"?
At any time whats the "good next word" is based on the state created by our inputs thus far (including chemical/physiological state, like decaying memories, and so on). And not only not "everything fullfil it", but it can be only a single specific word.
(Same as if we include the random seed among an LLM output: we get the same results given the same training and same prompt).
You could say the process chosen is somehow predetermined (even if the choices then are all made by using randomness), but then really the word "prediction" has very little meaning as the criteria to what is a "good next word" have a nearly unlimited and ever changing range as the generating process changes.
That's also exactly what an LLM does.
It's still only a single specific word if (as I wrote above) you take the seed into account too (i.e use the same input, including same random seed value).
If you mean to answer "yes, but LLMs use a random number generator, whereas humans can actually pick a word at random" I'd answer that this is highly contested. Where would the source for such randomness be in the universe (exept if you beg the question, and attribute it to an "soul" that is outside the universe)?
An LLM is bound to what an LLM can, while humans can construct and use tools to go beyond what humans can do. Being a universal function approximator does not give access to all processes in the natural world.
Unless you're Stephen King on a cocaine bender, you don't typically write a novel in a single pass from start to finish. Most authors plan things out, at least to some degree, and go back to edit and rewrite parts of their work before calling it finished.
The real issue is running out of the input window.
isn't this what abstractions are for? you summarise the key concepts into a new input window?
Yes, an insect (a praying mantis, perhaps) catching another is exhibiting some degree of prediction, and per my definition I'd say is exhibiting some (smallish) degree of intelligence in doing so, regardless of this presumably being a hard-coded behavior. Prediction becomes more and more useful the better you are at it, from avoiding predators, to predicting where the food is, etc, so this would appear to be the selection pressure that has evolved our cortex to be a very powerful prediction machine.
In what way, except as in begging the question?
Which human will?
We get prompts all the time, it's called sensory input.
Instead of "write a noval" it's more like information about literature, life experience, that partner who broke our heart and triggered our writing this personal novel, and so on.
You have to believe that humans have no free will in a certain way to have them be like an LLM, i.e, every action is externally driven and determined.
Free will doesn't have much meaning. If I dont base my action at time t, on their development based on inputs on times before t, what would I base it on?
It would be random?
Or would there be a small thinking presense inside me that gets information about my current situation and decides "impartially", able to decide in whatever direction, because it wasn't itself entirely determined by my experiences thus far?
Were would that randomness come from? Which would be the source of that in the universe, for it to occur in the mind?
If you mean pseudo-randomness, sure, LLMs employ that too.
>Ignoring information is an option.
Randomly ignoring information? If so, see above. If you mean intended informed ignoring of information, that's still determined on all the previous inputs.
A universal function approximator isn't enough to access all of nature.
I'm sure you've deducted hypothesis' based solely on the assertion that "contradiction and being are incompatible". Note, there wasn't prediction involved on that process.
I consider prediction as a subset of reason, but not the contrary. Therefore, I beg to differ on the whole assumption that "intelligence is prediction". It's more than that, prediction is but a subset of that.
This is perhaps the biggest reason for the high computational costs of LLM's, because they aren't taking the shortcuts necessary to achieve true intelligence, whatever that is.
No, exactly not! Prediction is probabalistic and liable to be wrong, with those probabilities needing updating/refining.
Note that I'm primarily talking about prediction as the brain does it - not about LLMs, although LLMs have proved the power of prediction as a (the?) learning mechanism for language. Note though that the words predicted by LLMs are also just probabilities. These probabilities are sampled from (per a selected sampling "temperature" - degree of randomness) to pick which word to actually output.
The way the brain learns, from a starting point of knowing nothing, is to observe and predict that the same will happen next time, which it often will, once you've learnt what observations are appropriate to include or exclude from that prediction. This is all highly probabalistic, which is appropriate given that the thing being predicted (what'll happen if I throw a rock at that tiger?) is often semi-random in nature.
We can better rephrase "intelligence is ability to predict well", as "intelligence derives from ability to predict well". It does of course also depend on experience.
One reason why LLMs are so expensive to train is because they learn in an extremely brute force fashion from the highly redundant and repetitive output of others. Humans don't do that - if we're trying to learn something, or curious about it, we'll do focused experiments such as "Let's see what happens if I do this, since I don't already know", or "If I'm understanding this right, then if I do X then Y should happen".