Most knowledge workers' activities aren't innovative or imitational - similarly, we compose stuff, so LLMs are a fair competitor.
Honestly it read just as well ;) We’re simpler than we imagine.
Indeed, creativity (not just randomness) is a quite high bar, that I think even most humans don't pass. Most of us don't create a lot of new things and ideas in our daily lives. We mostly follow the steps of those that came before them, with small adjustments here and there.
There is a pub:grill in Tempe called monkeypants. Is that a new idea or just clever?
LLMs have a rudimentary form of innovation. It's not quite as good as humans but it's getting there.
That’s a fairly bold claim. What supporting data do you have to justify it?
One is data driven based on evidence.
The other is logic driven based on axioms and the logical implications of said axioms.
My statement is derived from the later. Therefore evidence is unnecessary. It is niave to blindly faith in all truth in the hands of data without understanding nuances between the relationship of data and logic.
If you have a pure function that takes the input, the output of that function has 3 possible outcomes.
1. The output of that function is some transformation of the input.
2. The output of the function has nothing to do with the input and is thus generated randomly generated.
3. The output of the function is a combination of both random generation and input transformation.
In this case your brain is the function. Input parameters are 100 percent of your existing knowledge. That includes genetic knowledge such as instinctual/rational processing behaviors evolved into the structure of your brain through evolution and learned knowledge such as what you gain when you read a book.
The result of the "innovation" operation performed by the brain includes novel output by definition. Thus by logic it must be must include components that are from a certain perspective randomly generated.
I guess at first glance it doesn't appear randomly generated because we vet the output and verify it and iterate over several pieces of randomly generated information. Additionally it's not completely random as we only try and test ideas within the realm of possibility. For example I'm not going to account for the possibility that my car will transform into a rock tomorrow that's just too random.
But make no mistake. Innovation must be partly randomly generated. Even the idea itself of composing two existing components of knowledge together is itself randomly generated.
That being said if we want to be pedantic, there's no real known way to randomly generate stuff via an algorithm. So I use the term "randomly generate" very loosely. Think of it in a similar way to random number generation on your computer: Random from a practical perspective but not technically random.
You are assuming that a pure function has access to some source of randomness. I think that's a bit less than pure unless a source of randomness is one of the inputs. Doesn't change your point at all, but it distracted me from what you were saying.
Speaking as someone who has occasionally invented new things, I think you underestimate the variety of outputs that can arise from a lifetime of accumulated experiential cruft and a situational fitting function (e.g. "we need a working teleporter to stay competitive"), without bringing in randomness at all.
But then out of that set of ideas how do you choose which ideas to compose to formulate the new idea?
So you have idea A and idea B. And you randomly formulate a new compositional rule as an idea: A + B. Because A + B didn't exist as an idea before, it was randomly generated by you.
>I think you underestimate the variety of outputs that can arise from a lifetime of accumulated experiential cruft and a situational fitting function
Well it's actually possible to mathematically calculate the total amount of compositions If I know the total amount of experimental cruft you have accumulated in your lifetime. It's a combinatorics problem and the output of that is, you're right, extremely huge. We have fitting functions that reduce it, of course, but within this fitting function we're just iterating through all the remaining possibilities or aka "randomly generating" the new idea.
Creativity will be explainable in terms of composition and randomness in the same way that chemistry is explainable in terms of physics. Yes, these are the building blocks, but the building blocks by itself don't help you much because the complicated phenomena still hasn't been explained. All of the magic is in how the building blocks come together to produce the output.
But you will note that the point of my response had one objective to prove this statement:
"All innovation is composition plus random generation."
Human innovation can arise by using these experiences as source data for composition of text or images. LLMs, by contrast, are limited to training on text and images/video exclusively.
Right but my concept still applies. What's unique about the LLM is that it is fed a massive amount of textual data in such a way that it can essentially output text AS if it were a human that does experience those emotions. From the perspective of you and me, this is no different then what we experience in real life.
How can you be sure the people around you feel emotions just as you do? Do you just assume it? Why shouldn't you assume it for an LLM? The human like the LLM uses English to describe and communicate to you. This in terms of raw logical evidence we can't confirm if LLMs feel emotions any more than our ability to confirm whether other humans can feel emotions. The technical evidence for both is relatively identical.
Again I like to emphasize that it's not "truly random" we can't even define randomness formally with a function so this doesn't exist. It's more of a combinatorics algorithm where we iterate through every possible and likely permutation.