> Sure it is, more specifically, it is part of a definition. You want it to be explicitly this or that, but things aren't that simple. Sure, we can put a dictionary definition on it and call it a day, but that isn't very helpful. When I talk about innovation in this context, I roughly mean the ability to think of solutions where no previous examples are directly available.
A letter can part of some alphabet, but it is not itself an alphabet. A word can be part of some dictionary, but it is not itself a dictionary. An anecdote can be part of a dataset, but it is not itself a dataset.
I think the definition you now give seems to be reasonable, but I'd like to compare that with the example to illustrate the problem of vagueness here:
> Give an LLM the documentation of a library/framework it has no training data on and ask it to implement a specific thing based on that documentation.
vs.
> When I talk about innovation in this context, I roughly mean the ability to think of solutions where no previous examples are directly available.
The example you give is the kind of thing that LLMs are really good at: translating one thing into another thing. The original use case of the transformer model was natural language translation, and they can do that well even when there's no explicit map from the input language to the output language, as they learn an intermediate representation for all.
The same applies when the languages are jargon and synthetic: "business plan", "code" and "documentation".
Is that innovation? It meets the example you gave, but does it meet the definition you gave? It depends exactly what you mean by "no *previous* examples": myself, I would say that "documentation" counts as "a previous example".
But the more I think about it, the less that actually feels like innovation. It's in-context learning, which is neat, but it doesn't seem to meet what I'd expect from real innovation — at most it is the corporate (and government) self-congratulation that gets called by the same word.
> Extending the definition of innovation, I am highly doubtful the current generation of LLMs can implement frameworks/libraries based on new principles unless very specifically prompted. In which case you have a human architect behind it all, and I still believe you will end up with code that follows paradigms and structures that are at most "current".
This is a confusing point: that sounds like it's about following a new principle, whereas I would expect "innovation" to involve designing and specifying a new principle, determining its weaknesses through experiment. I believe LLMs are used as part of larger systems to do this, though I have no frame of reference for the quality, and I would accept that "LLM as component of bigger system" is different from "LLM" — a letter can part of some alphabet, but it is not itself an alphabet, etc.