It feels like there’s an aspect of “I want to believe” in this, except in the other direction — if LLMs can come up with new information, it threatens one’s identity of what it means to be human.
(As for evidence: https://twitter.com/d_feldman/status/1636955260680847361?s=6...)
I see this argument made all the time. While it might be correct at a Micro-Level ("all it does is predict the next word!!!" "it just finds patterns!!!") I dont think that semantics matter here.
It has demonstrated to create new things based on existing ones.
ChatGPT will happily create a fan fiction about the rust mascot Ferris, in the style of the Bible.
I am 99% certain that wasn't in the training set
You can take some information, combine it with other information, apply various kinds of reasoning to it, and get something new as a result.
E.g. (1) Bob is a monkey. (2) All monkeys like banannas. Would you call it "new information" if I tell you that Bob likes banannas? LLMs can do this to varying degrees of success, so probably not.
Maybe you mean something like in Information Theory, where information is something that resolves uncertainty?
It'd be interesting to know, what is it you think humans are capable of that LLMs can't be?
I wouldn't be surprised to learn that this is how humans work, too.
If this statement is true, it is true for humans as well. LLMs can and have authored original works of art that are no more traceable to specific training influences than any work of art created by a human.