Beliefs are not rooted in facts. Beliefs are a part of you, and people aren't all that happy to say "this LLM is better than me"
Yet, whenever I ask it to do something novel or creative, it falls very short. But humans are ingenious beasts and I'm sure or later they will design an architecture able to be creative - I just doubt it will be Transformer-based, given the results so far.
That's a new result, but I don't know about novel. The technique was the same as earlier work in this vein. And it seems like not much computational power was needed at all. (The article mentions that an undergrad left a laptop running overnight to produce one of the previous results, that's absolute peanuts when compared to most computational research).
People still call things other people do novel. There's clear social proof that humans do things that other humans consider novel. Otherwise the word would probably not exist.
Just today I wrote a python program that did not resemble anything I'd written before, nor had I seen anything similar. I had to reason it out myself. That passes thr test that the original comment set.
Anyway, I've coded all kinds of wacky shit with claude that I guarantee nobody has implemented before, if only because they're stupid and tedious ideas. They can't all be winners, but they were novel, and yet claude code implemented them as confidently as if they were yet another note taking app. They have no problem handling novel ideas, and although the novel ideas in this case were my own, its easy to see how finding new ideas could be automated by exploring the combinatorial space of existing ideas.
For example, what I refered to that I've written is a dynamic storage solution for n-dimensional grids, that can grow arbitrarily in any direction, and is locally dense (organized into spatially indexed blocks of contiguous data).
I had never considered this problem before, and I certainly had never seen a solution before (even though there may well be one).
I worked it out on paper, considering how integer lattices can be partitioned and indexed, and then I transformed that into a design which I then implemented. Working purely from the design, not considering existing solutions.
Surely if we tell the LLM to do enough stuff, something will look novel, but how much confirmation bias is at play? Tens of millions of people are using AI and the biggest complaint is hallucinations. From the LLMs perspective, is there any difference between a novel solution and a hallucination, other than dumb luck of the hallucination being right?
We also can’t ignore than it was humans who thought up this problem to give to the AI. Thinking has two parts, asking and answering questions. The AI needed the human to formulate and ask the question to start. AI isn’t just dropping random discoveries on us that we haven’t even thought of, at least not that I’ve seen.