I am confused, every day I read on HN that AI's can just interpolate the data they have seen in training, and that they are structurally incapable of coming up with something new, creative and not in the training distribution.
The AI in this case didn't create a novel technology- it merely used the existing technology without basing the new design on a previous one. The whole "human couldn't come up with it" is because the possible design space is so large, there's no reason a human would start where the AI did.
The thing the AI did better than humans was brute forcing a solution faster. Still a very handy thing to have, but it isn't "creating" in the sense that it invented new materials or fabrication processes or anything novel.
No. That can be said about LLMs, but not about all forms of AI. The technique used is not a LLM.
Sadly we've bastardized the term AI that, if it ever meant anything, it's meaningless now. The currently most voted thread in this post discuses the topic.
> In our new approach, the architecture begins essentially from nothing and is progressively assembled through successive iterations. The system explores the design space by generating myriad candidate circuit combinations and mapping the resulting performance trade-offs as it navigates this landscape. Because the process is not biased by prior human design choices, it can produce completely novel circuit topologies that look markedly different from those created by human designers.
The key tho is can they solve problems not easily solved before with prior techniques. Further can they identify problems not readily presented. Then identify novel solutions. Etc. The answer is emphatically yes they can. These features don’t have to literally exist in their training data, but the supporting highly convoluted network of associations of all their training data does have to in some complex space allow for it to produce these answers. It’s not the same as they’re stochastic parrots at all.
Are they creative? No, because they don’t have awareness. My personal imprecise definition of creative requires both self and awareness as well as free will. There is no driving awareness in all AI architectures, it all derives from extrinsic impetus. Creativity is derived, IMO, from a layer of our minds that is not readily assessed or measured and is only indirectly expressed through language, art, and music. Hence it is not directly trainable and therefore a learning model can’t learn it by reinforcement. It can learn the proxies, but the proxies are not, as we all deeply know, the same as our experienced awareness. We are not our words, our art, our music. We try hard to bridge it, but it’s impossible and you and I know this to be true from experience. In fact we can not even examine our own awareness because it’s not directly observable or possible for us to directly reason about. This is core to a lot of philosophy, especially mid and far eastern philosophy of the mind, the self, the five aggregates of Buddhism, etc. Psychology points at it, and modern psychology avoids it because it’s practically difficult for outcome oriented treatments.
While I have no hope for a rigorous definition (I don't think it's possible), there are two very distinct kinds of creativity:
1. Result is sufficiently novel for the system itself, i.e. it never seen it previously. This kind is too trivial to even talk about.
2. Result is novel for the side observer. This kind of creativity is meaningless because it depends on at least one unknown (side observer).