Ask HN: Have you seen anything original produced by generative AI? | Hacker News Reader

Ask HN: Have you seen anything original produced by generative AI?

5 pointsjazzyjackson9 comments

I was watching this youtube video, "The AI Revolution is Rotten to the Core," and while it's mostly just confirming my biases, it makes a strong claim that I've on the fence about: that these models can only interpolate between points included in its training set, can only mix what has already been done, unable to create anything original.

Now I'm amenable to the view that "original" art is only ever a mixture of what's already been done, but are there alternative perspectives on originality? Is originality a prerequisite to AGI? Are there any proposed frameworks for measuring whether a system is interpolating points within its training set, or, what is the opposite of interpolation?

View on HN

These models are pretty much always extrapolating [0]

Whether the extrapolation is crude/low-rank or astute/high-rank is a question of memorization vs generalization. That gets into the question of whether or not the model is over-fitted or under-fitted. There are certain heuristics borrowed from high dimensional statistical physics that can be used to guess how good the test performance of a model will be on a typical task without even knowing what the test data is [1].

Originality for me means finding better answers to sub-tasks, and then combining those answers together in a better way. This is the nirvana of cross-entropy minimization - the emergence of capability results from gaining the ability to amass a wider range of skills, improving upon them, and percolating those improvements towards multiplying the leverage of other skills.

How long such a thing can keep improving with current tech, who knows, but you should really think critically about whether that sounds just like interpolation through the corpus.

[0] Learning in High Dimension Always Amounts to Extrapolation - https://arxiv.org/abs/2110.09485

[1] https://github.com/CalculatedContent/WeightWatcher

Thanks for pointing to the video; I didn't know it and just had a look at some parts; there is even a HN discussion about it: https://news.ycombinator.com/item?id=37629753. Unfortunately I couldn't find any information yet about the video author's background; he usually seems to publish videos about games, so he is likely not a specialist for most of the topics presented in the video (please post a reference if my conclusion doesn't apply), and the conclusions presented seem rather arbitrary and tendentious. But he brings up a lot of interesting points which are worth a closer look.

There is a very long debate in the sciences about what intelligence and creativity is. The questions will probably only be answered conclusively - if at all - when we know exactly how the brain and consciousness work in detail; this will still take some time.

Nevertheless, scientists who specialize in the subject already have well-founded expert opinions. In terms of creativity, there are established tests, and apparently ChatGPT performs very well here; see e.g. https://neurosciencenews.com/ai-creativity-23585/.

I do not find the point that models merely "interpolate between points included in its training set" (in many dimensions, or even non-linear, depending on the network architecture) troubling, since essentially human creativity, as we understand it today, is also a combination and "interpolation" of what we have learned and experienced. The conclusion that "mixing what has already been done" is "unable to create anything original" is wrong and contradicts experience. Just take a look at the patent database to see how many inventions there are that merely re-arrange or combine something existing in a slightly different way. A somewhat prominent theory of how inventions are accomplished is e.g. https://en.wikipedia.org/wiki/TRIZ or https://en.wikipedia.org/wiki/Systematic_inventive_thinking.

> these models can only interpolate between points included in its training set, can only mix what has already been done

You've just described Art. All art is in response to previous art, usually a variation of the same, but with a new twist (that tries to stand defiant/transcendent of the past).

Art is not necessarily a response to previous art. However this is hard “validate” as it’s very likely that the person producing the art has experienced _some_ art before.

It generates plausible (and many implausible) scientific theories. Unfortunately they are increasingly being censored, but some of their wild speculation was actually interesting intellectually and something that was neither obvious not something easy.

I m increasingly convinced that interpolation and extrapolation is what my brain does when it generates this comment.

Define "original" technically generative AI is always doing something original it's just the novelty and "creativity" that's in to question. Those are likely in the human realm. Even the prompts we cater are themselves what hold the spark of creativity.

Dude in 2ch (a.k.a 4chan in Japan) prompt stable diffusion to make hentai that girls with broken limbs and uterus fell off like a flashlight. The result is astonishing and mind blowing to me. I was so shocked when I first saw that. I think it is first of its kind.

You don't need to know Japanese, just click into the images. But warn you that stuff being drawn inside is something 1400-years-old elf that never get old.

https://itest.5ch.net/fate/test/read.cgi/liveuranus/16660895...

wtaf

lol I'm not going to go looking but I appreciate the answer, that does indeed sound original

Reply on news.ycombinator.com