I feel really bad for artists because unless you need something very specific or a big studio, you are going to be replaced sadly.
Even indie games need very bespoke art. Yeah, you can now generate images that are passable, but that doesn’t expand to making a cohesive world of items unless you’ve developed an eye for it like the person in the original post has. Hell, their post shows how important artists are because they needed that pre-existing knowledge to convert from an image to a useable state.
This also doesn’t expand to making fun animation or lighting that works for your game, or to interesting visual effects.
The concept art in the post are pretty generic looking within the genre. If that’s all people are aiming for, then fine, but it’s highly reductive to say it’ll replace artists. It’ll be a tool in the tool chest.
None of that is even touching on how much artists are involved in making sure a game also runs well on the system, while working with engineers.
I really just don’t think people understand how much art direction goes into even indie games. Something like fire watch or journey is immense.
Let’s take the concept examples in this image. Why do any of the details exist in there? Once you start thinking about the details of the world, you’ll start wanting to fine tune things. As you do this, surprise! you’ve turned into an artist yourself.
I just think we don’t teach art appreciation , or even appreciation for things outside our domain, to people. We see “image is good” and think that’s all an artist brings. Engineers are especially susceptible to this. We think in binary results.
So it’s easy to think “it’ll replace my need for artists” if that’s our mindset, but I think that line of thinking comes from not understanding that the journey is an important part of the result.
b) Training datasets can only be made by humans.
c) A paid tier of Stable Diffusion is obviously coming. It will be differentiated by a better (and more custom) training dataset.
d) No serious developer would be caught dead using the stock free tier Stable Diffusion.
e) Big studios will most certainly hire closely-guarded artists to curate and expand their proprietary training datasets.
The current situation where you'd download billions of free images off the Internet only works once, and only if you somehow justify it as a research endeavour. Once this thing is monetized intellectual property laws will kick in.
A tool like Copilot can more or less automatically improve via telemetry, I’d expect the same thing as image models catch-up. I’d also expect the human signals to get further and further downstream of the actual creation process (e.g. Gameplay tester reports visual bug versus artist manually edits character)
An aside, but people use and pay for Copilot, it is out of the research lab phase.
None of this is solved algorithmically.
They won't be able to compete just on data because lots of people are producing custom models, and they can be blended together like a stylistic and thematic pallet. Plus, if you own the pipeline you can use embeddings, dreambooth in specific elements and set it up in batch mode doing a random walk through the latent space for cheap. This stuff is not hard to set up and run, and with money and expertise, you can create something that is both unique looking relative to other AI art, and more optimized for your workflow than a service.
Image gen services will compete on ease of use, general quality and access to models that are larger than can fit in 24-48g vram without a big up front cost. There will probably be some services that provide specific features that people use even if they own their own pipelines, but the core customers will be smaller shops who don't use it enough to justify a real investment.
AI can make training datasets too.
For example to replace the human generated stuff from stable diffusion you could have some random-ish image generator coupled with some sort of image classification AI. As long as you have a good enough classification AI (or even more than one) that tells you what images are, you can focus on random-ish image generator algorithms to generate training data for another AI to generate images from descriptions.
(this is obviously with lots of handwaving and there will be problems that need to be solved - e.g. to avoid 99% of the generated training data be stuff like "noise on noise" but have some form of variety :-P), but the point is AIs generating data for other AIs is something that isn't far fetched and you don't need to think in terms of a single AI either)
(though https://commons.wikimedia.org/wiki/Commons:Freedom_of_panora... may matter here)
SD is open source, the paid stuff is going to be the paid models like this I think.
I don't think many would claim that the world would be a better place if regulations were put in place to limit electronic computers in order to keep human computers employed.
2 - You're twisting my argument. I don't care if artists are employed or not, or that some jobs are transitioned out from the economy. I care that people who put in work get the value proportional to that work. You should, too.
When you use one of these AIs that have been fed millions of images in order to train them and generate an effective output, you are necessarily consuming the images themselves, without which the AI wouldn't do anything. In that process, the artists - whose copyrighted work is, again, fundamental to the development of the tool - have been paid nada, they have not even consented to the use of their images in the training process. How does that track?
This would be a very different conversations if these AIs only used public domain art, of which there's plenty. But then again, it wouldn't be much profitable, would it?
Of course not. So why is it different for an AI?
Otherwise, you have to agree that we're talking about apples and oranges here.
AIs don't get "inspiration". They get the source images they need to function. An AI also can't produce an output that's outside of the realm of their dataset.
And if I told you that, as someone who has done art for decades, that the human creative process is very similar to how an AI is trained on existing images, would you believe me and move on?
> Because if that's the case, I'm afraid you have a very odd idea of how these AIs function.
The design of neutral nets, by definition, were derived from the workings of the human brain.
Why should I believe you and move on? "making art for decades" doesn't make you an authority on any of the relevant subjects: "how art is processed in the brain" nor "how AI processes these images." I don't think you understand the fundamental differences between the process of looking up references/inspiration and kitbashing.
Think about it: you live your life. You experience things. You experience art, and experience emotions or have interactions with other humans grounded in that art. You form connections with certain styles or techniques.
If you then turn around to create art, you form in your mind a general idea of what you want to create. You then draw on your past experiences to actually create the physical art. What process other than statistical extraction from your mind could it come from?
For sure I believe there are things that we don't understand about the human mind. I think the impact of drug use on art creation is very interesting, for example. It indicates that random chemical processes in our brains can play a large determining role in the actions we take (and in this case, the things that we create).
But to say that humans do not use some sort of inbaked statistical world model in the creative process seems wrong to me.
This isn't some hypothetical. I went through the art portfolio scene and survived 4 years of critiques - I know about the sacred process called the "creative process". None of my and my peers' work would exist without the inspiration of the centuries of art work that stood before us. This is what we call art in the industry and by the public masses. The criteria you established for "why AI art isn't art" applies directly to the "conventional art". So I have to ask, why is AI art different?
The issues of copyright infringement with AI are real though. Much of today’s AI is directly copying subregions of training data, and can sometimes be prompted to reproduce images from the training data verbatim. Humans don’t do that unintentionally, even though sometimes they do mean to steal from others. Suggesting that art school is the same thing as a training dataset is a bit hyperbolic.
And, much like with our brains, when it happens it doesn’t actually exactly reproduce parts of the source image. But, you have actively pay attention to notice what happened. It makes an image that is overly similar conceptually. To our brains that feels the same. So, that’s enough to convince someone at a glance that it is the same.
But, if you look at an overfit result of “The Beatles Abbey Road album cover”, you’ll see things like: Band members are crossing the road, but they are all variations of Ringo. Vehicles from that era are in the background, but they are in a different arrangement and none of them are directly from the source. The Band members are wearing suits, but they are the wrong style and color. There are the wrong number of stripes on the road. It’s not the same as a highly skilled human drawing an iconic image from memory. But, it sure is darn similar.
And, besides all that, everyone working in the tech considers the overfitting of iconic images to be a failure case that is being actively addressed. It won’t be long before it stops happening entirely.
In the meantime, I’d challenge anyone to try to make an overfit result that significantly reproduces a specific work of every promoter’s favorite, Greg Rutkowski, using Dall-e, Midjourney or the Stable Diffusion models released directly by Stability AI. Greg’s pixels aren’t in the model file to be copied. Only a conceptual impression of his style.
Not really, though that is another legitimate issue.
I was talking about 1) the fundamental training and inference process, which remembers pixels, not concepts or techniques. Today’s AI learns to create imagery in a fundamentally different way than people do. And 2) image generation AI based on text prompts like Stable Diffusion can easily be asked to reproduce training data by having a prompt that is narrow and specific enough. This is not over fitting, it’s a function of the fact that some inputs are quite unique, and you can use the prompt to focus on that uniqueness.
I’d like to see examples of using SD to copy some specific piece of art that hasn’t been plastered millions of times across the internet. Sure, you can get a decent Mona Lisa knock off. Maybe even a strong impression of the Bloodbourne game cover art marketing material. But, reproducing a specific painting from Rutkowski would be quite a surprise to me.
Here are the examples you requested: https://techcrunch.com/2022/12/13/image-generating-ai-can-co...
Yes the training process looks at pixels, because that’s all it has. That’s the point. Humans don’t look at pixels, they learn ideas. It’s not in the least bit surprising that AI models shown a bunch of examples sometimes replicate their example inputs, examples are all they have, and they are built specifically to reproduce images similar to what they see, I’m not sure why you consider that idea “loaded”.
Read the paper. What I found is that a random sampling of the database naturally found a small subset of images that are highly duplicated in the database. Researchers we able to derive methods to produce results that give strong impressions of images such as: a map of the United States, Van Gogh's Starry Night, and the cover of Bloodborne :P with some models and not at all with others. The researchers caution against extrapolating from their results.
> We speculate that replication behavior in Stable Diffusion arises from a complex interaction of factors, which include that it is text (rather than class) conditioned, it has a highly skewed distribution of image repetitions in the training set, and the number of gradient updates during training is large enough to overfit on a subset of the data.
Is this sarcasm? The history of art is full of artists who created their own, signature, unique and original styles. Take, I don't know, Vincent Van Gogh, for an example, who had a very distinctive style, so distinctive that he didn't even start a school of art probably because it would have been too blatant to copy him. There was nobody before him who painted like him. Who did he "copy" then?
Hell, when humans first made art, back in the time we lived in caves, their art styles, which are still absolutely unique, had nothing to copy from, simply because there weren't any artists before them (by definition: "when humans first made art").
So, yes, humans learn how to create art from each other, but they also created the whole idea of art entirely on their own, and they can take what they have learned form others and turn it into something completely new, never before seen.
Now, you show me an original art style created by an "AI". Show me AI art that isn't only borrowing and copying, but goes beyond that, like human artists can.