Hacker News Reader

Comment by contravert | Hacker News Reader

I just want to add my $0.02 currently working at a games studio that is integrating AI generated art into our art pipelines.

Midjourney definitely generates really high quality art based on simple prompts, but the inability to really customize the output basically kills its utility.

We heavily use Stable Diffusion with specific models and ControlNet to get customizable and consistent results. Our artists also need to extensively tweak and post-process the output, and re-run it again in Stable Diffusion.

This entire workflow is definitely beyond a Discord-based interface to say the least.

View on HN

If you would give a talk about this, I would watch it - despite being out of the graphics for almost 10 years now. Really want to hear from the trenches about the workflows, benefits and challenges you have.

Here is a test I did the other day of rough sketch (hand drawn) -> clean line work (AI) -> coloured (AI). This workflow gives 100% control over the output because you can easily adjust the linework in the intermediary step.

https://twitter.com/P_Galbraith/status/1649317290926825473?c...

This is using Stable Diffusion and the Control Net Lineart Model. The coloured version is pretty rough but it was a quick test.

In my opinion Stable Diffusion is vastly superior to Midjourney if you have the skill to provide input to img2img/ControlNet.

I have some other earlier workflow experiments on Youtube if you're interested in this kind of thing https://www.youtube.com/pjgalbraith

use https://github.com/deep-floyd/IF, it uses LLM to generate exact art you need.

The image quality of DeepFloyd is much lower than Stable Diffusion 1.5 though, it's a pretty major tradeoff. Can definitely be part of the workflow since it really is good at composition, but right now it's not a replacement.

Deep Floyd doesn't allow commercial usage, such as a game studio using it.

They said that they will change the licence for the final version IIRC.

Really curious about the challenges you've been seeing as your art team's been trying to integrate AI generated art into your pipeline.

Excited by the potential for artists to accelerate their creative process with advances like ControlNet, but makes sense there's a lot frustrating about the process today.

Exploring tools that might help. If you might be interested in chatting, email me at tom@adobe.com. Thanks!

> AI generated art into our art pipelines

I'd be interested to know where the art ends up in the game? Do you mean 2D backgrounds and billboards in-game? Or are we talking cut-scenes and menu screen art?

Reply on news.ycombinator.com