Game prototype using AI assisted graphics

0. Read the manual thoroughly, don’t speed read or skim.

1. Set the seed so that it’s only your prompt that changes while iterating on it “——seed **”

2. Set stylize to a low number “—-stylize 200”

3. Build the scene up by adding one part at a time (use a thesaurus to try out different words)

4. Weight the different prompt segments “SOME WORDS::10”, try to make the weights add up to 100% for easy math

5. Use negative weights to remove things you don’t want “BAD THING::-1”

6. Once you have zeroed in on a good prompt then increase the quality “—-quality 2”, try out different stylize values, and start rolling the seed many times.

7. Expect it to take 40+ iterations to get a good prompt and maybe another 20+ rolls to get a good seed. This can be less once you get good, or more if go for more advanced scenes. Your first few good images should take a few hours to get.

Consider shape-net - while being able to generate samples from the shape-net distribution would be a very impressive achievement, and these would certainly be useful as "assets" (decorations in a game scene), they don't have any of the real structure of these objects that would allow real interaction. They are effectively just inflatable decoys compared to real objects.

In short, "AI" isn't going to replace the artists, it's going to massively amplify their productivity and we don't know of any limit to consumer demand for realism and rich interactivity in games. If anything the number of game artists is going to increase, only the bar will be raised on the quality of their product.

https://i.imgur.com/UIBXXvj.png

We're definitely at the point where these tools can do amazing things, and yet they still manage to make bizarre defects in the output that no human would ever imagine.

I guess text-to-3d offers the potential of automatically creating entire worlds with very little human/artist intervention. But text-to-2d matches current processes with more chances for an artist to intervene, and also has more training material for art styles.

The rigging + animation is done by mixamo's AI so just touch up after.

He had to do a UV unwrap and then align the textures to this, this is time consuming just because it requires thinking about where to split the 3d model, to place the seams, then to fit it all in a texture, where you have to think about how much space to give each texture (more space = more quality). And then map the texture into this space too. This is a known pain in the 3d modelling for decades and no doubt AI will solve this in the next decade.

Using Mixamo once again saves an huge amount of time over hand rigging the mesh and instantly gives you access to a large library of animations that can be projected onto the Mixamo rig.

This is a very clever use of the available tools by very experienced character artist in my opinion. Your average HN reader is not going to be able to achieve this in 4, 18 hours or otherwise due to simply not having the required skillset to get from one end of the workflow to the other. Kudos to the author.

And then maybe modeling will take a hit too?

I do think it's cool thought. But ya I am not that happy with what I was saying. Could still be a good thing overall?? Who knows.

But I want to point out quality and artistic knowledge.

The poster is clearly a talented artist. They knew how to direct the images to what they wanted and likely rejected many iterations before they got to this point.

They knew how to then take that image and translate it to 3D , and make it come together. They knew how to make a topology flow that makes sense for their prototype etc…

I think a lot of people see “good output” and forget about the process and knowledge+experience required to make something good.

I take photography as an example. Every one with the means likely has a camera today. It still takes a brilliant eye to take an amazing photo.

You could take two brilliant photographers and someone without expertise to a gorgeous location. The two experienced photographers will have a higher likelihood to come out with amazing but very different images. The layperson could maybe take a good photo but the chances are much lower because they likely don’t have the experience to think about composition or story telling.

I liken these advancements in AI to that. Yeah, some people will put out pretty images, but there’s so much more that goes into making something great out of it that people underestimate

This kind of tech enables an indie to build a prototype with AI generated content, where the art direction is clear, then get funded in order to hire artists for the touch up. Kickstarters are often very visual, so this kind of stuff is required up front.

For a while at least coherence and consistency are not within reach of those systems, but that doesn't seem impossible. Perhaps for a little while longer I can see jobs emerging for curator of prompts, able to fine tune the experience for what is being generated and knowledgeable about the rapidly changing state of the art.

I've been playing text-based adventure games that I have made with ChatGPT and they are fascinating. Try the below command in ChatGPT (don't read the text below if you actually want to play it though!).

> Hi, I would like to play a text-based adventure game with you. I will say commands and you will respond either describing the situation or saying what the characters have said. Responses should be kept fairly short. The game will not end until I say the word 'endthegame'. In a combat situation, you will tell me how many hitpoints the character I am in combat with has, and I will describe the attack - you will evaluate that attack and decide how many hitpoints to deduct from the enemy. The game is set on a spaceship, however the spaceship is under attack. There is a portal gun. Part of the solution should be to teleport to the enemy ship and kill the enemies, and this should be telegraphed to the player after a few turns. The goal will be to stop the enemy ship from destroying our ship. I will say some things that will not be possible in context, or that are too vague, and if this happens you should respond saying why I cannot do the action, or that it was too vague. In general you should try and accomodate the players actions though, even if they have negative consequences - but not allow them if they are too vague or are multi-step.

Can the above now be considered 'source code' to a game? A text-based game which is fun to play (at least I find it fun!) and with free-flowing narrative, where your actions have consequences.

It's pretty wild to go into this game, where you can go up to characters and start 'proper' relationships with them, talk in natural language, and be able to make genuine choices about how to solve a problem that the developer didn't even dream of.

For a more pertinent current example, all you need to do is compare the mobile games industry to the computer games industry. The cost to make a mobile game is very low, and tons of games are released every day. The cost to make a successful mobile game is still very high, because all the money that is saved on development costs are spent in marketing instead.

Also, the video market collapsed in the 80s precisely because of low quality/oversupply games.

https://en.wikipedia.org/wiki/Video_game_crash_of_1983

So perhaps, it is the marketers that have a bright future from all of this.

Previously the transition was over the ages, ie iron, bronze, steel etc. But recent progression means we're going to start seeing stuff like that happen faster; I think atm a huge automation stopper is the cost. We manufacture stuff in China because it's cheaper, they don't automate as much as they could because: * Human workers can still be cheaper, because they pay them so little * Job creation is forced by gov because it would otherwise become a HUGE problem (in China/similar).

https://www.youtube.com/watch?v=RG7B29FBZKU

> As these images are created by an AI, no one has a claim to their copyright. [...] At least this is how I feel about AI art.

Perhaps he needs an AI to create some empathy.

> Prompt building

> I started by ordering the AI (Midjourney in this proto, but I use stable diffusion more) to make me a model sheet with turnaround images of a character.

b) Training datasets can only be made by humans.

c) A paid tier of Stable Diffusion is obviously coming. It will be differentiated by a better (and more custom) training dataset.

d) No serious developer would be caught dead using the stock free tier Stable Diffusion.

e) Big studios will most certainly hire closely-guarded artists to curate and expand their proprietary training datasets.

Even indie games need very bespoke art. Yeah, you can now generate images that are passable, but that doesn’t expand to making a cohesive world of items unless you’ve developed an eye for it like the person in the original post has. Hell, their post shows how important artists are because they needed that pre-existing knowledge to convert from an image to a useable state.

This also doesn’t expand to making fun animation or lighting that works for your game, or to interesting visual effects.

The concept art in the post are pretty generic looking within the genre. If that’s all people are aiming for, then fine, but it’s highly reductive to say it’ll replace artists. It’ll be a tool in the tool chest.

None of that is even touching on how much artists are involved in making sure a game also runs well on the system, while working with engineers.

I really just don’t think people understand how much art direction goes into even indie games. Something like fire watch or journey is immense.

Let’s take the concept examples in this image. Why do any of the details exist in there? Once you start thinking about the details of the world, you’ll start wanting to fine tune things. As you do this, surprise! you’ve turned into an artist yourself.

I just think we don’t teach art appreciation , or even appreciation for things outside our domain, to people. We see “image is good” and think that’s all an artist brings. Engineers are especially susceptible to this. We think in binary results.

So it’s easy to think “it’ll replace my need for artists” if that’s our mindset, but I think that line of thinking comes from not understanding that the journey is an important part of the result.

What about self-learning chess engines? All they take as input are the rules of chess. And the output (the games they play) can sometimes be described as peaces of art. My point is that it should be possible for AI to create art without tons of (input) data. Even if it is not what we experience at the moment.

It is not, you just have to try many times and post-process a lot.

Once you know what tokens trigger the right answers it's relatively easy to get where you want, as usual with these tool you should be flexible when it comes to details, but the overall design can be defined

https://www.midjourney.com/app/feed/all/

I don't think generic AIs are the right approach anyway, I think we're better off with precise procedural models with a lot of knobs and some knowledge of rules (eg. metahumans).

Maybe you can put an AI in front of it to generate what you want using the procedural models, but pixel output is just not good enough for games.

Consistent character creation is a big topic the community interested in and there are many tricks people use for that purpose.