Hacker News Reader

Comment by lwb | Hacker News Reader

To me this was the least interesting part of the Stadia reveal. State share was the real innovation imo. Would any serious game studio use style transfer ml to actually replace their visual artists? Didn't seem like the quality was there.

That said, it is an awesome PoC. Just not something I see being practically applicable.

View on HN

> Would any serious game studio use style transfer ml to actually replace their visual artists?

No, a serious (AAA) studio would not. OTOH, the platform might intend to support producers that aren't AAA studios, and whose limited pre-release resources are focussed on gameplay more than art assets, and those producers might view it very differently than AAA studios.

Indeed, I'd be curious to see what some indie devs do with it.

Imagine a game where you step into a portrait and are suddenly in the art style of the image. You suddenly walk into a countryside that feels american-gothic style, and then time-jump to 1940s New York in the style of Nighthawks.

(different) Stylization as a medium in games isn't that explored, and the opportunity for indie devs to play with that is, IMO, really cool.

The Witcher Blood and Wine expansion actually did a great job with this. There is a main storyline quest where you enter a haunted painting

Would it help in game development? For example giving developers better looking playable prototypes while the artists are still working out their designs?

You still have to do all the work of building the geometries for the art assets, are you really saving that much on not having to color/texture those assets?

Maybe? One of the hacks of adding more assets without having to do much work is just recycling existing assets by assigning different color schemes/textures.

That's one thing, plus look at the result. This makes a very lowpoly game quite impressive visually.

And this is just a trivial graphic filter through a neural network. A lot more can be done with AI.

Come on, think bigger. If they're doing this for art styles you can bet your bottom dollar they'll be doing something using Google images. No longer having to actually create 3D meshes - just a Google image search, click a button and have the geometries automatically created for you. Then just add an automatic style.

>> Come on, think bigger.

True.

Actually Google is just seeing the wealth of innovation in procedural generation and in computer creativity, knowing where this is going, and just want in on the action.

And since as a cloud gaming provider, they stand to make much more money than tool vendors, their incentive is much bigger.

Any technology that can automatically perform the tedious task of coloring 2D art would be a godsend to indie game devs as well as amateur artists.

Hypothesis: the rough and relatively unpredictable results are not so much a good fit for finished products as it is for _rapid visual prototyping_. A good art director could use this to quickly settle on a rough goal around which the rest of the game would be targeted. I'm aware the pipeline usually goes from art director -> development, but this could help, say, a competent dev team with a gameplay direction but no artist yet, or a long-standing developer that wants to use its existing engines and mechanics to draft new, distinctive franchises.

This is the use case they highlighted in the keynote. It doesn't appear to be a system designed for shipped games.

I think the more interesting applications will come when you get tools like this used by visual artists, complementing handmade design. The demo is cool but uses style transfer very bluntly, applying one style to the whole image with high intensity. I imagine the future of this is something more like neural "shaders", able to apply real-time pixel alterations to scenes or objects within scenes. Used carefully, they could increase the range of creative choices a visual artist can make.

This is where I think the value lies.

I have been thinking about a process where a procedurally generated template is completed by artists and subsequently used as a base for training an AI to apply to further procedural generations.

I also wonder if you can do micro style adjustments to unify artistic style, train a net on one particular artists work, the rest of the team copy the general style but use a final pass of a style transfer to make it a closer match.

Honestly seems more like a user-end personalization than something a developer would use. In a game settings menu, you choose an image that maybe you uploaded to use as the game style? Definitely something people would use. Especially if they figure out how to make it selectively apply to only certain textures/objects or parts of the screen, which seems likely.

Street Fighter 4 had a feature kinda like that (with predefined selectable styles). AFAIK it was not popular. Sometimes my friends would turn it on as a joke.

Exactly...This is a cool gimmick to try once, but it will seriously distort the intended art style.

It is another example of forced intrusion of DL techniques creeping into places where it is not needed in the first place.

Gimmick is the perfect way to describe it. I really don't understand why they even included it.

Actually, I wonder if there's potential in something like this for accessibility purposes. If you have one of the many variations of color-blindness, for instance, could such a restyling be helpful? Could other neurological disabilities be similarly aided by dynamically restyling visuals?

There are many low-tech solutions that can be applied instead of shoehorning in neural networks for this purpose.

Yes, but they also require active participation from the developer and many developers are unwilling or unable to spend the time to implement them, especially when there's a rather large range of disabilities to account for.

It's possible that tech like this could offer a user-definable solution to these issues.

One incredible use is actually in network training for things like AR and autonomous driving. You want a slightly fake world and you could easily try out various network approaches using a free fake world found in game engines. Microsoft has something for this but cannot find a link or the name right now.

I would have been much more interested in using deep learning to make human faces more realistic. Somebody was experimenting with it in the past quite successfully IIRC, but I'd love to try it on a real game.

Maybe used as a player entering dream like equivalent after eating magic mushrooms. Other than that you’d be hard pressed to see this in any game in memory

> Would any serious game studio use style transfer ml to actually replace their visual artists?

I doubt that was the point. IMO it is more of a palette testing tool.

I was baffled with the style transfer announcement. It seems arbitrary, random, and out-of-place. Style transfer (applied to art) is really neat, but I can't see it being used to re-clothe games for production.

Before I get downvoted, yes I do think it could be useful for prototyping and conceptualization during development, but I doubt anyone would actually ship a complete game with the entire camera view transformed by style transfer.

Reply on news.ycombinator.com