Hacker News Reader

Comment by jsheard | Hacker News Reader

It's... neat? But I'm struggling to think of what the applications of this would actually be. 2D artwork usually doesn't have a consistent 3D space, which they acknowledge, but they don't seem to have overcome that problem in any useful sense. The scenes are barely coherent once they move from one of the originally drawn camera positions.

View on HN

Both Futurama and Family Guy sometimes use 3d rendering for vehicles for example, and render it in a cartoon looking style and composit it with flat 2d animations.

Maybe similar kind of things could be an application of this.

Another possible use-case might be a game development studio developing a license game based on a 2d cartoon, but making the game 3d. They could use this as a tool for visualization while planning and developing, to iterate quickly and to reference how the original 2d could translate into 3d.

Not really? In those examples the hand crafted 3d assets already exist, this thing could at best recreate the 3d geometries the show creators made themselves. That seems useful mainly for cloning someone else's work.

“Similar kind of thing” meaning for another show that wants to do the same but who have not created the 3d assets yet.

Team of 2d artists draw the desired vehicles for the cartoon from two or three angles. Software like this makes a usable 3d model of it.

If you were making 2D drawings with the intent of turning them into a 3D model then you would draw them to be coherent in 3D in the first place. The whole novelty of the research in the OP is that they're trying to reconstruct drawings that were never intended to make realistic sense in 3D.

Even if AI has a place in the 2D to 3D part of a pipeline, surely you'd still want the 2D artwork be unambigously representative of what the 3D asset should look like, rather than providing self-contradictory input data and praying that the AI can magically make it make sense.

True. For the second use-case I mentioned it still applies though. Where a studio is making a licensed 3d game based on an existing 2d cartoon.

A show that's being doing purely 2d art can't just integrate the 3d art in their pipeline on a whim. If they can, they probably already have the skills to make the model outright.

SpongeBob brazenly violates 3D space rules (I mean, they also have fire underwater...). The writers and artists both draw heavy inspiration from Looney Tunes, where such rules are broken because it's funny to break them.

A refined version of this could be used to make stereoscopic versions of cartoons.

On the other hand you are probably better off only using the depth prediction and filling any voids in using image generation instead of this mapping process.

I think this is just a device used to demonstrate and advance the technology. I doubt this has a real application in this context given how little work is needed to 3D model these kinds of environments anyways.

With future advancements you could pump out video games for many series.

While rough, these do look better than some implementations of the artwork for cartoon games.

I could see some value maybe in giving an artist feedback on where the model detects inconsistencies between different viewpoints.

That assumes that consistency between viewpoints is actually desirable - part of the charm of 2D animation is that things can be stylized or exaggerated or simplified in ways that don't come naturally in a 3D workflow, where the "default" is for things to fit together realistically and any deviation from that takes additional effort.

If you do want numerous 2D artworks which share a realistically defined 3D space then that can easily be done by making a very rough 3D scene and then painting over it, you don't need any AI for that.

If consistency was highly desirable you'd just model the 3d space from the start...

Maybe you could better construct a 3d model of a demolished landmark from old paintings and photos?

The renders it creates are underwhelming, but it seems good at determining the location and angle of the camera.

I could see it being used to create a "scratch track" that human animators animate on top of. An aid to tweening.

Reply on news.ycombinator.com