Trellis – 3D mesh generative model
trellis3d.github.io
trellis3d.github.io
I'm not putting it down, it truly is an amazing achievement, and it feels like it marks the end of hand-made assets. I don't even feel bad for the artists, I just feel bad for myself, because I want things made by people, for the inherent property that they were made by people. This is the same reason I don't care for procedurally generated games, I want to walk worlds that arose in the minds of others, not just worlds, if I wanted a procedually generated world that just exist for no reason in particular, I'd walk outside..
I don't want content, I don't just want stuff to sift through, I want pieces of art, made by my peers with their own visions, ideas, values, insights and personalities. They don't need to look that good, they just need to have been made with purpose.
I don't know why, but this comment reminded me of an experience I had a few years ago, when I started exercising outdoors. I rarely went outside prior to that and stayed in a relatively dark room.
One day I looked at the sky and thought: Wow, these clouds do look like the ones in video games, thinking of Horizon and Assassins Creed. This just pertaining the comment about the "procedurally generated" outside world.
While looking at the assets I also felt a bit of sadness. I was looking at the "Two-story brick house with red roof and fence." and was then thinking about how it reminded me of the three.js animation/keyframes example [0].
I asked myself if we will lose something very valuable. The three.js example was hand-crafted by persons, a real intention behind every choice made, while with Trellis it's just "poof, there it is", an amalgamation of all work found in the internet and possibly in games.
Some value will be lost through AI, but this makes handcrafted content even more valuable. The question is, if we will really value this enough for it to be sustainable for the artists.
A significant portion of game developers hate level design and the only reason they don't do procedural is because it's hard, so they are forced to build hand-crafted worlds. I'm one of those and I would find it pretty hilarious if anyone played my game thinking the levels "arose" in my mind, like I'm some kind of profound artist. I take great pride in other aspects of game development, but my level design is not one of those.
And that shows, that really, really shows :)
Now they get to make even more of the soul-less trash in shorter time..
I'm not putting this tool down, it's an amazing technical achievement, and the results are absolutely mind-blowing, but, to me, it is what it is, and it's just like, my opinion, dude, not some statement of absolute truth.
You hating level design and wishing you didn't have to do it at all has absolutely no bearing on my wanting games where the assets are made by hand.
Conversely,me not wanting products made by people who don't like making them, should have absolutely no influence on you. I don't care if your passion is some other field of game creation, go do that, and have someone who enjoys level design do the levels, if you can't, well, then I guess you'll have to just accept that I might not want your game, and that's okay too, for both of us, you don't have to make something _I_ in particular like, and I don't have to accept your criteria for what I like.
I want, as an inherent property of the stuff I consume, a few things whose merits can be argued endlessly about, but I'm not arguing about their merit, my opinion, my criteria for selection is inherent property itself.
I'm not arguing whether there are any difference, I'm not arguing one is better than the other, I'm not arguing why one should be chosen over the other, I'm simply stating that among my selection criteria is that particular property of origin. It in itself, alone, nothing about it, just it.
I want movies recorded on actual film, not movies that look like it, inherent property not its merit.
I want books written by human minds, not books that "you can't prove was not".
I want paintings painted by pencils held in human hands, guided by human hearts and minds, regardless of whether I am looking at a photograph of that painting, the property of it's origin is important to me, not its merits of lack thereof.
So yeah, you can attack the merits of doing things one way or another all day long, but you don't get to say what I can an can not chose as my selection criteria.
Is it soulless when a sculptor doesn’t source their own clay and marble directly from the earth?
Or when a musician uses an instrument made by someone else, or a composer uses digital sounds recorded by someone else?
There are many different forms of artistic expression and many rely on relatively mechanical and “soulless” work being done for us by someone or something else.
Tools like this can open up new and creative world building options to people who previously didn’t have access to 3D models. It increases the opportunities for creative expression rather than diminishing them.
Not everyone needs to do everything. And if someone’s amazing idea can get out of their head and onto paper/film/video or into a game I’m all for it.
There will be a lot of AI shovelware junk. But it doesn’t all have to be that way. Now more people compete on larger landscape of ideas.
The artistic message would be disjointed, muddy, but indisputably an unmitigated human expression.
So you put an artistic director in charge of curating and unifying the collective work.
Still a human expression.
This is what AI represents, and what the prompt writer represents.
The data in LLM is undeniably human. Everything it "knows" is an extension of and exclusively composed of real human data.
The prompt writer has a choice how much of his own human input to prioritize, and how much raw humanity to allow spontaneously.
Oops, old.reddit.com/r/outside/ is leaking again
Look at people like Martin Nebelong - they’re learning how to leverage AI without losing the human in the loop.
Well, guess what, very soon even I could do that. So what do these studios have in store to make us come back to them?
This shifts the attention to story development and away from graphic designers. It does not mean cheaper games. It means more successful indie games with fewer team members. It also means fewer games because as I understand it right now, the only reason new games are pumped out is to keep the larger industry perpetually employed and other more time consuming projects funded enough to be developed.
Art is much more than pictures on your monitor. If you want pieces of art, made by your peers, visit your local galleries and buy it. I don't know who you are or where you live, but I'm willing to bet that where-ever it is those local galleries exist - and the artists that exhibit there would love to sell some of their work.
And you can be sure that human-made art will remain, and be valued, because art is what humans love to make most of all.
EDIT: I went looking for those threads and found my own comment wishing for this 5 years ago https://news.ycombinator.com/item?id=22642628
The next step is to automatically add "nodes" to the 3D images where the model can pivot, rotate and whatnot and then boom, you have on-demand animated, interactive content.
Feed it some childhood photos and recreate your memories. Add an audio sample from a loved one and have them speak to you. Drop into VR with noise-cancelling headphones for extra immersion. Coming soon! Click here to join the "Surrender Reality" waitlist
The rest of the artists are not dumb, and they have a lot of talent. I'm sure many of them will use the same models to come up with their own games.
How this whole thing will play out is anyone's guess. Long term, I'm hoping for new types of jobs to come into being. But who can say?
And in addition to that it's also useful for rending still pictures. 2D generated images by AI so far have incorrect lighting and many errors. Once it's a 3D scene rendered by something like Blender (which is free), it's going to look flawless: lighting is going to be correct (and configurable) and all the little details that are wrong are going to be easily fixed.
We already have insanely powerful tools and apparently from here it's only going to get way more powerful real quick.
My gut says a 3D engine + this would be a superior solution to the current approach of rendering rasterized video directly from the latents (coincidentally, Sora got released today).
It may not be tractable to train a network to rig and animate meshes, as well as setting up an entire scene to be a "digital twin" of random videos, bit I imagine such a set up would have finer-grained control over the created video while keeping everything else in it the unchanged
The next step is to generate models with higher quality mesh topology that allows animation and editing without breaking the mesh. I've done a lot of retopologizing and if I (or AI) were to rig these models as-is there would be all kinds of shading and deformation issues. Even without animating they are glaringly triangulated up close. But I suspect really high quality 3D asset generation is just around the corner because all you'd have to do is join up the approach seen here with AI quad re-meshing based on estimated direction fields and feature detection, which is also getting scarily good.
Well not really sure what you're talking about here wrt nodes (adding in arbitrary rotation/zoom sounds great in theory if all you're looking for is a lazy susan or spinning exorcist heads), but the next steps will likely be more around ensuring sane symmetrical topologies, better UV maps, and automatically building rigging (FK/IK) to allow for easy animation.
It's limited to mostly human shapes, but I've personally used it in combination with the 3d-pack in comfyUI to generate 3d models and rig them, starting from a text prompt.
Trellis looks like a more capable model generation tool than TripoSR and marching cubes, which is what I was doing in Comfy: https://github.com/flowtyone/ComfyUI-Flowty-TripoSR - It worked but models ended up having a slightly "melted wax" appearance.
This type of flow is definitely already here for low quality assets (think mobile games). I'm excited to go play around with Trellis, looks like a significant bump up in quality.
Hopefully one day we get some open source alternative to Mixamo that plays nicely with the rest of the open ecosystem.
Which resulted in this: https://video.non.io/video-2732101706.mp4
Honestly not bad at all. Getting to the point of being able to use this as game assets.
I was hoping you could upload several images from different angles to help it, but that doesn't appear to be a feature.
I'm not saying anything about the quality of the model here - just that the F117 is almost certainly going to be an unfair test.
It does have failure cases but the success rate is fairly high and when it works, the resulting meshes are reasonably usable (maybe not to game dev production standards - but that still leaves plenty of other use cases)
The result was pretty good for the mesh, at least 100x faster than having to do it from scratch
Guess its more trained on natural and biologic structures and textrues, rather than more structural or symetric data.
From a performance pov I've imported ridiculous meshes not intended for games into unreal. It worked.
I can see this being ok for anything other than triple A games right?
1. Unless we're strictly talking about static meshes (such as for scenery), building out the rigging/animation for garbage topologies is going to be absolute hell.
2. Your performance POV was for ONE mesh, but if your quad/triangle count is 10x higher than it needs to be, how well is it going to scale particularly on low-end gaming platforms?
3. Lighting/shadows/reflections can produce some pretty UGLY effects when your surface polygons are a mess.
This is using meshes which seem a lot easier, and I'm unsure who's working on BREP or at least models that have some feel for dimensions and physics.
ie It could still be meshes, but the designs would be physically reasonable like how a skilled tradesperson may make a device with their intuition without running FEA, etc.
With mesh faces now supported in BREP, I'm more optimistic about a mixed modeling approach, where you can do the braindead find-the-cylinders conversion but keep the rest mesh, not needing to force it into some eldritch contortion of BSurfs.
The main advantages of BREP are:
(1) You capture design intent because of the explicit dimensioning and constraint model, which of course is still not used enough and 2D drawings are useful.
(1a) This intent is often needed (even if just implicitly) during the manufacturing process as machinists, toolmakers, etc. + their software (CAM, G-Code, etc.) convert them into physical parts.
(2) They are human understandable and editable.
(3) The legacy ecosystem uses old CAD, and it's very hard to switch - ie Boeing isn't putting 777 in a new CAD unless it has some massive advantage.
So having BREP, or perhaps a mixed approach like you suggest with the feel of BREP (feature tree, direct modeling, etc.) approach would ease the transition.
I haven't cracked any of these open in Blender to look at the topology so they could be (and probably are) an absolute mess, so it's likely that attempting to make changes to the model may prove to be more trouble than it's worth, but still neat to play around with.
Images / Resulting models: