Hacker News Reader

Comment by wkat4242 | Hacker News Reader

I can't wait for this to become usable. I love VR but the content generation is just sooooo labour intensive. Help creating 3D models would help so much and be the #1 enabler for the metaverse IMO.

View on HN

VR is especially unforgiving of "fake" detailing, you need as much detail as possible in the actual geometry to really sell it. That's the opposite how these models currently work, they output goopy low-res geometry and approximate most of the detailing with textures, which would be immediately register as fake with stereoscopic depth perception.

Agreed.

Everyone I see text to 3D, it’s ALWAYS textured. That is the obvious give-away that it is still garbage.

Show me text to wireframe that looks good and I’ll get excited.

Yup. I'm doing a VR project, urban environment. Haven't really found a good enough solution for 3D reconstruction from images.

Yes, there is gaussian splatting, NeRF and derivatives, but their outputs _really don't look good_. It's also necessary to have the surface remeshed if you go through that route, and then you need to retexture it.

Crazy thing being able to see things up to scale and so close up :)

By output you mean the extracted surface geometry? Or are you directly rendering NeRFs in VR.

Given the scale it wouldn't be wise to render them directly. There's also the issue of being able to record in real life without changes happening while doing so.

I should've have clarified it, but yes I was talking about the extracted surface geometry.

I find it much easier to remesh and deal with textures with a crappy 3d reconstruction vs working with 2d images only. I also shoot HDRI and photos for PBR. I find sculpting tools super useful for VR, but yeah its still an Art even with all the AI help.

Not meta VR, but one of my favorite things to do in gran turismo 7 with my PSVR2 is just “sit” in the cars and look around the cabins. The level of detail the devs put in is on another level.

Would be nice if they modelled real purchasable cars this way. With functional knobs and hud would be even better.

Knobs in cars are not a thing anymore... No need for 3D for that ;-)

Instead they have to model screens and the operating systems that allow us to change the temperature with only 6 clicks!

A number of manufa turers are heading back to knobs for frequent functions after customer sentiment whiplash.

I love simracing in the rain (rf2 is really punishing in particular and actually looks quite good) mainly for similar reasons.

> _really don't look good_

If you use "*" instead of "_" you can write in italic :) just a thought

does displacement mapping not hold up in VR?

I think displacement maps are often made by starting with high detailed models and converting some of the smaller details to normal, bump, reflection? maps etc.

correct. the process is called texture baking. however, displacement uses displacement maps, which are grayscale, essentially depth maps. displacement is not very useful for realtime rendering because then the displacement has to be calculated in realtime. displacement is mostly used in offline rendering. realtime rendering just uses normal maps.

It absolutely does, I don't quite get comments here. All VR games use normal assets with all their normal (map) shenanigans. It works the same as on desktop. Sure if you look at something super close the "illusion" breaks, but it has nothing to do with VR, same thing happens in flat screen games.

Maybe people mistakenly think that most standalone Quest games don't have those maps because they don't work? Well it's not the case. The standalone games (especially on Q2 vs Q3) have just very low performance budget. You strip out what you can to make your game render in 90 fps for each eye (each eye uses a different camera perspective so each frame scene has to be rendered twice).

I'd liken it to the trend from 5-10 years ago for every game to have randomly generated levels.

It does't feel like an expansive world - it's the same few basic building blocks combined in every possible combination. It doesn't feel intentional or interesting.

this is why I love half life alyx. it just gets so much detail in VR space in a way that no other game ever has that makes for a truly immersive experience.

Funny, pretty much everything on my Meta Quest seems to be "fake" detailing, without much detail in the actual geometry.

I mean, yes it's obvious because the GPU is only so powerful. The difference against my Xbox is night-and-day.

But even if VR is unforgiving of it, it's simply what we've got, at least on affordable devices. These models seem to be perfectly fine for current mainstream VR. Maybe Apple Vision is better, I don't know.

We need more of the stable diffusion tool set in 3d AI creation. Upscaling does incredible things in SD adding in all kinds of detail and bringing 512x512 to 4k. Inpainting to redo weird arms or deformities. Controlnets like outlines, depth, pose etc to do variations on an existing model.

The real fun begins when rigging gets automated. Then full AI scene generation of all the models… then add agency… then the trip never ends.

Half Life Alyx bakes everything on textures and is the best looking vr game.

Alyx doesn’t use models that look like cheese left in a hot car.

We leave that for Half Life 1.

I would push back on this a bit. The best games I've played on my VR headset are not "realistic". "Fake" is more about consistency I guess, if everything around me in VR looks equal amount of "goopy" then I don't think it will feel weird.

Doom 2 in gzdoom VR mode is amazingly convincing for how janky the graphics are. Even with the enemies as 2D sprites it’s quite compelling, so it’s my go-to rebuttal for all the “VR games need amazing graphics” sentiment. Good low-detail artwork is fine.

There are a few services that do this already, but they are all somewhat lacking, hopefully Meta's paper / solution brings some significant improvements in this space.

The existing ones:

- Meshy https://www.meshy.ai/ one of the first movers in this space, though it's quality isn't that great

- Rodin https://hyperhuman.deemos.com/rodin newer but folks are saying this is better

- Luma Labs has a 3D generator https://lumalabs.ai/genie but doesn't seem that popular

Regardless of cost, the meta verse is a dead end concept - at least until we get Ghost in the Shell-style computer-brain interfaces. Second Life probably was peak popularity for this kind of meta verse.

Reply on news.ycombinator.com