Plenoxels | Hacker News Reader

Plenoxels

alexyu.net

109 pointsunwind11 comments

So it doesn't use a neural network, but it is still optimized by gradient descent. Differentiability is the key!

At this point it's not "NeRF" but "DONeRF" that they should go toe to toe with. https://github.com/facebookresearch/DONERF

Interesting, but the video is only available at 480p for now, so it's hard to actually see any details in the demo. I assume this was uploaded very recently and YT is still processing the higher-quality versions.

Don't hold your breath, a lot of these projects are done at what we consumers would consider low rez.

If you scroll down on the page there are high definition clips of the results.

Forgive me if I have this all wrong, but in summary it seems to be that they have created a faster algorithm for creating a 3D (built of voxels, not polygons) from a set of 2D photos of that scene.

Am I right?

The actual representation is using voxels to index a spherical harmonic coding of scene detail (I think), and that includes transparency so the novel view synthesis accretes light along the ray length. This lets it represent transparency and other lighting effects that aren't possible with simple photogrammetry.

It seems to be a faster way of calculating differentiable volumetric geometry from a scene (whether the input is computer-generated or captured from the real world). I didn't get it at all until I explored the linked NeRF site first.

No, AFAICT it's actually building a representation of a more detailed ray-traced 3D geometry in a way that can be rendered real time from any angle and with realistic lighting

Per the paper it's not quite real time (15 fps), and they suggest converting to a different representation (PlenOctrees) for real time.

But yes, it's far more realistic than your simple mesh-and-texture; the video shows it handles reflections and transparency. When NeRF came out (~2020?) it was a major step forward in light field quality and this technique appears to maintain that quality.

Reply on news.ycombinator.com