How is it related to NeRFs?
And how is it different/better than rendering triangles/voxels/SDF?
Is it just ray-marching these 3D gaussian shapes?
Why is a gaussian used specifically?
NeRFs are made of a dense grid of tiny AIs that each can answer what the inside of that grid cube looks like from any angle. You march through the grid to see how a whole scene looks from a particular angle.
Gaussian Splats target the same goal with a different approach. A splat is just a fuzzy oriented blob. You render them by sorting them front to back then marching through them per pixel until it gets opaque enough.
The Gaussian aspect is important because Gaussians are differentiable. With that you can start with a bad estimation of the splats, render the bad estimation, compare it to the photos and back-propagate the difference between actual vs desired to iteratively improve the bad estimate to a better one.
Triangles are great because you can make solid, closed surfaces from them. But, you have to maintain the connectivity between the vertices. That makes adding and removing triangles from a mesh a major challenge.
SDFs are great because you can easily make crazy changes to the topology without ever thinking about connectivity. But, you have to be able to describe the shape of the whole scene with math. For realistically complicated shapes, that adds up to a lot of math per marching step.
Splats are loose, independent blobs. They are easy to individually manipulate. Adding and removing them is trivial. But, it’s difficult to get the appearance of a solid surface from them.
> The Gaussian aspect is important because Gaussians are differentiable.
Sure, but why Gaussians and not any of the many other well-known smooth functions? Is it because of lack of imagination (since the Gaussian is a sort of "default" choice)? Or is there some reason to pick gaussians, specifically?
https://hannibunny.github.io/orbook/preprocessing/04gaussian...
It's better than other representations because...no one has been able to reconstruct scenes with other representations this well.
Gaussian splatting is not popular because it's a good rendering method. It's popular because there's now a good technique for generating great 3d gaussian splats from 2d images. And now that you have gaussian splats, it's nice to render them quickly.
If you wanted the inside of an apartment or house like matterport it would easily be 500mb to 1gig of data and forget it if you try an outdoor scene