Gaussian explosion

“I am what now?” “Everyone calls it that”

Anyone knows other videos / channels that combine info and dry quality humor?

It's better than other representations because...no one has been able to reconstruct scenes with other representations this well.

Gaussian splatting is not popular because it's a good rendering method. It's popular because there's now a good technique for generating great 3d gaussian splats from 2d images. And now that you have gaussian splats, it's nice to render them quickly.

NeRFs are made of a dense grid of tiny AIs that each can answer what the inside of that grid cube looks like from any angle. You march through the grid to see how a whole scene looks from a particular angle.

Gaussian Splats target the same goal with a different approach. A splat is just a fuzzy oriented blob. You render them by sorting them front to back then marching through them per pixel until it gets opaque enough.

The Gaussian aspect is important because Gaussians are differentiable. With that you can start with a bad estimation of the splats, render the bad estimation, compare it to the photos and back-propagate the difference between actual vs desired to iteratively improve the bad estimate to a better one.

Triangles are great because you can make solid, closed surfaces from them. But, you have to maintain the connectivity between the vertices. That makes adding and removing triangles from a mesh a major challenge.

SDFs are great because you can easily make crazy changes to the topology without ever thinking about connectivity. But, you have to be able to describe the shape of the whole scene with math. For realistically complicated shapes, that adds up to a lot of math per marching step.

Splats are loose, independent blobs. They are easy to individually manipulate. Adding and removing them is trivial. But, it’s difficult to get the appearance of a solid surface from them.

If you wanted the inside of an apartment or house like matterport it would easily be 500mb to 1gig of data and forget it if you try an outdoor scene

Most of actual research I've seen (i.e. papers) are not Unity/Unreal, they rather are CUDA/Python.

The "Unity Gaussian Splatting" is not a research per se, just integration of that existing technique into Unity. As for why Unity, well yeah that's because I have experience with it, and am comfortable using it.

There's many other people experimenting with in in web ecosystem (WebGL, WebGPU, three.js, aframe etc.), that you can say is properly open source.

Blender as is right now is lacking certain functionalities to efficiently do gaussian splatting. However, for upcoming Blender 4.1 several pieces are landing - I have extended PLY file importer to be able to import custom attributes (as needed by gaussian splats), someone else contributed APIs to be able to write compute shaders from a Python addon. Once various pieces like that are in place, someone could indeed make a decently performant gaussian splat add-on or something.