3D Gaussian Splatting in a Weekend

The magic of Gaussian splats is their ability to render photorealistic outputs without material properties, explicit ray tracing, etc. They do this by synthesizing complex light transfer in the scene via these fuzzy blobs overlapping — but they need to be able to change color and transparency with view angle to recreate much of that light transport. Hence, relatively heavyweight data.

There are many approaches to reducing the data volume, and they get increasingly complex when you add a time component. Not even worth listing publications here because it's changing so quickly, just plan to look at the SIGGRAPH pre-prints in a couple of months. Exciting times!

The data savings are huge, but the compression step is slow and requires a GPU.

It would be awesome if we could find a better/faster method of compressing these 45 dimensional monsters.

i.e. the colour (and possibly other surface properties) vary depending on their direction, which is (or at least can be) encoded spherically (as spherical harmonics).

The width/size of each point/splat is also not just a radius, it can be anisotropic, and have an orientation in space, so again, it can vary its size depending on orientation when rendered.

It has been mildly amusing watching the AI crowd learn about point clouds though, and use things the VFX industry was using in the early 00s (spherical harmonic encoded materials - we had light-dependent as well for relighting - points with direction and anisotropic widths, etc)...