N-Dimensional Gaussians for Fitting of High Dimensional Functions | Hacker News Reader

N-Dimensional Gaussians for Fitting of High Dimensional Functions

sdiolatz.info

67 pointsjasondavies11 comments

When I think of turning sequences of images into gaussians, I think of the difficulty of getting generalizable information that can be re-rendered out of the pipeline; textures and lighting, basically. From the description at the top of the paper, where they mention adding dimensions for things like albedo, I got excited.

But the demos don't do any re-rendering / change of lighting / etc, so I can't tell if this paper is just a 'super high render quality at same training time' paper, which is of course great to have, or if it has a shot at being extended to get us scenes that can be adjusted as to lighting and texture in-engine.

Any experts care to chime in?

You should be able to add those features as extra dimensions.

I wonder how well this works for scenes with multiple characters, posed skeletons for the characters, etc

How would you avoid degenerate parts of the configuration space, where characters clip through each other, clip through level features, etc?

Could it theoretically scale to render a rave crowd with 1000 random people dancing under wild lighting effects?

12d gaussians video has a moving light source and z rotation.

The penultimate video has a lighting change

This problem specifically (3D reconstruction with representation fitting) is really an overfitting nightmare, they just adapted to it not really overcame it. Nonetheless interesting work.

I get the feeling that some abstract geometric/topological approaches will let us get under fitted general pretrained NERFs that rapidly specialize to a given context, let you combine separate contexts into a composite scene, and maybe let's you give behaviors to entities in the scene.

At that point you're kind of trying to generically train over all possible scene graphs. But maybe some kind of geometric delocalization scheme lets you train on small fragments of larger scenes, transfer learning between scenes, and quickly represent and render generic scenes.

The point of a Nerf is effectively overfitting a neural network to a scene anyways (I think they say this in the original NeRF paper - or somewhere similar).

I expected Gaussian processes.

Can someone relate this to Gaussian processes?

A Gaussian process fits a single high dimensional Gaussian, for example, by treating n observations along a single dimension as a n dimensional space.

Gaussian mixture models fit a large number of low dimensional Gaussians for example you might imagine 2D data generated by several 2D Gaussian superimposed.

This approach is just an example of the latter. It uses higher dimensional Gaussians to capture extra information from a scene, but not in the emulation of an infinite dimensional space in the way that defines Gaussian processes.

To add to a sibling comment, if you're interested in learning a bit about the both the Gaussian (as in a density estimator like Gassian Mixture Models, aka GMMs) vs Gaussian Processes (GP), I have some write-ups here: [1] and [2].

[1] Fun with GMMs https://blog.quipu-strands.com/fun_with_GMMs

[2] This is a larger article on BayesOpt, but I've a section dedicated to GPs: https://blog.quipu-strands.com/bayesopt_1_key_ideas_GPs#gaus...

If you understand what a Gaussian Process is, you understand what a Gaussian Distribution is. This work estimates the parameters of many Gaussian Distribution in order to fit the approximate geometry of a GD to a geometry in a scene.

Reply on news.ycombinator.com