Hacker News Reader

Comment by markisus | Hacker News Reader

Gaussian Splatting allows you to create a photorealistic representation of an environment from just a collection of images. Philosophically, this is a form of geometric scene understanding from raw pixels, which has been a holy grail of computer vision since the beginning.

Usually creating a Gaussian splat representation takes a long time and uses an iterative gradient-based optimization procedure. Using RGBD helps me sidestep this optimization, as much of the geometry is already present in the depth channel and so it enables the real-time aspect of my technique.

When you say "big deal", I imagine you are also asking about business or societal implications. I can't really speak on those, but I'm open to licensing this IP to any companies which know about big business applications :)

View on HN

Thanks! That makes a lot of sense, I might dig into this after work some more.

By "big deal," I meant more for people specializing around computer graphics, computer vision, or even narrower subfields of either of those two -- a big deal from an academic interest perspective.

Sure, this might also have implications in society and business, but I'm a nerd, and I appreciate a good nerding out over something cool, niche, and technically impressive.

So, is there some amount of gradient-based optimization going on here? I see RGBD input, transmission, RGBD output. But, other than multi-camera registration, it's difficult to determine what processing took place between input and transmission. What makes this different from RGBD camera visualizations from 10 years ago?

There is no gradient-based optimization. It's (RGBD input, Current Camera Pose) -> Neural Net -> Gaussian Splat output.

I'm not aware of other live RGBD visualizations except for direct pointcloud rendering. Compared to pointclouds, splats are better able to render textures, view-dependent effects, and occlusions.

Except that no view-dependent effects that would benefit multi-view consistency are present in your splats.

So yes, it's very much like the RGB-D visualizations from 10 years ago, just with splats instead of points.

Here is an example of a view dependent effect produced by LiveSplat [1]. Look closely at the wooden chair handle as the view changes.

I'll concede that ten years ago, someone could have done this. But no one did, as far as I know.

[1] https://imgur.com/a/2yA7eMU

Reply on news.ycombinator.com