Show HN: Turning a Gaussian Splat into a videogame

Those tend to move in the wind. Animations don't work well with splats. Or with any data structure except polygon meshes.

Edit:

Responding in the parent because HN says I'm "posting too fast" and should "slow down".

> Have you seen https://www.4dv.ai/

Yeah, but this seems to be just a 3D GS video (captured from several different camera angles), similar to how an ordinary 2D video is just a series of still frames. For 3D games this would be unsuitable since animations often have to be generated on the fly based on game physics. Even for pre-baked animations the memory cost of loading each frame individually would be too inefficient. For polygon meshes you have just a single static mesh that is deformed over time.

> Dreams managed to animate splats on the PS4. Admittedly, not quite the same type of splats, but there is probably a middle ground here where it can be made to work

I'm pretty sure Dreams only allowed animations as translations and rotations, not something that approximates soft skeletal animations. And even translations and rotations would be problematic since 3D GS scenes rely on baked lighting which would then result in objects no longer fitting the scene.

I have another data point: my ten year old ThinkPad. I get about 10 FPS. Lowering the quality doesn't seem to increase performance.

But I am amazed by what I am seeing, and amazed it runs at all!

I think it won't be long before the whole world is mapped, and "playable".

DonHopkins on July 12, 2021 | parent | context | favorite | on: I Stopped Using Emojis

>What we saw was, if you go too far in that [representational] direction because you want to be inclusive, people don’t see themselves represented and they’re not going to use it. You have to have enough specificity to represent you enough, but not so inclusive that your emoji palette is hundreds of thousands of emoji.

Scott McCloud wrote a whole book about this: "Understanding Comics".

https://en.wikipedia.org/wiki/Understanding_Comics

>One of the book's key concepts is that of "masking," a visual style, dramatic convention, and literary technique described in the chapter on realism. It is the use of simplistic, archetypal, narrative characters, even if juxtaposed with detailed, photographic, verisimilar, spectacular backgrounds. This may function, McCloud infers, as a mask, a form of projective identification. His explanation is that a familiar and minimally detailed character allows for a stronger emotional connection and for viewers to identify more easily.

https://en.wikipedia.org/wiki/Masking_(illustration)

>The masking effect or masking is a visual style, dramatic convention, and literary technique described by cartoonist Scott McCloud in his book Understanding Comics in the chapter on realism. It is the use of simplistic, archetypal, narrative characters, even if juxtaposed with detailed, photographic, verisimilar, spectacular backgrounds. This may function, McCloud infers, as a mask, a form of projective identification. His explanation is that a familiar and minimally detailed character allows for a stronger emotional connection and for viewers to identify more easily.

Scott McCloud and Will Wright discussed masking and other issues in their 2002 GDC discussion, "When Maps Collide":

https://www.gdcvault.com/play/1022567/When-Maps-Collide-A-Co...

Understanding Comics and masking influenced The Sims 1 graphics architecture and design (using detailed pre-rendered 2d+z sprites for the environment and simplistic real time 3d graphics for the people), which fortunately ran fast on the common un-accelerated 3d graphics hardware of the time (greatly expanding the user base), and synergistically enabled user created content (which was essential to its success) which I described in this earlier post:

https://news.ycombinator.com/item?id=3676313

>Going 3D at that time in history meant that the quality of the graphic would take a huge hit, as well as the rendering speed, and fewer people would be able to run it because it would require a high end computer, so it was just not worth it.

>Using 2D pre-rendered sprites means that the artists can use as many polygons, rich textures and lighting techniques as they want in 3D Studio Max, and tweak them until the sprites look perfect, and that's exactly what the user sees. You just could not approach anywhere near that quality with 3D graphics at the time. Of course things are a lot different now!

>That was during the time that The Sims was also in development. One reason The Sims was successful is that it did not try to be full 3D, and ran well on low-end computers (the old computer that little sister inherits from big brother when he upgrades to a gaming machine). It used a hybrid 2D/3D system of z-buffered sprites, with an orthographic projection constrained to four rotations, three zooms, and only the characters were rendered with polygons into the pre-rendered z-buffered scene, using DirectX's software renderer.

>I developed the character animation system and content creation tools for The Sims, and when the EA executives were reviewing the technology to decide if they should buy Maxis, to justify our approach I bought them a copy of Scott McCloud's book Understanding Comics, which explained a concept called "masking" --

http://www.themedianinja.com/glenn/legacy/default_links/anim...

>Hergé's Tintin comics are a great example of how that works: The idea is that by making the background environment very realistic (i.e. rich pre-rendered sprites from high poly models), and the characters themselves more abstract (i.e. efficient real time 3d texture mapped low poly models), the readers (players) can more easily project themselves into the scene and identify with the characters. Much in the same way an abstract happy face can represent everyone, while a photograph of a person's face only represents that person.

>The other fortunate consequence was that it was easy for players to create their own characters and objects by editing the textures and sprites with 2D tools like Photoshop, without requiring difficult 3D modeling tools like 3D Studio Max, so that enabled a lot of user created content by kids instead of professional artists, which was essential to the success of the game.