This is what Doom looks like on a holographic display

> 1) Isn't this just based on "naive" implementations? Surely there are optimizations to not have to render all 45 view planes at all times similar to how 3d rendering doesn't render scenes that are not visible until the character moves to change perspective?

Maybe you could add head tracking to prune some angles, but if you were going to do head tracking, other comments have reports of good results without a lenticular setup. You'd probably lose out on the multiple viewers aspect.

I would think naively that there's some amount of work that could be shared between so many renders from nearly the same viewpoint. But that probably requires driver changes to make it available, I did find an article from NVidia about rendering four viewpoints in one pass, which might be applicable and help somewhat: https://developer.nvidia.com/blog/turing-multi-view-renderin...

For some context, I've been making games across console/mobile/VR for about 12 years so I can provide (hopefully) some educated responses.

For 1)

- Their SDK implements this rendering strategy, so either they haven't been able to justify the effort for additional optimization, or (much more likely) there isn't a tenable general purpose solution that doesn't fall apart horribly in many circumstances. This latter point is true of many optimizations in games, they work because you can very narrowly scope them to what you are doing.

- Frustum and occlusion culling are used, like you mention, to avoid rendering meshes that are off-screen. This isn't a panacea though, there can be significant CPU cost to doing so to the point where it can be more performant to disable it. Case in point, I used Umbra occlusion culling in my VR game in only a few scenes because my CPU budget for 90 FPS was so small that the PS4 couldn't keep up.

- Those 45 planes are a series of slices that are very closely projected in space, so the odds that there isn't any work to do is almost nil. Even more true when you consider dynamic scene elements, such as lighting and shadow casting.

For 2)

- All of the challenges present for 3D content creation are unchanged with this device, it just adds more on top (limited FOV, depth blur, and fringe blur). You can't wizard away seeing color separation at the border of the screen, it's just fundamentally the hardware's limitations.

Another example, the PSVR headset's OLED panels cause a purple blur to happen on edges that have too much light/dark contrast. The solution? Call an API and change a floating point "base brightness" level for the headset, or change your lighting -- for each level (this is Sony's recommendation!). Entirely game and scene dependent, and not something that an automatic solution could accomplish.

- Some of the "magic" techniques invented for VR performance may have the possibility to be applicable (skipped frame re-projection, foveated rendering), but those are based on the math between two camera projections not 45.

- Any improvements that could be created by a Carmack-level person are going to be market driven. The population of people that fit in that category would have the opportunity to work on whatever they want, and VR was compelling to many of them because of how damned cool it is (was?). By comparison, this tech is a novelty that a regular consumer will probably never see.

The big difference is, unlike VR, you don't know where the viewer's eyes are, so you can't jump from 3D to 2x 2D so easily, you genuinely need a volume resolution instead of an area resolution. I suppose another way to think about it is, if they're splitting 4k into 45 planes, you're chopping 8.3 MP into 0.184 MP (~430x430px) x 45 "pixels"/voxels of depth resolution. 3840x2160 pixels is the same number of samples as a 202x202x202 voxel cube. A 1000^3 voxel display equals 31600x31600 pixels of resolution, which is maybe possible to render at 30fps with a very, very simple application and a top-end GPU. 3000^3 = 164300x164300. We would need to rethink the entire graphics stack to figure out a "GPU" that could more efficiently "draw" to a hologram at high resolution and refresh rate. Maybe something like DLSS but 3D. Even then, the bandwidth will be insane. 1000^3, 1 billion voxels, * 32 bits of color/alpha, is 32 gigabits per frame. 30fps would need 960 gigabits per second of bandwidth. Displayport 2.0 is 80 gbps. The fastest commercial optical transceivers are 100-400 gbps, although top-end GPU memory bandwidth is approaching 8000 gbps, good enough for 2000^3 @ 32bit @ 30fps or 1000^3 @ 240fps. The "GPU" will need to be tightly integrated into the display.

It would be better if we could just have 3x 2D projectors that somehow interfere in a medium to display, but I don't know of a way to accomplish this. Not to mention none of this allows you to be inside the displayed content as VR does.

The migraines is due to several factors, example:

1. Your brain recreates the 3D environment from physical objects and is "trained" to certain distances and movement, etc. When you use 3D glasses, either movie or VR, the "stereo" effects are anything but natural. There's a lot of stuff flying around at "non-natural" distances, especially near you. The effects are exagerated otherwise at some point you will "stop noticing it's 3d unless you pay attention". The exagerations are compensated by your brain and also causes eyestrain. Look at a pen, closer, closer, at some point you see 2 pens, now do that back and forth for 30 minutes, You will have the same pain.

2. Perceived movement vs. internal ear and balancing your body. VR is awesome at screwing everything up. I love VR but can't stand it for a long period because of that. Your brain expect you falling, or this thing comming at you to hit, you walking should have some motor feedback, driving too. Nope. No G's, no wind, no nothing. To put it in simple terms... brain tilts

you can look up a lot of stuff about this on the internet with VR sickness, motion or 3D sickness... lots of articles, i.e. https://en.wikipedia.org/wiki/Virtual_reality_sickness

I had problems playing Quake on a 2D screen. It went out with intense exposure. I would do 30 minutes, get sick, start the next day, eventually I've managed to pull out 12 hours in a row. Life was fun and simple back then ;). I don't have that luxury anymore for VR but I'm sure it would be the same process to desensitise through gradual increased exposure.

... Ah.

It takes probably less than $300 to DIY this from a off-the-shelf 15.6" display and a piece of dot lenticular sheet...

It doesn't seen to be, or rotating the thing with a fixed camera wouldn't produce the effect, I guess ?

https://lookingglassfactory.com/tech

Surely the depth effect only worked with one eye closed?

And that video is over 13 years old! Too bad this idea didn't catch on more widely.

Thanks. I just created a nice storage box for my Wii, got everything organized and put it on the bottom of a large pile of things to sell.

This sounds way too interesting, so I'll be doing some digging, apparently. Any idea if this would work effectively on a large projector screen?

Hm. I haven't really looked too deep into TrackIR, but I always thought it tracked your head movement to simulate head rotation within the game. So you could look around more naturally. The linked Wii "tracking" demo used the tracking to find your position relative to the display to change the rendered perspective accordingly. Does TrackIR also allow this?

I think that sums it up pretty perfectly. It's the single reason I watched the video and my first thought was "there's no way that works that well"

From the comments, here, it sounds like it might. There's some interesting applications, here. Thinking back to last night, it would have been really nice if when I was looking at the split-screen in my driving game with my kids, my own corner had that kind of depth to it. Hell, driving/flying games, in general, suffer from realism issues due to not being able to faithfully produce the effect of looking out a window.

Doing this on a larger screen (and throw in a transparent LCD after you work out how to bend all of this into that, assuming it can be done) and you could have a light-switch that changes the scenery from "outside" to "outside somewhere else"[0].

[0] I remember seeing that in Back to the Future II as a kid and thinking it'd be really nice to have on a cold, gloomy, February day in Michigan.

I guess that’s not how people imagine holograms based on sci-fi.

Because they aren't holograms. A hologram is "a recording of an interference pattern which uses diffraction to reproduce a 3D light field" [0]. This is something completely different.

[0] https://en.wikipedia.org/wiki/Holography

Because it's a lenticular screen.

This seems to happen a lot. Not the first time I've seen this crop up in an HN thread.

Thanks, not one I'd encountered.