Next-generation video encoding techniques for 360 video and VR | Hacker News Reader

Next-generation video encoding techniques for 360 video and VR

code.facebook.com

49 pointstilt8 comments

Video presentation of the same material: https://www.youtube.com/watch?v=zpuUUxMfaJs

ok i have a 360 camera and this is really sweet but what could i even do with the video once encoded with facebook's transform? yet at least... guess i'll have to watch the video and wait a few month/years for more adoption. I'm having a lot of fun using my camera to generate sky/light maps for 3d animation. its a ricoh theta s. a very weird product ahead of its time in a fun but buggy way!

I see nobody focusing on 360 degree (whole-sphere) stereo videos. It is much bigger challenge, I think.

Google's Cardboard team is working on Jump [1], which is a 16-camera rig with a computer vision post-processing step to create stereoscopic 360 video.

Lytro is working on Immerge [2], which is a light field camera for "cinematic VR". So in addition to stereoscopic 360, you should be able to move your head around within the volume of the sphere as well.

[1] https://www.google.com/get/cardboard/jump/

[2] https://www.lytro.com/immerge

The challenge on 360 stereo is on the creation side. While you are able to capture decent mono 360 video with two or three cameras 360 stereo is a different beast you need at least 6-8 cameras and a lot of processing.

I think you need much more than 6-8 cameras, at least 60 - 80 cameras, so you can get a decent image for each spherical angle of a head. The challenge is to efficiently compress 60 almost identical video records.

http://jauntvr.com/ http://vrse.com/ http://nextvr.com/ https://www.felixandpaul.com

and several others.

We (The Ambassadors Lab) did a full 360 stereo live action short last year for Terre Des Hommes. Bought an Ultimaker 2, 14 GoPro Hero 4 Black cameras and made a pentagon style rig.

The stitching/vfx pipeline was mainly the two of us on nuke stitching, warping, projecting and painting. Using ocula for some stereo processing. It was a great experience and the result was fantastic. The feeling of presence was great. Especially when viewed using an Oculus Rift.

Youtube click drag version and google cardboard stereo versions can be found here: https://www.terredeshommes.nl/en/vr-experience

Making of here: https://vimeo.com/129114672

If you're in the neighbourhood of Amsterdam we'll gladly give you a propper oculus viewing.

Right now we're in the post production phase for a music video for the dutch band De Staat which we also shot in stereo.

Stereo, we think, is definitely the way to go.

Shameless plug: we are looking for frontend developers that don't mind getting involved in projects like this ;)

Reply on news.ycombinator.com