Hyperlapse makes first-person videos smooth and speedy
news.microsoft.com
news.microsoft.com
https://hyperlapse.instagram.com
http://blog.instagram.com/post/95829278497/hyperlapse-from-i...
Microsoft's version uses post processing on their servers to generate a 3D model of the scene. They then synthesize a new video based on this data.
Microsoft's approach is more time intensive and can lead to strange artifacts in the video. But lets them generate a smooth video if, for example, the person wearing the camera glances around quickly mid shot.
The Microsoft product is not limited in this way from what I recall of their research. It is much more robust.
It has gone almost entirely unnoticed, but it creates outputs a world better than this Microsoft app. I get that they're going for 'the videos you already have', but if I have a mobile device with so many sensors, why wouldn't I use them?
(1) - I mention that because whenever something pushes the edge, if someone finds that it doesn't work on their own Android handset, they tend to be the absolute loudest about it, declaring that therefore it must not work on anyone's handset. It is the most bizarre aspect to apps in Android land.
Standard video stabilization crops out the pixels on the periphery to create consistent frame-to-frame smoothness. But when applied to greatly sped up video, it fails to compensate for the wildly shaking motion.
Hyperlapse reconstructs how a camera moves throughout a video, as well as its distance and angle in relation to what’s happening in each frame. Then it plots out a smoother camera path and stitches pixels from multiple video frames to rebuild the scene and expand the field of view.
[1]http://blogs.microsoft.com/next/2014/08/11/hyperlapse-siggra....
Not to be confused with the hyperlapse paper from 2014 that was instead using a very computationally heavy structure from motion pipeline. Quoting the 2015 paper: "Our approach resides in between the Instagram approach and that of Kopf et al [the 2014 paper]".
This made me get a tickle in my throat just two seconds in. Definitely very uncomfortable to watch.
If you took the image stabilization you linked and sped it up the end result would be garbage. Different techniques for different goals.
You can 2x the video on Youtube to see for yourself. The major drawback is cropping and computation time, which is where Microsoft's technique excels.
The new technique is described here: http://research.microsoft.com/en-us/um/redmond/projects/hype...
In essence, it would remove frames that produces really big sudden movements or it would "re-shape" it to make it consistent or smoother and it does all that with some very deep algorithms. Ultimately your long timelapse or fast video shows up as if there was done with expensive stabilization and great care. My only complain is why iOS is being left out :(.
For much better demonstration of how cool this is: https://www.youtube.com/watch?v=SOpwHaQnRSY
Very different from the work of Keith Loutit and Tony Leech where a stationary camera is key.
http://www.keithloutit.com/ https://vimeo.com/11445353/
Filming everything in motion and then calculating stability is an interesting problem.
Well done to Microsoft and Instagram for attempting it.
edit: fixed links
[1]: https://www.youtube.com/watch?v=_CvzPyQF7fA [2]: http://imgur.com/w1mLrM6 (not the same frame in both pictures, but you get the idea)
This is probably a pretty hard video to do since so much of the field is just plain white snow.
We develop a dynamic programming algorithm, inspired by dynamic-time-warping (DTW) algorithms, that selects frames from the input video that both best match a desired target speed-up and result in the smoothest possible camera motion in the resulting hyperlapse video. Once an optimal set of frames is selected, our method performs 2D video stabilization to create a smoothed camera path from which we render the resulting hyper-lapse.
This is great to hear! Feel free to reach out if you have any questions adsolank at microsoft dot com
http://research.microsoft.com/en-us/um/redmond/projects/hype...
It says the hyperlapse is constructed from selecting the right frames and doing a projection onto the proxy geometry.
Solution: play it back at 30x speed.
wouldn't the 3d extraction + reconstruction for smoothness alone be super cool?
edit: Or do they need so much redundancy and variety in the images that they need to have an "oversampled" source?
The error I got:
Sorry - nothing to see here.
There may be several reasons: The app you’re looking for doesn't exist or the app developer is not currently running a test or you are not eligible to participate in the testing program. If you received the link to this page from someone, we recommend you contact them for more information.