Hacker News Reader

Comment by sniklaus | Hacker News Reader

I am confused, a quote from Wikipedia: "The Ken Burns effect is a type of panning and zooming effect used in video production from still imagery."

Our framework takes a single still image and animates it via panning and zooming while adding 3D parallax. Does adding the 3D parallax not make it the Ken Burns effect anymore? Please let me know in case I am misunderstanding anything.

Yes, the examples that we have shown do not zoom as much as common Ken Burns examples. This stems from our framework processing the input at low resolution due to limitations in deep learning and the overall complexity of the problem. As such, there is not enough detail that could be zoomed into. This will be improved in later generations.

View on HN

> Does adding the 3D parallax not make it the Ken Burns effect anymore?

Yes.

Specifically, the Ken Burns effect describes taking a still, 2D image and panning and zooming on the 2D image in a motion picture.

Adding 3D parallax makes it something quite different in the eyes of many of us on this forum. It is cool! We would just not use the words "Ken Burns effect" to describe what you've made.

On the contrary I think it basically serves the same purpose as the Ken Burns effect, except it tastefully incorporates the natural depth of the image.

One might even argue Ken Burns would have paired the panning and zooming with this parallax effect if he could.

I look forward to this enhancing basically all photo slideshows for a long time to come.

That is a nice way of putting it, thank you for chiming in!

I know nothing about this. But it seems obvious that, whatever we call your work, Ken Burns' use of panning and zooming is a subset of it.

Thank you for clarifying, I see where you are coming from! My background is in research and it is fairly common to extend / reuse an existing terminology, but I totally understand and accept your point of view.

There is probably a name for this kind of "taking a 2D image, separating it into 2d cutouts, and panning with parallax". It's an effect that's been around for a while (hand done, not automated as you have it) but it's never been referred to as a "Ken Burns Effect"

Googling "How to make a parallax video from 2d pictures" brings up lots of tutorials, none of which mentions "Ken Burns Effect"

https://www.google.com/search?q=how+to+make+parallax+video+f...

Conversly searching for "How to make the Ken Burns Effect" will show that it's not the same effect your software makes

https://www.google.com/search?q=how+make+the+ken+burns+effec...

I think adding the 3D parallax gives the footage a very unique effect, and quite different from the traditional Ken Burns 2D scale and pan. I would say enough to make it deserve its own name, such as the sniklaus effect. Ken Burns is Ken Burns. This is something quite different, even though it may be inspired by KB.

I would be happy if it gets its own name in the future. I am voting against "sniklaus" effect though since this was a team effort and I could not have done it without my mentors.

Would it not be possible to do the image processing on the original image to create several pixel masks which then draw in the high resolution image data?

For instance, you take the source image at resolution n*n and do all processing at n/2 or n/4. Then at the moment you are going to composite the finished image you instead draw "pixels" from a source image which contains X,Y not RGB. Then you upscale the output image x2 (or x4) and replace each X,Y index with the relevant pixel from the source image (adding X%2,Y%2 or X%4 to the indexes to return to source resolution).

That is called guided upsampling and we are in fact doing just that to get a better estimate of the scene geometry. Specifically, we estimate the scene geometry at a resolution of 256x256 pixels (for a 1:1 aspect ratio, it varies for other resolutions) and then use the original image as a guide to upsample this estimate to 1024x1024 pixels. So yes, it could be used to a greater extent but it needs more work to be able to do this.

Reply on news.ycombinator.com