FILM: Frame Interpolation for Large Motion
github.com
github.com
You can get 60 fps upconverted footage from motion interpolation to not exhibit the effect by adding copious gaussian motion blur.
The same problem (plus perceived judder due to low fps) appears if you play a video on a screen equipped with correctly tuned strobe backlight.
It's one of the reasons as to why simple kernel interpolation (blurmotion) with most kernels does not exhibit this effect either.
You’re right in that it doesn’t change the amount of blur, I’m not sure why the person you’re replying to thinks that. But it does change the aesthetic of the blur.
Perhaps they’re thinking of the issue with the Hobbit where the higher frame rate at a given shutter angle (180 let’s say) that shortened the blur compared to a traditional 24 fps film.
They adjusted this for the second hobbit film to leave the shutter open longer (not a 360 shutter, but something like a 270) to bring back some of that feeling.
Though I don’t think I saw the 48fps in cinemas in Australia
It felt like the camera was always bouncing and moving around randomly in a way I found very displeasing...
also, i think the 3:2 pulldown (24->29.97fps) and interlacing that typically causes jitter/blurring is absent at framerates that are even multiples (30->30 or 30->60)
to take my response from elsewhere , consider a single drawn frame impact that is held for 2 or more frames. That timing is integral to the performance.
When you interpolate, it no longer holds the right amount so the performance changes.
People may scoff at that, but it’s a huge deal for artists. It was hell when we had dailies and motion smoothing on one of the TVs turned on (if we weren’t using screening rooms). Suddenly all the notes were different and acting didn’t feel right.
They capture one *field* every 1/50th of a second, and interlace it.
The reason for this is that it was hard to scan a full 625 lines in 1/50th of a second, and it would have required ludicrous bandwidth for the AM transmitters of the day and massive output valves to drive the scan coils in TV sets which would then produced huge flyback spikes.
So the decision was to scan 312.5 lines per field, offset slightly, with each pair of fields forming one 1/25th of a second frame. Telecine was easy in the UK because you could speed the film up a little and then each TV frame would be two fields per film frame.
If you shoot 50i you can deinterlace to 25p by combining the two sets of scan lines from each field with a bit of clever processing to hide the "combing" caused by things moving between fields, or you can double it up to 50p by just copying each line out twice with a bit of clever processing to smooth out the steps. I could post some video clips showing this if you were interested.
Of course modern digital TV cameras are capable of shooting 60p as you say, but anyone shooting "telenovela" stuff is going to be shooting 60i.
This is why all your video editing software has got those ballachey 59.94/24.97 modes.
It gets worse because for telecine you need to need to show two frames of 24fps film for every three frames of video, so you need to slow your film down to 23.976fps so it fits.
Ballache.
1. https://www.idtech.com/blog/what-does-animating-on-ones-twos...
2. https://www.awn.com/tooninstitute/lessonplan/timing.htm#:~:t....
3. Search for "A good example would be this cut, and especially the first four shots." in https://animetudes.com/2021/03/06/the-kanada-style-in-contex...
> not something to be fixed.
The first does not imply the second in every case. There are many, many instances where a creator with unlimited resources would have chosen to drawn more frames. And now we can. We don't have to indiscriminately interpolate everything to 120 Hz, but I think it's clearly possible to make huge improvements to a whole lot of hand drawn animation.
I recently watched Avatar 2 in IMAX 3D with 48 FPS, and it looked incredible. If only all media in the future could let go of the arbitrary 24 frame limit, which was made a hundred years ago because filmmakers wanted to conserve their film usage.
Most animation is shot for 24 fps on twos, so 12ish frames (we usually go between 8fps and 24fps)
The difference is that we design poses to read at a given frame rate and have performance characteristics at those rates. If you were to interpolate it, you’d change a fundamental characteristic of the acting.
How long a ball makes contact with a surface is important and interpolation struggles to understand something that should stick to a surface for an extra frame versus something that should be blended.
Yes you’ll potentially get lower strobing but you’re also now getting a different performance. Maybe you don’t care, but it’s fundamentally changing things away from the intention.
This is a different discussion than just simply what frame rate video is shot at.
> I remain unconvinced by those arguments
Though hopefully that does go some of the way to convince you that the arguments are legitimate. But personal subjective preference can override intention of the author.
0: http://gurneyjourney.blogspot.com/2014/07/elongated-in-betwe...
Are you talking about inconsistent frame timing aka judder? 24 divides evenly into 120, so you shouldn’t have that problem on a 120hz tv anyway.
If motion is too fast for a given framerate, my mind immediately stops interpeting it and I stop being able to focus. I absolutely hate 24fps content with pans, because the moment there's a pan, I just see a blurry mess and it takes several seconds to readjust after the pan is over. It's annoying, it's distracting, and I hate it.
Likely due to variable frame rate, I noticed this too. If they kept it at 48 FPS throughout, it probably wouldn't have been so stuttery, but they decided to make it randomly 24 FPS within the same scene which was almost unwatchable.
With lower frame rates (24/25) you can expose individual frames longer, which leads to more motion blur with movement relative to the camera. Motion blur can be an deliberate choice, or a necessity: with higher frame rate the exposure time per frame drops, which means you need more light per frame, which means you either need more budget to light things or you need to push the sensor/film more which typically degrades color and noise performance. Nowadays this is not a big issue, but it can still be relevant if you e.g. do night scenes with available light or similar. Of course having double the images also affects the budget in different ways when it comes to storage, post production, offloading times, battery consumption, how well you have to perform your practical effects and so on.
If budget is not an issue, high frame rates change how a film or a scene feels as well. Things feel more real. Shorter exposure times/smaller shutter angles are traditionally used to make things feel more stressful. So habing a higher frame rate can be a good thing, but it can also be a bad thing, and it all depends on the story you are telling. For example a war zone action shocker with shaky immersive hand camera might profit from the high frame rates as it feels more real (and thus more shocking), while a fantasy story like the hobbit might feel a bit too real and thus uncanny using the same thing.
Art is very often about creating an aesthetical distance to the reality. That is why most masterful paintings don't look like photographs, why black and white photographs can aestheticise their motif, why directors never show people going to the shitter in certain genres etc.
IMO high frame rates in film are heavily influenced by the narrative aesthetics pioneered by video games. Many modern films have a video-gamey feel to them as well, in the way they show motion, physics and the camera angles/motions that are chosen.
So frame rate in film is an aesthetical choice, and like everything choices can be made wrong, especially since films are made 4 times, once on paper, once on set, once in the editing room and once in the head of the viewer. Maybe what seemed like a good choice on day one on the set turns out to be a bad one on the last day. Maybe on set everything seemed like the right choice, but in the editing room the focus of the story shifts and you regret not shooting it differently.
I wonder when we will see variable frame rate films where film makers can use the frame rate as an aesthetical choice on a per scene basis..
The current best way to do this that I know of involves a ton of manual masking of elements at different z depths and the results will look still worse than if you had done it in camera from the start.
Nobody wants to have to do that to 40 minutes of a 90 minute movie unless there is a really good reason to do it.
There’s this. It looks like they used it in Avatar 2 actually. Not sure how widespread it is otherwise. https://www.pixelworks.com/en/truecut
60FPS animation as the default is superior to the choppyness we have nowadays. But the industry has to adjust their process, and upscale animation may never look right.
I think having good interpolation technology *in the hands of the artists* is awesome: You can make buttery smooth animation for a much lower cost when you want. But you get to keep your full creative control over the timing and framing whenever you want it.
The timing of motion and the importance of how long a frame is held for is important to how characters are portrayed and the impact of actions.
Take a punch or a bounce. The time of contact may be one drawn frame but it may be on screen longer to sell it. What should interpolation do here? If if it smooths everything but the impact, then the impact will feel too sticky and lose its force. Similarly if it smooths the impact as well then it goes by too quick and it feels too weak.
Animation is full of these choices and it’s a common problem in video games for example where animators animate at a given frame rate but games can run at variable sampling rates along their animation, causing unwanted artifacts of motion or ruin the performance.
https://www.youtube.com/watch?v=IK-Q3EcTnTA
https://www.youtube.com/watch?v=unRdyQYwL5k
If you pause the video and step through one frame at a time, it's very clear which frames are from the original film and which frames were conjured by the algorithm. There's no AI involved, so the algorithm doesn't really understand background from foreground, and objects in the artificial frames are distorted and unnatural. But at 100 fps you don't notice this, or at least I didn't, and motion seemed very smooth and fluid.
The soap opera effect definitely held. It didn't feel like watching a classic movie from 1982, it felt much more like watching a contemporary reality TV program. There was definitely a sense of realism, but the realism was not that you're witnessing an actual post-apocalyptic society clinging to survival. It was realistic in the sense that you felt like you're on a film set, and if you could just peek outside the frame you'd see George Miller and the camera and microphone operators with the lighting and rigging and make-up and costume departments hanging around the catering truck.
A lot of people felt that HFR ruined Peter Jackson's The Hobbit series (or was one of the many contributing factors) because it broke the pretense of fantasy. In my experience raising the frame rate did the same for The Road Warrior. Those outlandish costumes looked even sillier at 100 fps, but man those stunts are incredible! You do get a visceral sense of just how dangerous a lot of those stunts are. It's a miracle that nobody was killed.
https://film-net.github.io/static/images/000204/interpolated...
From an ML perspective, I would have assumed they would train on high frame rate video, but it seems they didn't, they're estimating motion of pixel clumps. I don't see how it can ever do things like eyes/smiles/teeth. Useful for reconstructed pseudo slideshows in Google Photos with added vampire effect.
https://docs.nvidia.com/video-technologies/optical-flow-sdk/...
if nvfruc is better quality + faster then the comparison is that nvfruc is better.
I've seen 2-3 other examples of insane name duplication in CV research in the last couple of months.