Buttery Smooth “10fps” (2020)
wunkolo.github.io
wunkolo.github.io
I have another hypothesis on "smoothness" of animated images. Most of computer-generated animations are unnatural and not cinematic because the objects jump between frames instantly while in real life and on a camera film they move smoothly. The hypothesis is that if the animation was closer to real life, then it would look really smooth even with lower fps, for example, 24 fps or 30 fps.
For example, when moving a mouse cursor, it jumps between points every 16 ms instead of moving smoothly. You could argue that a human is unable to notice such tiny details, but here is a simple test to find out: take a mouse and move it in a circle, so that it makes 2-3 turns per second. If a real object would move with such speed, all you would see would be a blurred circle (you can take any real object and see for yourself), however with a mouse I can clearly see a circle made of about 10-15 mouse cursors. So the animation of mouse cursor is unrealistic, computer-like and it can be easily seen (I noticed this effect long time ago but initially thought that it is a problem with Linux or slow graphics card).
So to make a smooth animation you could use oversampling: generate an animation at say 6000 fps, and blend every 200 frames together (remember about non-linearity of RGB values when mixing colors). If by chance anyone has some free time and curiosity, could you test it?
The former is often hated because it causes motion sickness, but it can help for rapid camera moves on low framerates. The later is generally just pretty good looking.
Those techniques have actually been around for a long time in traditional, handrawn animation.
https://www.traditionalanimation.com/2017/smear-speed-motion...
There are two kinds of choppiness that happen in recorded video:
1. The amount of time/motion between captured frames (framerate).
2. The amount of time/motion captured by each frame (shutter speed).
You are right that the instantaneous shutter used in rendering maximizes the choppiness of a given framerate. The solution you described is a feature in most rendering software called "motion blur". If you want perfect motion blur, you have to render enough intermediary frames to cover the motion in a scene across every pixel it moves across.
There are a few other less obvious places shutter speed choppiness comes into play:
1. Cameras. Some digital cameras have a physical shutter that opens and closes, and all of them have a speed at which light is captured by their sensors. When using a shutter to record video, there is a non-zero amount of time that is skipped in recording.
2. Lighting. Most lights have a framerate that matches the local standard alternating current rate: 60hz in the US, and 50hz in Europe. This is why NTSC is 60fps and PAL is 50fps. Different types of lighting fixtures have varying amounts of flicker, because they don't keep the same amount of brightness over each alternation of current. Florescent lighting is well known for (often) having a noticable flicker. You can see the effect of flickering lights by waving your hand in front of you. If it looks like a blurred motion, your lights aren't flickering. If it looks like you have many transparent fingers staying in place, then you have flickering lights.
Because resolution has been steadily increasing in popular media, but framerate has not, most filmmakers have been using faster shutter speeds to compensate. If you have motion across 10% of the screen over 1 second at 30fps, then during each frame, that motion will cross 1280px*10%/30fps = 4.266px of a 720p video, or 3840px*10%/30fps = 12.8px of a 4K video. A slower shutter will give you motion blur across those pixels, yeilding an effective resolution close to 1/4.266 of the 720p video (168.75p) or 1/12.8 of the 4K/2160p video (still 168.75p). A faster shutter would remove the blur, replacing it with an uncanny teleportation: the same that makes your fingers look copied and transparent when you wave them in front of a CRT monitor.
In order to get a smooth and clear/focused motion over 1 second across 10% of a 720p video, you must have a framerate of at least 128fps. 384fps for 4K. Now consider that most motion we capture in film is faster than that; some of it much faster.
What you suggest trying is actually possible with quite some games/engines. Some other commenter mentioned the source engine, I did it once with trackmania a good decade ago. It does very much amplify the feeling of speed.
It’s more or less dedicated to that issue
Another thing that can help is black frame insertion (thus removing the sudden “jump”)
Only if you limit yourself to displaying the gif in an <img /> tag. You can parse the frames yourself and render them with a <canvas> at a faster rate using browser's requestAnimationFrame (which sync's to the refresh rate of your display.)
The only advantage of gif I know of is that the creator of the asset has some control of the playback on the presentation platform. Other than this, it is a technically sub-optimal format. But no other formats support the creator control aspect.
You might want to display media supplied by users.
Convert a video of black video to mp4 and gif, both will be lossless.
Whilst some gifs are not compressed it appears to be niche for them to not have been. Or for the creator of the gif to have not constrained themselves during the creation process.
Perhaps important for NFT creators. Whilst not overly concerning themselves with the resources involved, they shouldn't be troubled by file size either.
You can have a magical decoder that circumvents this, but that's no longer a gif.
"1 second long" is pretty misleading, especially when telling people to look for different amounts of time to render. None of the gifs from 8-30fps are properly trying to be a second long.
If you have software that assigns the same delay to each frame, that's a flaw you need to compensate for or switch to different software, and the author did neither.
Let's use the 30fps gif as an example. It runs 11% too fast. Either it should be rendered with 33 frames, making it run 1% too fast, or it should use frame timings of 30ms 30ms 40ms 30ms 30ms 40ms making it exactly 1 second total.
Constant 30ms will jitter all by itself, and if anything 30 30 40 should reduce jitter or even eliminate it.
It's almost perfect except for one almost deal breaker thing (which applies to both Giscus and utterance.es): to comment you have to click through a very scary permissions dialog that claims it can "Act on your behalf" in your Github account, without any qualification. I'm pretty sure the permissions dialog is misleading, but it sucks. I expect that it really cuts down on the number of people willing to leave a comment. And I doubt it will be fixed because Github probably doesn't really care to support this use of their API.
Since it’s powered by GitHub discussions, and the discussion itself needs to be public, anyone wanting to comment but not wanting to authorise Giscus to act on their behalf could go to the discussion on GitHub and leave the comment there directly I assume
What should the "act on your behalf" mean? Is it safe to provide my readers Giscus?
Did you know pretty much all modern browsers support <marquee>?
But another one I'm less happy with is removing choice from the user. I hate auto-playing videos so I have that turned off, but this does not apply to gifs. And I don't mind auto-playing videos either if they take the place of reaction gifs.
They're _supposed_ to be 1-second-long loops, but the 60 FPS one is rounded to 50 FPS on export, so it takes longer to play and desyncs.
Gif can't do smooth 60fps even if it could do 1-delay frames. They chose centiseconds for some reason I've never understood, when milliseconds would have worked fine and avoided this issue. If your file shouldn't just be a video anyway, then the only fix is to use apng and demand sites support apng.
That this ancient format is still as prevalent as it is on the modern web is somewhat dumbfounding and probably unintended by its original creators.
Milliseconds would have allowed more than a minute for a frame. Centiseconds only benefit you when you want a frame time between 1 minute and 10 minutes, so basically never.
> vii) Delay Time - If not 0, this field specifies the number of hundredths (1/100) of a second to wait before continuing with the processing of the Data Stream. The clock starts ticking immediately after the graphic is rendered. This field may be used in conjunction with the User Input Flag field.
Additional info: https://en.wikipedia.org/wiki/GIF#Animated_GIF
This doesn't really answer your question, but it's all I could find so far.
Incidentally, that same GIF uses the text extension, something basically no modern thing that can show GIFs supports anymore.
There's a flag in GIF89a that allows the viewer to skip ahead to the next frame on user input. No modern GIF viewer supports this anymore either. BOB_89A.GIF used this feature to let you skip ahead if you didn't want to just passively watch the slide show.
Browser vendors didn't provide any alternatives to animation that just work until very recently. It's not dumbfounding when you take that into consideration.
For some reason GIF support was grandfathered in everywhere yet APNG, ability to just upload or embed a video and (god forbid because photoshop still can't open it) WEBP all took decades to be ubiquitous.
The 24/25 fps gifs certainly have the opportunity to look great (movies are mostly at 24 fps), but the hard-edge rendering of the black circle makes 'em look real choppy. I'd bet that rendering it at 50 fps, then averaging pairs of frames together and displaying it at 25 fps would yield something that seemed much smoother.
With some care, I bet one can go sub-25 fps on a gif and still have something that feels kinda "smooth".
The smoothness of video depends on the angular distance objects move in your field of view between frames. You can see this in the demos: the lower framerates are relatively smooth at the ends of the animation where the motion is slowest, and all framerates (even the highest) become considerably smoother when you decrease your browser zoom.
After all all we see is an illusion anyway (has to be interpreted by low energy consuming brain)! They can probably distract us from choppyness.
This is most noticeable when a camera pans horizontally along a very close vertical object very quickly (~1s for a full screen pan). This will give you more than a meter per frame on a movie screen and is very jarring (to me).
60fps YouTube, Twitch, etc.. content looks pretty great, though. Shame Hollywood won't try again.
Ang Lee has done a couple of HFR films since, but neither was a box office success. I’ve read that Avatar: The Way of Water (which Cameron previously said would not be released in HFR) will be released in 48fps, which would AFAIK be the first big-budget HFR release since the Hobbit trilogy.
Far more here: https://kevinlisota.photography/2020/04/understanding-video-...
I can only imagine errors to show up by "doing it wrong", that is, aliasing artifacts due to not lowpass filtering the signal beforehand.
That said, I'm not totally sure that a 50FPS signal is acutally capped to 25Hz since a 50FPS GIF is usually not interpreted as a signal sampled at 50Hz but as a piecewise-constant signal with 50 "steps" per second. So the first question is whether these "jumpiness" is an inherent, desired property of GIFs or an artifact of bad renderers. Can somebody clarify this?
Based on this "divide by 2" confusion and your mention of lowpass filtering, I'm thinking that the confusion is that you're thinking of this as audio or other wave-based signals. With audio the "volume" (amplitude) is a relatively minor detail and the key thing-of-interest is the frequency at which that amplitude is oscillating; not so with video. The signal doesn't follow a wave-pattern and we don't (normally) care about the frequency one bit. To apply video-numbers to audio; to represent a 60Hz tone, you'd need to represent that the amplitude goes down and back up 60 times per second; a 60Hz tone for 1 second is 120 "discrete" events, 60 "down" and 60 "up". If the samples are only at 60Hz then it would seem that the amplitude is staying constant, you wouldn't be able to observe that it's oscillating so quickly. And that oscillation is how we perceive audio. But it's not how we perceive video, light amplitudes don't normally oscillate like that, they stay steady and when they do change we notice that as a discrete event, not as part of a frequency. In fact, many displays (especially CRTs) do flicker, and are specifically counting on us not being able to notice this oscillation.
If the video signal emits a new frame at 50Hz but the screen updates at 60Hz, naively that means that 80% of the video's frames will be on-screen for .0166s, and 20% of them will be on-screen for .033s. While humans might not notice a .0166s difference on its own, we're pretty good at noticing changes in rate and will notice the jitter.
You could argue that similar to a lowpass filter for audio, that the display should interpolate between its samples to hide this jitter. It could; some TVs do this, but computer monitors don't because this necessarily introduces a delay of at least one frame (but realistically the software to do this will introduce more delay than that), and for interactive use we sure do care about lag/latency; even more than we care about jitter. TVs that do this will often have a "video game mode" that disables this and other features that introduce latency.
The biggest issue with an FPS change is movement. To keep speeds consistent, which the human eye is very sensitive to, you need to calculate a continuous velocity for every piece of the image and then interpolate the position of every object separately. And then you have to fill in all the gaps that leaves.
If you don't want to throw tons of CPU and guesswork at the problem, there isn't a solution to changing the framerate. Instead get a screen with a framerate that is either variable or 200+ or both.
Only if you bandwidth limit the output. In this case a frame blending video shader would do the trick.
Another way to put this is that a GIF is already a sampled function. For example imagine intensity of each pixel in a particular GIF is a function sin(x * t). Nyquist just says you can only accurately reconstruct this for 25px wide GIFs at 50FPS without aliasing.
And as you pointed, you'd still have to bandlimit the output.
Modern TVs do some clever work to detect 24 fps content in a 60hz signal, extract it, and present it properly to the panel. This takes a fair amount of processing power, and it isn't really feasible on computing platforms where the video content is not always presented full screen.
"With a 50 FPS GIF played back on a 60 FPS screen, some frames get shown twice"
Frames shown twice is an aliasing artifact that happens without a lowpass filter. It's exactly the same as spatial aliasing when you show a 50px image on a 60px area, "some pixels get drawn twice". Anti-aliasing gets rid of that.
"Do you have a plan to transform a video into frequencies and back so you can do a lowpass filter?"
Well, I (somewhat naively) assumed that this works the same as converting a still image into frequencies: Fourier transform. Only for an animated image the signal is 3d, not 2d.
edit: Filters normally don't do a FT explicitly. You use FT for the theory, but filters operate directly on the signal.
"Like the sibling comment, I wasn't sure what you mean when you write "50 FPS (max 25Hz)". "Hz" is literally "per second"; "50 frames per second" is "50 frame-Hz". Because we're talking about frame-oriented video, the "frame" is implied; FPS and Hz are equivalent units."
I was very sloppy with this. by 50 FPS (max 25 Hz) I mean that 50 FPS is equivalent to a sample rate of 50 Hz, which can only capture signals of less than 25 Hz. FPS and Hz are only equivalent when they refer to sample rate because FPS can't really refer to anything else.
> I'm thinking that the confusion is that you're thinking of this as audio or other wave-based signals
Doesn't most lossy image compression make the same assumption, e.g. JPEG?
> but computer monitors don't because this necessarily introduces a delay of at least one frame
I agree with this one, at least AFAIK you can't really lowpass-filter a realtime signal without adding delay.
A GIF is already a sampled signal.
Yes, you can LP-filter the GIF but that's no longer reproducing the original signal, which wasn't band-limited to begin with.
You're right that, if the original signal that the GIF samples is band-limited to 1/2 the Nyquist rate (which is a big "if") it can be accurately reproduced without aliasing by passing it through a reconstruction filter.
This brings me back to this question:
> So the first question is whether these "jumpiness" is an inherent, desired property of GIFs or an artifact of bad renderers.
Like for pixel-y displays, I assume that this was never well-defined for the GIF format, so what is basically a property of the first GIF renderers became a de-facto property of the format.
I'd argue that GIFs that exploit this property and rely on the "jumpiness" to look good are not band-limited, require a "jumpy" renderer and will suffer from the 50Hz/60Hz problem. On the other hand, GIFs that don't exploit this property are meant to be continuous and are just sampled at a low frequency, and for those re-sampling in time will allow to fix the 50Hz/60Hz problem.
Though another problem will turn up then: when the latter were sampled at a low frequency, they were likely not lowpass-filtered beforehand, so they already contain aliasing artifacts you cannot get rid of anymore.
> Doesn't most lossy image compression make the same assumption, e.g. JPEG?
Yes, but the key difference is that there's no temporal component; the wave is across the screen, not across time. When one speaks of a low-pass filter in image processing, it's generally operating on the horizontal- and vertical-axes, not the time-axis.
To put "not wave-based" in to other words: Unlike audio, video across time is neither continuous nor band-limited.
1: Frame judder, as a result of some frames from the 50hz content being displayed for more refresh cycles than others.
2: Image tearing, which is caused by the content being updated in the middle of the 60hz refresh cycle.
Ideally you want the framerate of your content to be an even factor of the display's refresh rate. So for a 60hz display, that means you want the framerate of your content to be 60, 30, 20, 15, etc.
Also a huge pain for old games. Lots of crummy conversions that straight up just run 17% slower on the PAL version.
I can think of one, which is avoiding Twitter video compression but that's kind of a weird case.
When I was working at apple we used a form of this technique for the mobile versions of product pages for that reason - https://www.sublimetext.com/~jps/animated_gifs_the_hard_way....
Another one is that gifs are much easier to share, save, and reuse; with gifs, it's right-click and save, with videos... I don't even know where to begin.
The world would be a much better place if Internet Explorer had used Gecko :)
Maybe someone out there can clear this up for me and convince me why we need 120+ fps displays?
EDIT: Well, nevermind, a quick google search corrected my misconception about 30fps.
I think the problem is actually the opposite. In real life if you follow a moving object with your eyes then you expect its image to stay at a consistent place on your retina with no or minimal motion blur. If you follow a moving object on a typical LCD screen with your eyes, then for 30fps the object stays still for 1/30 s on the screen, while your eyeballs are still moving, causing motion blur on your retina.
Telling 24, 30, 60 apart is very trivial.
As for 120fps, you need to keep in mind that playing a real-time rendered computer game is a totally different beast than playing back a steady video. There are lots of other factors involved. Some headroom in FPS makes lots of difference due to various other things like jittering (judder?), input delays, response time etc. (also it's possible some people simply can tell 60/120 apart, even by just watching a video.)
It is possible to improve on this without increasing the framerate with strobing the backlight/black frame insertion.