Motion blur all the way down (2022)
osar.fr
osar.fr
What's interesting about rendered content is that it can extend that (e.g., a shutter angle beyond the duration of a frame), playing with something we thought we had a handle on.
https://cinemashock.org/2012/07/30/45-degree-shutter-in-savi...
Only objects not moving relative to the frame can be seen sharply.
Trivial. Drag your white cursor quickly across the screen against a black background. You will see clear gaps between the individual cursor afterimages on your retina.
Double the FPS, halve the size of the gaps. On a 240Hz monitor I can so clearly see gaps such that if they were halved the gaps would still easily be visible. Ergo 400Hz would still easily be distinguishable from continuous motion.
To put numbers on this, consider a vertical line 1 pixel wide moving across a 4K screen in 1 second (that's not even that fast). At 480Hz that's a shift of 8 pixels per frame. So you'd need at least 8x the framerate for motion of this line to be continuous.
That's the outcome of aliasing, not of the FPS limitation itself. You could analytically supersample the motion in the time domain and then blur it just enough to remove the aliasing, and the distinct images would then disappear. Motion blur approximates the same result.
I found most of the information on Wikipedia[0], and the limit seems to be at about 80hz, but together with movement, some people can see stroboscopic effects up to 10khz.
[0] https://en.wikipedia.org/wiki/Flicker_fusion_threshold#Strob...
Hmm, this is not accurate (or I don't understand what you mean). 100Hz CRT TVs available in the 90s/00s did not interpolate frames to get smoother motion – they only existed to reduce flicker. I think such TVs also existed in NTSC markets (120Hz)?
Anyway, ever since the late 00s, pretty much all the TVs you can buy from a store do come with an interpolation algorithm to artificially display a higher frame rate image (e.g. 100Hz/120Hz) from a lower frame rate source (e.g. 23.976/24/29.97/30/50/59.94/60 fps) – which (personal opinion) looks terrible (and can be turned off from the settings – but the default is always on). This is an interesting side tangent when it comes to motion blur, because the blur is prebaked in the input signal and cannot be easily removed. Thus, the end result always has an artificial look.
For instance, if the source material is shot in 24fps with the typical 180 degree shutter angle, each frame spans 41.6ms of which the shutter was open for 20.8ms. Then your TV interpolates that to be 96Hz or whatever. However, the individual output frames still look (or, with the added artifacts etc., mostly look) like the shutter was open for 20.8ms per frame. However, each frame now spans 10.2ms which is a shorter time than the shutter speed!
By the way, this is also the reason why CRT and Plasma screens had much better motion clarity than LCD or OLED. The former flash each frame for a short time, while the latter "sample and hold" the frame for the entire frame time (e.g. 1/60th of a second for 60 Hz). 60 FPS on a CRT looks probably more fluid than 120 FPS on an OLED.
Another option for games is to indeed add a lot of frames by using reprojection techniques. This can approximate the real camera movements without needing to render a ton of expensive frames in the engine. This also is already used in VR, just currently not at overly high frame rates. This great article goes into more detail:
https://blurbusters.com/frame-generation-essentials-interpol...
Something like 1000 FPS with reprojection are apparently quite realistic, which should solve the problem of tracking blur without reducing screen brightness.
Is a 1000 FPS screen more realistic than a screen capable of the higher maximum brightness needed to compensate for black frame insertion though? HDR screens are already a thing and you could already gain persistence improvements there for LDR scenes without needing any new hardware by always driving pixels at max brighness but only for a reduced time depending on the target luminance.
Or just reduce the ambient light enough - I even run even my LDR monitor at 10% brightness.
One advantage of higher frame rates (as opposed to strobing) would be quicker input responses to, e.g., moving the camera. That's not overly important on a normal screen but quite significant on a VR headset where we expect head movements to be represented very quickly.
I think interpolation up to several kilohertz is the best solution, preferable starting from a moderately high frame rate (e.g. 200fps) to minimize the latency penalty and artifacts.
[0] https://en.wikipedia.org/wiki/Smooth_pursuit
[1] https://en.wikipedia.org/wiki/Flicker_fusion_threshold#Visua...
When I snap my head (or eyes) around, I don't see a blurry image, I see the new image, and my brain drops the intermediate data. You can test this by looking at yourself in a mirror, focusing on one eye, then another. Do you see your eyes or face or anything blur?
Adding blur just delays the presentation of the new view when moving your POV in games. It's distracting and unrealistic.
This isn't the best test since your brain compensates for saccades by censoring input for a while. And it has built-in motion stabilization which allows you to read a sign while walking down the street.
A better test would be to look at your finger or hand while waving it very quickly. You'll see motion blur as your finger moves. In certain cases, you can see individual "frames" as an afterimage. E.g, if you look at a modern car tail light, the LED isn't on consistently but uses PWM to blink really quickly. So if you move your eyes while looking at a tail light at night, you'll see a series of dots rather than a blurry image. Once you learn this trick, you can distinguish between analog lights and PWMs by noticing if they are blurry or discrete.
For fast moving things outside of ego motion, I can see a reason to use blur (e.g., your hand waving).
Indeed, it's almost impossible to turn your head slowly without your eyes fixing on points in your field of view, and then saccading to the next point. Try it. This is a hard-wired response. (You can perform some tricks to do it, like completely blurring your eyes, but that's not really the point.)
Meanwhile, if we watch a video with the camera turning, it's easy to keep our eyes staring straight ahead.
This shows that there is going to be a difference between a head naturally turning and watching a video of a head turning.
I think you're describing saccades as just being the "dropping out" of frames, and not including the extremely-relevant motion that our eyes do during a saccade.
I think the only thing I can say here is - I dont see anything weird when I turn my head, and when FPS games added blur to "character moves his own body", it felt much less natural, not more. So, to answer question a few posts up "how would we do it?" I'd just say "The way we always have before we got enough GPU to make everything look like a movie"
Actually I think motion blur is more of a (futile) attempt to hide low frame-rates because we don't have enough GPU to render the ever more demanding scenes.
You say it yourself: "your brain compensates for saccades by censoring input for a while. And it has built-in motion stabilization which allows you to read a sign while walking down the street."
On the rare occasion I've been able to play a video game at 120 fps—my projector is limited to 60hz, sadly—I've found that I prefer to play without motion blur.
The biggest three crimes are blurring an object too far, blurring things that shouldn't be blurred, and blurring an entire scene (which I guess just results in #2, so maybe there's really only two crimes). The most important thing is that motion blur should be subtle. Properly-done motion blur doesn't make a game look blurry, it makes it look more real and smooth.
If an object is moving 50 pixels between frames, then the blur should not be more than 50 pixels wide. Really, it should likely be only 25 pixels to make the blur more subtle. But for some reason, racing games love to apply a full radial blur to an entire scene while going fast, and to me it just ruins it.
Likewise, an object that is not moving relative to the camera should have no motion blur at all. This is where so many games get it wrong. As you rotate, they blur the scene using post-processing. In a completely static scene, this makes sense and is a very computationally-efficient way to perform motion blur. But if you're rotating because you're tracking an object, the tracked object should not be blurred. If you're in a racing game and the car next to you is going the same speed as you, the other car should not be blurred.
Really, the only guaranteed way to get correct motion blur is to render multiple entire frames in a row and then blend them, but you need a LOT of samples to make this look good, or else something like a vertical line looks like a series of vertical bands as it flies across the screen, rather than a smooth blur.
The best is probably a hybrid approach. Render each object in the scene on its own and apply a post-process blur on a per-object basis based on which direction it is moving relative to the camera. Though Z-ordering could end up becoming a major challenge.
No, it doesn't make sense because due to eye movement that's not how turning looks in the real world. Any game that blurs the scene on camera rotation is doing it wrong even if nothing else is moving at the time.
It's impossible to do it correctly without very low latency and high precision eye tracking so the best you can do is to not blur anything that the player might be focusing on.
It's photorealism, not realism.
Curiously, before the classic paper on modeling shutter efficiency [2] came out in 2005, all renderers used in VFX production used box shutters. I.e. the shutter opens instantly, stays open for the specified duration and then closes instantly.
When you watch movies like "Jurassic Park" or "The Mask" and see scenes with extreme motion blur, that's PhotoRealistic RenderMan with a box shutter.
The first in-the-wild 1:1 implementation of the parameterization in [2] was done in [3] in the same year the paper came out (hasn't changed until today). It was first used on "Charlotte's Web" (only the spider character done by Rising Sun Pictures has it as they used [3] for it).
Pixar added it a few years later too and went a tad overboard with theirs in [4]. Most offline renderers nowadays have this feature and call it a "shutter curve".
[1] O. Navarro et al.: Motion Blur Rendering: State of the Art (https://citeseerx.ist.psu.edu/doc_view/pid/fc23fb525cafa8fe6...)
[2] Stephenson, Ian: Improving Motion Blur: Shutter Efficiency and Temporal Sampling (https://staffprofiles.bournemouth.ac.uk/display/journal-arti...)
[3] https://www.3delight.com/, see specifically https://nsi.readthedocs.io/en/latest/nodes.html
[4] https://renderman.pixar.com/resources/RenderMan_20/cameramod...
(unless some non-linear effects in human visual perception cancel all that out, but it should at least be mentioned if so)
I highly recommend 240 Hz if only for the smoothness and low latency of mouse movements. A 60 Hz mouse pointer is really, really hard to go back to.
I can tell the difference between a 30 and 60fps game, but only if I pay attention to it. As long as it's steady and not bouncing between the two I just don't care.
Microstutters also really annoy me. But it's too much to ask for if a lot of things still force 30fps scroll on me (for example AMD's own Radeon software).
I wonder, though, if maybe an interface with smoother transitions is more soothing and maybe having a higher refresh rate does affect one's mental state over time...
However the light emitted by the moving object should be lower than the static object. So as the distance increases, the object should become darker and darker.
"Infinitely fast" being a stand-in for "sufficiently fast" and gated by the speed of light, of course.
If you're going to make up nonsense, you're not talking about physical reality you're talking about cartoons or magic and fantasy.
When something moves fast in photography it has motion blur and that spreads its coverage over a larger area. In this demo the motion blurred 'torus' and 'sphere' both get made artificially more opaque to make the animation work. You can see it happen.
Reproduction of reality is not the goal, because it's unachievable.
Larry Ellison used to have a TV projector in his house, with the light output for a drive-in movie theater but aimed at a small screen, so he could watch movies in broad daylight. That was before everyone got bright screens.
Also, simulating realistic processes is not incompatible with tonemapping the result to be able to display it on limited screens.
I think I still like the first half, it's a reasonable dive into the detail of what motion blur is, or should be in theory. The second half is a slightly crazy, extremely condensed explanation of how the shader works for this particular "torusphere" animation based on motion blur. I find that part mainly useful because otherwise the code would be impenetrable at this point, at least to me. In retrospect the transition between the 2 parts feels a bit like jumping into a frozen lake, sorry about that!
I've played with motion blur as a function of projection of 4D extrusions a few times, but the practical context (video, textures) ends up leaving it an impractical approach when compared to sampling in hardware with caching. This write up leaves me thinking "maybe just one more time".
Fast analytical motion blur with transparency: https://www.sciencedirect.com/science/article/pii/S009784932...
http://www.softdorothy.com/Blog/Entries/2011/10/26_Making_Th...
What gamers really dont like is when you move the camera slightly and the entire scene turns into a deep fried oil painting.
TLDR; motion blur always makes sense atm. Not less so for games.
For games the difference it makes depends on the FPS and the distance an object moves within the FOV.
Estimates say humans can see max. 60FPS. Even if your game runs at 120FPS and something moves accross the entire FOV -- that is two discrete samples of that object at most. You will get strobing.
Only if you render the geometry with motion blur will this look natural.
The alternative is probably to have mutliple 10k FPS but that is wasteful.
If your biological limit is around 60FPS, it should be cheaper to generate motion-blurred frames at this rate (where more samples are used in areas with more motion blur) than rendering every pixel of a frame at muliple 10k FPS.
Estimates say humans can see max. 60FPS.
This is a ridiculous statement. You know that's wrong if you've ever switched your phone between 60 and 120hz. Even without personal experience, most HN readers would be aware that, for example, when the Oculus Rift folks were developing their headset they found 90hz to be the minimum to avoid visual disturbances and nausea in an immersive environment.
However, there is a threshold on how much information your brain can process and tests indicate this maxes out at the temporal detail you can fit into 60 'discrete' frames per sec. For many people it is actually much less.
That is: in general.
Specifically this threshold is differs based on certain variables; the most important one being available light (number of photons per unit of time).
I.e. the less light, the more motion blur there is because there is more temporal integration happening in your brain's visual system. Estimates say your brain goes down to a temporal information density of less than 10 FPS in very low light circumstances [1].
Your brain does perceive motion blurred frames differently. How do I know? I worked in VFX on many blockbuster movies. We do preview stuff using realtime graphics (aka what game engines do) w/o motion blur at equal frame rates as the final frames (which are with motion blur, ofc).
Even on projects where we used 60 fps there is visible strobing w/o motion blur. And as I said: even when you preview stuff at 120 fps or more there is visible strobing with geometry that covers big distances in your FOV.
Motion from game engines w/o motion blur does not least look like game engines because of this. There is also a night and day difference between post-process motion blur, based on 'smearing' the image based on a motion vector pass and true 3D motion blur.
"The Hobbit" triology was done at 60FPS. But it still had motion blur, for obvious reasons.
[1] https://www.sciencedirect.com/science/article/pii/S004269890...
EDIT: typos/grammar
And as I said: even when you preview stuff at 120 fps or more there is visible strobing with geometry that covers big distances in your FOV.
Exactly. The game has no idea what is and isn't moving in my FoV because it doesn't know what I'm looking at, only what the camera is looking at. Subjectively, in my experience, games do a terrible job at predicting what I'm looking at and I'd prefer they stop messing up my experience by trying. Strobing is better than smeary mess.
True, but it doesn't really change the point.
TLDR; gamers hating motion blur are not hating motion blur. They hate the way its implemented. It's far from the real thing.
The actual misunderstand here is that you are comparing apples to oranges. Motion blur for games is just as good and well-placed as it is for movies.
But only if it is actual true 3D motion blur that mirrors what happens in the real world (as in the parent article of this discussion).
In practice almost all games only blur the camera which blurs everything which leads to gamers then complaining about motion blur (and some even getting motion sickness/headaches).
Simple example: an object moves left accross the frame and the player tracks it while they're strafing fast to the right. Real motion blur means the object has almost zero blur since the player is tracking it (keeping it approx. in his FOV's center) while the BG is heavily blurred because of the strafing move the player is making.
When you only have camera blur, everything, including the tracked object is heavily blurred. That's what most games do and what gamers rightfully hate. But it is not the motion blur you see in film. Some games seem to do better now, I think this is called 'per pixel motion blur' in the games context (but I may be wrong).
> Exactly. The game has no idea what is and isn't moving in my FoV because it doesn't know what I'm looking at [...]
That is valid point if you had true motion blur and wanted to make this even better in a gaming context. See above.
I used to own and play around the FOVE VR headset (their SDK was absolutely terrible at the time and Windows-only, unfortunately, I sold it to a uni eventually) [1].
It had eye tracking. And that is indeed what you'd need to do the motion blur "better" for games where a player tracks fast moving objects accross the frame. Because you will have a slight discrepancy between the tracking of the eyes and the tracking of the look-at direction from motor latency and under-/overshoot.
I.e. calculate the difference in movement of eyes against final transform of geometry (including camera) to calculate how much motion blur you actually need.
For lack of eye-tracking hardware: I wonder if the game could guess what you're looking at and compensate the look-at vector with that to simulate fake eye tracking and get motion blur that is better received by some people.
That is the theory. In practice most games just do the "motion blur" wrong for starters. See above.
If that screen had e.g. 12,000fps, you would see the mouse cursor as a motion-blurred streak, not individial images of the cursor next to each other.
In real life, eyes track objects in order to minimise blur.
But realtime motion-blur implementations just blur everything that moves relative to the camera, which is not correct, and makes the human looking at the screen lose track of the objects they were looking at.
If we had pervasive perfect low latency eye tracking then a game could pull off something that actually looks plausible, but that is not the case, so the only games that can get away with it are the ones that are largely a movie and can take advantage of the motion blur cinematographically.