AAA – Analytical Anti-Aliasing
blog.frost.kiwi
blog.frost.kiwi
A big customer was furious that they had bought a part that didn't perform the way they wanted, so I was voluntold to fix it.
I was given a ridiculous timeframe to come up with a solution and present them to our customer in a big in-person meeting with all the decision makers. I managed to implement three different alternatives so that the customer would feel they had some agency selecting the one they liked the most. The best looking by far was a form of AAA.
This was one out of several of these big last minute fires I was assigned to solve. Years later my manager told me how great it was knowing that the could throw any crap at me and I would be able to fix it in time.
However, these sort of experiences are why I struggled with burnout during my career, which led me to save like crazy to retire as early as possible, which I did.
For younglings out there: when they ask you to do the impossible, remember that failure IS an option. Push back if you think they are asking you for something unreasonable.
Also, going to add "voluntold" to my lexicon :D
1. antialiasing should be done in linear rgb space instead of srgb space [1] [2]
2. because of the lack of (1) for decades, fonts have been tweaked to compensate, so sometimes srgb is better [3] [4]
Do you have advice on linear vs srgb space antialiasing?
[1] https://www.puredevsoftware.com/blog/2019/01/22/sub-pixel-ga...
[2] http://hikogui.org/2022/10/24/the-trouble-with-anti-aliasing...
[3] https://news.ycombinator.com/item?id=12023985
[4] http://hikogui.org/2022/10/24/the-trouble-with-anti-aliasing...
Unfortunately, this is completely context dependent. One central point is, whether or not the graphics pipeline is instructed to perform corrections (GL_FRAMEBUFFER_SRGB in OpenGL), as that changes the answer. Another point is, in which context blending is performed. Luckily the developer has full freedom here and can even specify separate blending for alpha and color [1], something that GPU accelerated terminal emulator Alacritty makes use of [2], though it doesn't do MSDF rendering.
One thing I can say though: The alpha, the fading of edge, has to be linear at the end or perceived as such. Or rather if the edge were to be stretched to 10 pixels, each pixel has to be a 0.1 alpha step. (If smoothstep is used, the alpha has to follow that curve at the end) Otherwise the Anti-Aliasing will be strongly diminished. This is something you can always verify at the end. Correct blending of colors is of course a headache and context specific.
> fonts have been tweaked to compensate, so sometimes srgb is better
This should not concern MSDF rendering. These tweaks happened at specific resolutions with monitors popular at that time. Especially when considering HiDPI modes of modern window systems, all bets are off, DPI scaling completely overthrows any of that. MSDF is size independent and the "tweaks" are mainly thickness adjustments, which MSDF has control over. So if the font doesn't match as it looks in another rendering type, MSDF can correct for it.
[1] https://developer.mozilla.org/en-US/docs/Web/API/WebGLRender...
[2] https://github.com/search?q=repo%3Aalacritty%2Falacritty+ble...
What WebGL2 is still missing is MSAA texture objects (it only supports MSAA render buffers), which makes it impossible to directly load individual samples in a shader (useful for custom-resolve render passes). That's only possible in WebGPU.
> Whole communities rally around fixing this, like the reddit communities “r/MotionClarity” or the lovingly titled “r/FuckTAA”, all with the understanding, that Anti-Aliasing should not come at the cost of clarity. FXAA creator Timothy Lottes mentioned, that this is solvable to some degree with adjustments to filtering, though even the most modern titles suffer from this.
I certainly agree that the current trend of relying on upscalers has gone too far and results in blurry and artifact riddled AAA game experiences for many. But after seeing this [1] deep dive by Digital foundry I find the arguments he makes quite compelling. There is a level of motion stability and clarity only tech like DLSS can achieve, even outperforming SSAA. So I've shifted my stance from TAA == blurry, TAA + ML when used right == best AA possible currently for 3D games.
Thoughts?
However, on a meta-level I find something like “r/FuckTAA” fundamentally entitled and ungrateful to the people who put years of their lives into making these games. Of course the loudest gamers tend to be smaller subgroup of the entitled toxic ones, so perception is distorted anyway. Plus, I do get it to some degree: if you invest a lot of money and time into powerful hardware to get beautiful graphics, you'd like to actually get beautiful graphics out of it.
Still, every time I read any article on the technical workings of modern graphics, it strikes me as a community full of extremely passionate people who care about squeezing beautiful graphics out of the available hardware. There are nicer ways to say "sorry but this particular technical solution/aesthetic trend doesn't vibe with me".
(Obviously this is not aimed at your nuanced take with a sincere question for discussion. And thank you for the link, will watch, because the tech still is an interesting topic to me even if I don't play these types of games)
What many gamers see in TAA and other anti-aliasing options is higher system requirements and worse image quality. Obviously, this isn't just caused by one factor but it's very noticeable if you compare games now to the games of yesteryear.
Here is a good collection of anti-aliasing methods. https://www.pcgamingwiki.com/wiki/Glossary:Anti-aliasing_(AA...
Modern graphics are generally less "accurate" in terms of rendering in exchange of way better visual effects like raytracing and better shading. There's a fairly linear progression of rendering quality throughout the history of 3D rendering. Games like Half-Life 2, Halo and the first CoDs had mostly exact forward rendering, the shading was fairly simple but the rendered image did not have spatial or temporal artifacts. Individual pixels could be discerned too. If you increased the resolution, you got a crisp image with crisp UI. These games usually used SSAA, MSAA or CSAA. These methods were very expensive because they effectively rendered at a higher resolution, they could have had double the shading cost. (I know that MSAA only does it for the edges but I'm keeping the explanation fairly simple)
Next, developers started to switch to framebuffer-based methods (FXAA and SMAA). These introduced blur to the edges of objects which was a downgrade in terms of image clarity. However, these methods are fairly fast and the games didn't exhibit motion-based artifacts. The shader is relatively fast and the only real cost is an extra framebuffer in memory - which the game already probably had for other post-processing effects.
Developers also started to use deferred rendering and more buffers in general. This led to an explosive growth of VRAM usage, so for optimisation purposes, they started using the console methods of checkerboard rendering and lower-resolution buffers. A good example of this is GTA V where you can see many dithering artifacts in the distance and a generally blurrier picture. These games still didn't have temporal artifacts - looking around did not generally reduce the image quality.
Graphical demands only got more intense, so developers started to "cheat" with forms of temporal anti-aliasing, which (as in the name) introduce temporal artifacts to the image. The stereotypical poster-child for this is Unreal Engine - it's seriously impressive in many aspects but developers just started to slap TAA on everything instead of optimising their shaders and LoDs. TAA is not really one single method - it can really vary in quality based on how many past frames are used, how artifacts are removed, etc.
A while ago, NVIDIA introduced DLSS and (real-time) Ray Tracing, which changed things quite a lot. With these techniques, not even the final framebuffer is at native resolution, but is instead being upscaled from a lower-resolution framebuffer using ML-based methods. Ray tracing also requires denoising because it's infeasible to raytrace every pixel in native resolution. It works surprisingly well, but there's another reduction in clarity. And of course, since the image is effectively being reconstructed (twice!), the temporal effects are horrible, even moving the cursor around can produce nauseating artifacts on more complex objects.
Currently, the gaming industry is at a stage where the massively better GPUs are not used to progressively enhance games but to cut corners and reduce the rendering quality for everyone. Games even use DLSS for their recommended system requirements now instead of being an optional enhancer for weaker GPUs. Currently, if you don't have a top-end GPU (4080 or 4090, basically), you have to make heavy compromises in the newest games in terms of image quality or use DLSS, even without RT. I predict that in the future, even top-end GPUs like the 5090 or 6090 will have to use DLSS to get playable framerates in 1440p with acceptable graphics, not just in 4K.
There is a reason why quite a few people are pissed off. Obviously, they might not know the exact causes of these problems but it's easy to see a correlation of TAA in game - bad rendering quality and choppy FPS, so they scapegoat that. I can't really blame them, I don't think it's realistic to expect gamers to be intimately familiar with the render pipeline.
I think the current wave of using temporal upscalers that take care of AA by design as a way to unlock free performance and or allow lazier development approaches is ill guided.
My bigger gripe with the state of the art of real-time 3D rendering is frametime inconsistency. UE5 is a particularly bad offender here. Counterpoint "Ratchet and Clank: Rift Apart" [2], it uses TAA, dynamic resolution, ray tracing and checkerboard rendering for certain passes and I find it looks stunning. No blurriness to be found here, and rock solid frametimes. If anything we are seeing the plague of developers chasing foto realism and Epic being their dealer.
I have done a few live visualization based blog posts, and they take me ages to do. I kind of think that's the right idea though. There is so much content out there, taking longer to produce less content at a higher quality benefits everyone.
[1] https://github.com/FrostKiwi/treasurechest/commits/main/post...
Also, forgot to mention that I really enjoyed the article, and I'm not even a gamer! You're totally right in that the graphics community is a lovely one to follow on a technical level, even if you don't play games. Articles like yours are part of the reason for that
Implemented the opening of links in new tabs. Turns out there was a package for that [1]. Will go to good use in up-coming articles.
[1] https://github.com/crookedneighbor/markdown-it-link-attribut...
For font and 2D vector rendering it's likely, in fact afaik, some solutions, such as Slug already do.
But for 3d rendering I don't know of any solutions.
For an intuition, consider two triangles that intersect the same pixel.
Consider if say one has 20% coverage and the other 30%, does that pixel have 50% coverage, 30% by one, 20% by one and 10% by another, or any other conceivable mix? It's very difficult to say without selecting specific points and sampling directly.
I think dreams might be using splats? It's possible that something like splats combined with a form of OIT along with an art style that is tolerant to certain types of artifacts could use Analytical AA. I don't know the specifics on Dreams, but i'd be surprised if even they would be using AAA.
Still, non-standard rendering approaches are very much a thing [1] and I could see setups like [2] be used in scientific particle visualizations.
[1] https://www.youtube.com/watch?v=9U0XVdvQwAI
[2] https://bgolus.medium.com/rendering-a-sphere-on-a-quad-13c92...
How to blend multiple shapes on the same quad, within a draw call is shown in the section "Drawing multiple?". There you fully control the intersection with the blending math.
Finally, there is always the gamma question, which is true for all AA solutions. This is not covered in the post, but that might mess with the results, as is true for any kind of blending.
[1] https://developer.mozilla.org/en-US/docs/Web/API/WebGLRender...
I think this is the same method Skia uses, and it doesn't work for rendering vectors from SWF files in my experience because of this problem.
Found it by going to the comments since the comments are GitHub issues the "x comment" is a link to the issues page.
On what GPUs and through what APIs did you see this? This seems fairly weird. I especially wouldn't expect Apple to have problems.
It's just so happens to produce visible artifacts in this case. I suppose for 3D scenes it's mostly fine.
I am sorry, but may I ask you, do you uphold your duties as a NeoTokyo fan, by promoting both the 2 best (un-)dead Source Mods, NeoTokyo and Dystopia?
https://store.steampowered.com/app/17580/Dystopia/ https://store.steampowered.com/app/244630/NEOTOKYO/
Sorry, was most likely funnier in my head then it really is, but I was committed to the bit once I had the idea. I was exited to see Neotokyo mention somewhere popular. NeoTokyo and Dystopia regulars compete in each others tournaments and are always looking forward to new players :)
Biggest Tournament of the year: - 5v5 matches of ~4-6 teams - broadcasted on twitch with - 1 camera person - 2-3 casters/commentators - definitely overproduced - to an audience of 15-25
Its so much fun ^.^
This article will probably help for future reference though!
On a meta level - I feel like I've seen anti-acronym sentiment a lot recently. I feel like it's never been easier to look these things up. There's definitely levels of acronyms which are anti-learning or a kind of protectionism, but to my mind there are appropriate levels of it to use because you have to label concepts at a useful level to accomplish things, and graphics settings of a game definitely are on the reasonable side.
And even if you know every detail, that's still the best course of action, I think. Which kind of antialiasing you prefer, and how it trades with performance and resolution is highly subjective, and it can be "none".
There are 3 components to rescaling/rendering pixels: aliasing, sharpness and locality. Aliasing is, well, aliasing, sharpness is the opposite of blurriness, and locality is about these "ringing" artefacts you often see in highly compressed images and videos. You can't be perfect on all three. Disabling antialiasing gives you the sharpest image with no ringing artefacts, but you get these ugly staircase effects. Typical antialiasing trades this for blurriness, in fact, FXAA is literally a (selective) blur, that's why some people don't like it. More advanced algorithms can give you both antialiasing and sharpness, but you will get these ringing artefacts. The best of course is to increase resolution until none of these effects become noticeable, but you need the hardware.
The best algorithms attempt to find a good looking balance between all these factors and performance, but "good looking" is subjective, that's why your best bet is to try for yourself. Or just keep the defaults, as it is likely to be set to what the majority of the people prefer.
I wouldn’t be surprised if there’s now a market demand for that to spread back to PC land.
PlayStation does have shining examples of user-friendly settings UI though, namely in their PC ports. Look at this UI in Ratchet and Clank:
https://x.com/digitalfoundry/status/1684221473819447298
Extensive tooltips for each option and any time you change a setting it is applied immediately to the paused game while you're still in the settings menu allowing you to immediately compare the effects.
It’s nowhere near as simple as 3 settings now there are different antialiasing techniques, path tracing lighting or reflections, upscaling (multiple algorithms) etc.
Nothing is all just fully positive and each has tradeoffs
The upscaling, frame generation, path tracing etc are not.
For the reason I mentioned, none of its objective you're selecting between trade offs of crispness, artifacts, better lighting and reflection and framerate.
It's no longer just resolution of rendering and resolution of textures and a few post effects.
Nowadays I'm a dad with practically no spare time between raising a toddler and work. Suddenly the question of "does the game overstay its welcome?" has become a thing; I almost exclusively play games that can be played in short bursts of time, and that deliver a great experience as a whole that can be completed in a relatively short playtime. I got a Steam Deck a few years ago for the specific purpose of separating my work computer from my gaming platform, and being able to pick up and play a game and pausing it without problems.
Even with the built-in performance overlay of the Steam Deck (which is very nice) it takes time to assess the quality-vs-performance results of every possible combination of settings. Often more than I would spend on playing a game.
I suspect that people like me either already are or soon will be the bigger segment of the (paying) customers though, so that is something to consider for developers.
And some games do give short explanations of what each type of technique does and how they compare, along with statements like "this usually has a small impact on performance" or "this has a large impact on performance" to guide me, which is already a great help.
The best game settings have a scene to illustrate the effect of choices along with estimated(?) performance. Unfortunately I haven't seen that too often (mostly Falcom games that PH3 worked on). I agree that stating the impact on performance is quite helpful when settings need to be lowered. Usually they are ordered "best performance" to "nicest looking" so I set the last one and only fiddle with them if necessary (a smooth 12fps works great for me so luckily that isn't often).
I've been using MSAAx4 in my rendering engine for some time and only recently have considered switching to a FXAA / TAA implementation. I'm actually not sure I'm going to go through with that now. I definitely learned a lot here, and will probably use the analytical approach for UI items, I hadn't heard about that anywhere.
Not often you see graphics-programming stuff on HN. For anyone interested in more graphics write-ups, this list of frame breakdowns is one of my favorite resources:
One of my favorites: https://acko.net/blog/how-to-fold-a-julia-fractal/. This helped me understand the relationship between trigonometric functions and complex numbers like nothing else I've ever seen.
And now today there’s this post and your comment here in the front page.
Which is another way of saying: a proof of God would be going through life without a constant stream of surprisingly meaningful coincidences.
Unbiased reality is inherently a blind but inventive "Creator".
SDF(or mSDF) isn't the future. It's already "good enough" classic.
> This works, but performance tanks hard, as we solve every > bezier curve segment per pixel
This is "the future" or even present as used in Slug and DirectWrite with great performance
https://sluglibrary.com/ https://learn.microsoft.com/en-us/windows/win32/directwrite/...
[0]: https://www.microsoft.com/en-us/research/wp-content/uploads/... [1]: https://patents.google.com/patent/US20070097123A1/en
Had to do a _lot_ of work to make it production-ready, as their voronoi-based tesselation goes pathological on a lot of Asian glyphs.
Though a little caveat from my side, as I have written both 2D and 3D rendering engines. Let me tell you, they could not be more different. It is not just another dimension but completely different goals, use-cases and expectations.
So instead of:
> Everything we talked about extends to the 3D case as well.
I would say the entire post is mostly about 3D, not 2D rendering. If you are curious about this topic being approached for 2D rendering here is a nice write-up I found about that: https://ciechanow.ski/alpha-compositing/
One particular criteria for AA techniques that no one cares about in 3D but is very relevant in 2D is correctness and bias. AAA for example is heavily biased and thus incorrect. Drawing the exact same shape multiple times in the same place will make it more opaque / darker. The same thing does not happen with MSAA which has a bounded error and is unbiased.
(Thick) line drawing is especially interesting to me, since it's hard [1].
I also stumbled upon this [2] recently and then wondered if i could use that technique for every shape, by converting it to quadratic bezier curve segments.
Do you think that's a path to follow?
[0] https://github.com/Lichtso/contrast_renderer
[1] https://mattdesl.svbtle.com/drawing-lines-is-hard
[2] https://scribe.rip/@evanwallace/easy-scalable-text-rendering...
- Implicit Curve Rendering (Loop-Blinn) and stencil geometry (tessellation-less) for filling
- Polygonization (with tangent space parameter distribution) of offset curves for stroking
> by converting it to quadratic bezier curve segments
Mathematically, the offset curve of a bezier curve is not a bezier curve of the same degree in the general case (exceptions are trivial cases like straight lines, circles and ellipses). Instead you get terrible high degree polynomials. You will have to approximate the offset curve anyway. I choose to use polygons (straight line segments), but you could also use splines (bezier segments), it is just overtly complex for little to no benefit IMO.
BTW, distance fields and offset curves are very similar. In fact the distance field is the set of all possible offsets of offset curves and the offset curves are the isolines on the distance field.
Here is a good summary of all the edge cases to think about in 2D rendering: https://www.slideshare.net/slideshow/22pathrender/12494534
About subpixel AA: Don't bother, LCDs are on the down trend.
OLEDs have subpixels, too. The real problem is the growing diversity in subpixel arrangements, making it impractical to have a good subpixel AA method for every display type on the market. And mobile platforms have it worse, because screen rotation means you need to be able to change AA methods on the fly.
The solution presented relies on signed distance fields, yet skims over the important part - a distance to what? In 2D it is obvious, we are measuring distance to an edge between the object and its background, to a silhouette.
In 3D, when objects may rotate producing self-occlusions, things get more complicated - what are we measuring the SDF against? The silhouette of a 3D object's 2D projection is ever-changing and cannot be trivially precomputed.
TAA covers all types of aliasing, while this only covers edge aliasing.
Many modern games use monte carlo based approaches for indirect lighting and other effects, which basically requires TAA.
Hmm... On my Android phone I definitely see a difference between 2x and 4x, but it's not "rounded" like the iPhone one.
To do alpha blending correctly you need to blend with what's behind the object. But what if you haven't rendered what's behind the object? You won't have anything to blend with!
This means you first have to sort all the objects you're rendering and render them in-order. It also means you can't use a depth pre-pass because you need to draw what's behind the object even for the parts you already know will be covered by the object in front of it. There's a bunch of more reasons to avoid it, but those are some of the basic ones.
An alternative is to draw only a few pixels of what's behind the object you're rendering at the edges of the object that's in front and then alpha blend those extra samples together, which seems to be the solution proposed in the article for actual 3D games. So then the catch is that you're doing MSAA, just with high-quality blending rather than the standard averaging of the samples.
Sure it's better than TAA, but come on, this can't be the ultimate end for gaming graphics... At least I hope it isn't.
For normal maps specifically, you can preserve lost information from downsampled normal maps as roughness: https://media.steampowered.com/apps/valve/2015/Alex_Vlachos_...
MSAA also doesn't require you to have only one color sample for each pixel but this can be varied per draw call (at least for desktop GPUs) - effectively allowing you to supersample some objects when needed.