Aliasing and you | Hacker News Reader

Aliasing and you

codinghorror.com

113 pointsBIackSwan43 comments

Interesting... in audio, we've basically had affordable super-sampling for a few years. That's exactly what "record at 96KHz or 192KHz" is; all your plugins run against a waveform that's higher-bandwidth than you can hear, and then you downsample and dither at the end to 44.1 or 48KHz in mastering. Some of the best equalizer plugins had internal upsampling, so that even if your source material was 48KHz, the processing was all done at 96KHz. The quality gain from the higher resolution offset the quality loss from resampling.

(In audio, there are other benefits to higher resolution, like latency; a 256-sample buffer adds 6ms at 44.1KHz, but only 1ms at 192KHz.)

> (In audio, there are other benefits to higher resolution, like latency; a 256-sample buffer adds 6ms at 44.1KHz, but only 1ms at 192KHz.)

But the buffer is there to allow you to gather data up before sending it across the pipe (physical or virtual) so you're not incurring latency you don't need to. Increasing the resolution of your data doesn't decrease latency, as you're just incurring that hit more frequently. If that worked, you could just do away with the buffer and send a continuous stream of data at whatever sampling rate you wanted.

I'm thinking of setups which often have some absolute minimum buffer size, maybe due more to the protocol (e.g. ASIO) than the converter design; my RME Digiface setup can't work below 56 samples, no matter what the sample rate.

Don't forget that the air between you and the speakers is another buffer, holding about 150 samples at 44.1 kHz, assuming near field monitors.

Yep, which is why it's so amazing that bands play to a beat that doesn't really exist - it's an agreed-upon moment in time that we pretend is the same for all of us, but when you're standing 20 or 30 feet apart, it isn't.

But latency is critical for tracking through headphones, especially singers; a little bit of latency can screw up your pitch perception as well as your timing.

Well, that's why we have hardware monitoring. You only really need the low latency stuff for recording software synthesizers / samplers or if you have a virtual amp, etc.

As someone who has spent a fair deal of time in the audio world both professionally and independently I'm pretty confused by this post.

The primary reason to oversample in the audio world is two fold:

1) To avoid aliasing, which is a somewhat different issue from graphical aliasing. ADC conversion creates aliases of the primary signal lower in the frequency band. For example if we sample at 40khz we can resolve signals up to 20khz. If the input contains content higher than this frequency it will alias into the pass band.

For example a 24khz tone on the input would appear as a 16 khz tone, a 28khz tone would appear as a 12khz tone, all higher frequency content will mirror around the Nyquist-Shannon point. Now if it were possible to build a "brick wall" filter, that is. completely block anything above the Nyquist-Shannon point and completely pass everything under this point, there would be no issue. But, in practice, this type of filtering isn't possible. As a result if your sampling at 48khz and you want to accurately convert all content up to 20khz you have to play a balancing act. You either provided flat response to 20khz and accept that some aliasing will make it through, or deal with some reduced level(roll off) at the higher end of your desired spectrum and block all audible aliasing.

This is really the primary reason for oversampling for high fidelity content as we can push the anti-aliasing filter well above the audible spectrum, allowing us to get both flat response to 20khz and block all audible aliasing. It should also be said that this issue is becoming less and less and issue as ADC/DACs are now implementing digital filters which can get far closer to the ideal 'brick wall' than is reasonable with analog filtering.

2) At a deeper level, oversampling increases resolution. Note that resolution here means actual resolution, that is bit depth, which is in opposition to the use of the term in the parent post. An example would be modern Sigma-Delta ADC/DACs.

Its common to use a low resolution part, maybe only 1-4 bits and oversample at a higher rate. For example, if your target output is 48khz @ 24 bits, you can use a 20 bit ADC and sample at ~12.28Mhz and downsample (internally within the ADC/DAC, really just averaging) to get your desired output. In practice a part may only operate on 1-4 bits but sample, much, much faster to get the desired resolution.

As for upsampling for the purpose of signal processing, there are a limited number of algorithms where this is useful and upsampling in and of itself introduces issues that have to be dealt with. The reason for upsampling tends to have more to do with processing accuracy limitations created by computation than anything else. For example bi-quad filter coefficients at very low frequency, say 5-8hz can exhaust the available resolution of a 32bit float at 48khz where as it may be completely fine at 192khz. This can be an issue with a sub-sonic filter in a subwoofer for instance.

You're right; I was being imprecise (or sloppy, perhaps). Graphical aliasing isn't the same as signal aliasing, though it's easy to equate the two conceptually.

I understand the value of using 96KHz instead of 48KHz so you can have a gentler filter, at least in the days before digital filters (as you say), but don't you agree that there's no further advantage to 192KHz in that respect? Maybe I'm justifying 192KHz by saying that it's for cleaner processing, when the real reason is It's A Bigger Number.

I've never heard anyone use "resolution" to mean bit depth; in fact, I've always heard it used to specifically mean sample rate - e.g. 96KHz resolution, 24 bits. But I've never studied DSP programming - I have a plumber's understanding, not a fluid mechanics engineer's.

How is graphical aliasing not signal aliasing? I've always thought it was, but then all my signal processing knowledge comes from image processing and not EE courses.

It is. The difference is that in audio it's generally associated with periodic waveforms, whereas onscreen it is more often problematic with isolated peaks. Where you have periodic, aliased structures onscreen, it's often called a Moiré pattern, but it's all the same effect.

FXAA is neat, but the idea of using pixel-based edge detection on games is, IMO, a bit silly. Why not use detect edges on the depth buffer? You'll much more easily see the contrast and you will be far, far less likely to blur your textures. I might have to whip up a quick WebGL demo of this.

FXAA and other similar techniques are only relevant when dealing with deferred rendering only. Detecting edges on the depth buffer may sound as the solution until you realize that two planes at right angles has an edge between that can alias. The change there is in the normal and not the Z.

Also modern graphics cards do not give programmatic access to the z-buffer as the z-buffer format is often proprietary. Deferred shading based renderers often duplicate the z-value also into a texture.

Deferred rendering can render large number of dynamic, unshadowed lights very efficiently. It can be used as approximate global illumination technique.

> FXAA and other similar techniques are only relevant when dealing with deferred rendering only.

Why is that? While techniques like FXAA are needed for deferred rendering due to the order of operations, I don't see why FXAA isn't acceptable on non-deferred renderers.

> Detecting edges on the depth buffer may sound as the solution until you realize that two planes at right angles has an edge between that can alias. The change there is in the normal and not the Z.

If you have two planes at a right angle to each other, you'd see that in the depth buffer. Imagine you're looking at a line of pixels and you're doing this on each pixel -- you'd look to see if the pixels on either side of the one you're on are trending in opposite directions from the current one, and know it's an edge.

> Also modern graphics cards do not give programmatic access to the z-buffer as the z-buffer format is often proprietary.

As far as I'm aware, all modern graphics cards allow you to render the depth buffer to texture just like the color buffer.

> Deferred rendering can render large number of dynamic, unshadowed lights very efficiently. It can be used as approximate global illumination technique.

Sure, but there's no reason you can't do post-AA using the RTT depth buffer and your composited color buffer.

> Why is that? While techniques like FXAA are needed for deferred rendering due to the order of operations, I don't see why FXAA isn't acceptable on non-deferred renderers.

FXAA is acceptable for the other techniques as well. However, MSAA is, in my opinion, a better technique as it uses supersampling at the edges and is already hardware assisted. FXAA's strength is that it is less expensive than MSAA memory and bandwidth wise. Both are a premium in deferred shading as you are using a 4x RBGA16 texture render target or something close to it :)

> If you have two planes at a right angle to each other, you'd see that in the depth buffer. Imagine you're looking at a line of pixels and you're doing this on each pixel -- you'd look to see if the pixels on either side of the one you're on are trending in opposite directions from the current one, and know it's an edge.

The question is can you get reliable edge detection with less number of texture look-ups on the z-buffer versus the RBG based edge detection?

> As far as I'm aware, all modern graphics cards allow you to render the depth buffer to texture just like the color buffer.

Not quite. You can render the z value to a texture, but that is done in the pixelshader and do not touch the hardware z-buffer used for z-fail/z-pass tests. It is this hardware z-buffer that is inaccessible, as it shares it memory with the stencil buffer which is used for stencil tests. Hence the proprietary format.

> Sure, but there's no reason you can't do post-AA using the RTT depth buffer and your composited color buffer.

See http://msdn.microsoft.com/en-us/library/windows/desktop/bb14...

> All render target surfaces used together must have the same bit depth but can be of different formats, unless the D3DPMISCCAPS_MRTINDEPENDENTBITDEPTHS cap is set.

As you can see, the bit depth restriction means that you would end up with 2x bandwidth usage with the z based edge detector, especially on the older hardware where you do not have the bandwidth to spare. It would be a different case if you can sample the hardware z-buffer.

> Not quite. You can render the z value to a texture, but that is done in the pixelshader and do not touch the hardware z-buffer used for z-fail/z-pass tests. It is this hardware z-buffer that is inaccessible, as it shares it memory with the stencil buffer which is used for stencil tests. Hence the proprietary format.

This was true for Direct3D 9 but 10+ allow you to bind the depth/stencil buffer as a texture, although there is the caveat that you can't have it simultaneously bound as a texture and as a render target.

See "Reading the Depth-Stencil Buffer as a Texture" on this page: http://msdn.microsoft.com/en-us/library/windows/desktop/bb20...

> The question is can you get reliable edge detection with less number of texture look-ups on the z-buffer versus the RBG based edge detection?

The way it's working in my mind has about the same number of lookups as FXAA, but I'm honestly not sure how that'd pan out in practice. We'll see.

> As far as I'm aware, all modern graphics cards allow you to render the depth buffer to texture just like the color buffer. Not quite. You can render the z value to a texture, but that is done in the pixelshader and do not touch the hardware z-buffer used for z-fail/z-pass tests. It is this hardware z-buffer that is inaccessible, as it shares it memory with the stencil buffer which is used for stencil tests. Hence the proprietary format.

No need for the internal format, though -- raw depth buffers are A-OK.

> As you can see, the bit depth restriction means that you would end up with 2x bandwidth usage with the z based edge detector, especially on the older hardware where you do not have the bandwidth to spare. It would be a different case if you can sample the hardware z-buffer.

One thing to keep in mind is that yes, you're rendering it to texture and you incur some overhead there, but you don't have to pull that data back to the CPU so you're dependent on bandwidth between the GPU and its memory, not main memory. Sure, it's double the memory used there, but that's negligible from a bandwidth perspective. Also a good thing to note that you generally already have your depth buffer if you're doing deferred rendering, and you can do all the AA based on that, in theory.

> he way it's working in my mind has about the same number of lookups as FXAA,

However, the FXAA will have a clear win as it works over a partly transparent polygon. So you will have to combine both an RBG and z based to get the same result as an FXAA.

> you're dependent on bandwidth between the GPU and its memory, not main memory.

The bandwidth I was talking about was the older cards which do not have bandwidth or fill rate to keep 60fps with deferred shading, no AA. FXAA would be a candidate there if it is a pure RGB technique on the composited frame, especially as it is cheaper than MSAA.

> This was true for Direct3D 9 but 10+ allow you to bind the depth/stencil buffer as a texture, although there is the caveat that you can't have it simultaneously bound as a texture and as a render target.

Missed that one as I have been mostly playing around in DX9 and only a bit of DX10.

There is the case of the billboarded or partly transparent texture. The z-based edge methods will not work in these cases as you render them with z-write off. Meaning, atleast in those cases, FXAA is a clear winner over both MSAA and z-based edge detection.

>> FXAA and other similar techniques are only relevant when dealing with deferred rendering only. Detecting edges on the depth buffer may sound as the solution until you realize that two planes at right angles has an edge between that can alias. The change there is in the normal and not the Z.

Why do you say that FXAA is only relevant to deferred rendering?

If you have a deferred rendering setup, you'll have the depth buffer and the normal buffer (a floating point texture with normal vectors) available. These can be used to do edge detection for silhouette edges and non-silhouette sharp edges.

I don't know the details of modern anti aliasing techniques, but I'd guess that you can get the best edge detection using a combination of the color, depth and normal buffer. Whether you want to do this for anti aliasing is a different issue, a simple RGB edge detect may yield better results.

>> Also modern graphics cards do not give programmatic access to the z-buffer as the z-buffer format is often proprietary. Deferred shading based renderers often duplicate the z-value also into a texture.

This doesn't make any sense. The depth buffer may be internally stored in a compressed proprietary format, but it's still available to the programmer as a regular buffer or texture.

You have plenty of ways to access the depth buffer for reading. The depth buffer can be rendered into a texture (either as floating point or regular 24 bit integer), which can be read back in a shader. You can do regular lookups where you get back the depth value or a shadow lookup with percentage closer filtering.

The depth values can also be read back from the buffers (using glReadPixels in OpenGL) to be analyzed on the CPU (not feasible for anything realtime) or saved to disk or whatever.

When doing cel-shading you usually render a pass that stores the depth and normal values of each pixel into a texture. Thus you have your own z-buffer and access to the normals.

If either the difference of the normal or the z-value is over a certain threshold you add some black color to the pixel to make the edge look thicker.

The algorithm analyses the contrast from the luminance of the pixels and does more than just edge detection. There's more in the paper that I don't understand with a quick look. Screen space edge detection (RGB or Z) + low-pass filtering is not a new idea at all, maybe FXAA is just the first that can make it work well.

FXAA smooths out stuff way too much for my liking (look at the character highlights and the detail on the horse):

http://www.geforce.com/Active/en_US/shared/images/guides/the...

I know very little about this stuff but wouldn't using the depth buffer in addition to the current algorithm make it possible not to smooth over details like this?

It's extremely fast, looks great, and is very easy to add. If you can produce something better games will immediately use it instead. Good luck.

SMAA is even better and faster than FXAA. http://www.iryoku.com/smaa/

Also there is software that automatically injects SMAA into your d3d9, d3d10 or d3d11 games: http://mrhaandi.blogspot.com/p/injectsmaa.html

Pulled directly from codinghorror comments, posted by: Mārtiņš Možeiko

I was hoping someone else would say that. Have you really tried SMAA? What is your opinion of it on a few games?

I thought it was a little odd that he says the proper way to solve the problem is to have a higher-res screen, just before he says that we can't do super-sampling because it's too expensive.

High-res screen: expensive[money]

Super-sampling: expensive[cpu]

A high-res screen isn't just expensive money-wise -- at the resolutions you need to avoid AA techniques, you're rendering at the same resolution you'd be rendering at for AA techniques. So it's more expensive in terms of money and in CPU/GPU time.

But wouldn't normal sampling on a high-res screen be just as expensive[cpu] as super-sampling on a normal-res screen?

I guess I'm probably the only person in the world that prefers the look of no-AA...

No, you're not alone. I've never liked the strange colored edge effects you get from normal AA. They're as annoying as jaggies.

As an aside, the FXAA touted in this blog post looks washed out and generally terrible.

Most of the time, you loose the "crispness" of the final image with AA and everything tends to look a little bit blurry. Couple that with fact, that it drops the FPS by quite a bit and over time I got slowly conditioned against AA. However in some games it looks very good. WoW for example has a much better image quality with AA. Of course it also runs with 60 FPS pretty much everywhere.

As the author mentions, non-AA looks worse in motion. I could take it or leave it for static images, but when the location of an aliased boundary is moving, it tends to flicker as pixels around the edge make discrete jumps across the boundary. It addition to looking bad, I also find the extra activity distracting.

That was a good primer for a non-gamer like me.

Can it be used to make flash graphics and webgl lower load too?

I don't know about Flash, but I'd say this is a bit unlikely. You need special shader programs for custom antialiasing tricks like FXAA.

For WebGL, you should be able to write the shaders and a multi-pass renderer framework that does the antialiasing. However, with WebGL, you're constrained to using somewhat simpler texture formats than you have available with a modern desktop. If you have to use 8 bit RGBA textures, you will probably suffer from color banding and other nasty artifacts compared to 16 bit floating point textures that are normally used on the desktop.

Amazing. I always thought that something better than super-sampling had to exist.

I hope this becomes standard on PS3 titles, it's painful seeing all those ragged edges on screen, knowing you have a few gigaflops of processing power sitting there, doing nothing :(

A number of PS3 titles use a technique called morphological anti-aliasing (MLAA) which is more computationally expensive than FXAA but can look better (to art directors and rendering engineers anyways :)) and, because it runs well on the SPUs, it is able to offload the GPU for it to begin on its batch of work.

Huh, I've always thought that that was how anti-aliasing in games worked; take the entire scene and smooth it, Photoshop filter style.

TIL.

If you are going to analyze all pixels on the screen, it would be better to use some kind of median filter[1].

It looks smoother and more focused: http://i.imgur.com/hsq84.jpg

[1]http://en.wikipedia.org/wiki/Median_filter

No wonder it looks smoother. The pixels are half the size!

Median filter has to sort pixels (3x3, or 5x5) and get the median (this is by memory, haven't looked at wikipedia). As such it might be or might not be better solution.

Also it can't be done in-place (or can it?)

You could do it in-place with a memory cost of one pixel row (for 3x3, or two for 5x5) plus one (or 2) pixels. That might make be worthwhile depending on where you're actually doing it. It might be more efficient - it's worth implementing both to see.

Am I like the only person in the world who hasn't been able to see jaggies in like.. 6 years?

Reply on news.ycombinator.com