Hardware Image Compression | Hacker News Reader

Hardware Image Compression

ludicon.com

73 pointsluu15 comments

The irony of hardware image compression is that the devices that need it most are typically older, bandwidth-constrained SoCs. However, these are precisely the devices that do not support modern formats.

Technologies like ARM AFRC and PVRIC4 can only be used on modern flagship devices. Since flagship memory bandwidth isn't particularly strained to begin with, we end up spending a massive amount of effort on optimizations that only benefit a fraction of users. In most cases, teams are simply unwilling to pay that development cost.

The driver behavior of PVRIC4 perfectly encapsulates the current state of mobile GPU development: 1. The API promises support for flexible compression ratios. 2. The driver silently ignores your request and defaults to 1:2 regardless. 3. You only discover this because a PowerVR developer quietly confirmed it in a random comment section.

This is a microcosm of the "texture compression hell" we face. Beyond the mess of format fragmentation, even the driver layer is now fragmented. You can't trust the hardware, and you can't trust the software.

While the test results for ARM AFRC are genuinely impressive—it's not easy to outperform a software encoder in terms of quality—it remains problematic. As long as you cannot guarantee consistent behavior for a single codebase across different vendors, real-time CPU and GPU encoders remain the only pragmatic choice.

For now, hardware compression encoders are just "nice-to-haves" rather than reliable infrastructure. I am curious if anyone has used AFRC in a production environment? If so, I’d love to know how your fallback strategy was designed.

I don't disagree given your "most" qualifier, but there's a case where every level of hardware would benefit: compression of textures generated at runtime, either via procgen or for e.g. environment maps.

This is in a frustrating state at the moment. CPU compression is way too slow. Some people have demoed on-the-fly GPU compression using a compute shader, but annoyingly there is (or at least was at the time) no way in the GPU APIs to `reinterpret_cast` the compute output as a compressed texture input. Meaning the whole thing had to be dragged down to CPU memory and uploaded again.

Agreed.

we hit some wired case on Adreno 530, ran into bizarre GPU instruction set issues with the compute shader compressor, that only manifested on Adreno 53x. Ended up having to add a device detection path, and fall back to CPU compression. which defeated much of the point.

Spark supports Adreno 5xx on both GLES and Vulkan backends. Getting the codecs to work on these devices and obtaining good performance was very challenging.

Unfortunately the article mentions nowhere why a GPU would ever need to compress (rather than decompress) images. What's the application for that? In the beginning the article mentions formats for computer game textures, but I'm pretty sure those already ship compressed, and they only need to be decompressed by the client GPUs.

Someone mentioned environment maps. Anything that's done with framebuffers or render-to-texture might benefit. e.g. Water reflections and refractions, metal surfaces reflecting the world, mirrors in bathrooms, panini distortion for high-FOV cameras, TV screens like the Breencasts in Half-Life 2

Why would they benefit from hardware compression?

The most immediate benefit is reduced memory use. Many devices are memory limited and with skyrocketing RAM prices this is becoming more problematic.

Oversubscription drops performance catastrophically, but even without running into memory limits, compression reduces bandwidth which increases performance and lowers power use. This results in better experiences and longer battery life.

But in order to be compressed, don't we have to load the image into memory first, uncompressed? I don't quite see how this could result in reduced memory usage.

There are many textures that can't be encoded in advance: images compressed with transmission formats such as jpeg or avif, procedural textures, terrain splatting, user generated textures, environment maps, dynamic lightmaps, etc.

Wasn't there also patent issues with early texture compression schemes? DXT1? Adding a delay of 20 or so years.

Yes, the underlying S3 texture compression algorithm was patented in the US in the late 90s. The last, relevant patent expired in 2018[1]

Direct3D called its variants DXTn, later rename to BCn. From what I recall, Microsoft had some sort of patent licensing deal that implicitly allowed Direct3D implementers to support their formats.

OpenGL had an extension called GL_EXT_texture_compression_S3TC[2].

Under "IP Status" the extension specification explicitly warns that even if you are e.g. shipping graphics cards with Direct3D drivers, supporting S3TC, you may not legally be able to just turn that feature on in your OpenGL driver.

[1] https://en.wikipedia.org/wiki/S3_Texture_Compression#Patent

[2] https://registry.khronos.org/OpenGL/extensions/EXT/EXT_textu...

I feel like it's a pretty weird omission taht basis_universal isn't mentioned at all? Can also target Khronos's ktx2 too. https://github.com/BinomialLLC/basis_universal

Because the article is about hardware formats, not storage formats. GPU hardware formats are left compressed in GPU memory, the GPU decodes it on the fly. Basis universal must be decoded to a GPU hardware format (or to plain RGBA) before it can be used by the GPU.

Reply on news.ycombinator.com