Near-lossless image formats using ultra-fast LZ codecs

The author of the OP also develops Basis, which is designed to take advantage of GPU texture compression formats. GPU formats are efficient in terms of memory space and bandwidth usage, but they have a fixed bits-per-pixel so they're not the most efficient for storage and transmission. There's also a rift between which formats are supported on desktop and mobile GPUs. Basis layers on a second level of variable-bitrate compression which is designed to be efficient to transcode to various compressed GPU formats (for both desktop and mobile) on the fly.

https://github.com/BinomialLLC/basis_universal

Delivering images in a GPU-friendly manner on the web could be a massive memory saver (PNG/JPGs end up being 32 bits-per-pixel in memory, while GPU formats are usually 4bpp or 8bpp) but web standards haven't caught up yet, for now the only way to display these formats on the web is inside a WebGL context. It works, but it's a lot of machinery just to show a picture.

GPU compression formats are still well subjected to entropy encoders such as LZ family.

GPUs can do LZ4 decompression. So you could still save ~50% PCIe bandwidth and a bunch of CPU time with these formats... but not VRAM, since (unlike BC7) its still a raw bitmap in memory.

Apparently modern GPUs losslessly compress textures in memory by themselves, but the only info I can find is that this is proprietary AMD/Nvidia/Intel voodoo magic.

Yeah. And where I live, available mobile data volume per month is still pretty limited in most plans, and it is increasing only slowly. Unlimited volume plans are very expensive.

ZPNG doesnt, but LZ4PNG supposedly would except there are no links to actual implementation and Google doesnt know either. does it exist at all?

Draco is just a bad algorithm. It's just an implementation overcomplicated paper from the 90s (Edgebreaker), basically compressing the index buffer but nothing else. And it also reorders the index buffer to fit topology, meaning your post-transform cache optimization is undone. And it's really slow, to the order of 1us/tri [0].

There are other papers and algorithms that are similar in size to Draco, but without the massive overhead, e.g. EXT_meshopt_compression. Even in 90s papers, there are far more competitive algorithms [1]

[0] https://medium.com/box-developer-blog/showdown-3d-model-comp...

[1] For instance, https://graphicsinterface.org/wp-content/uploads/gi1998-4.pd...

Then do that. It's not a video codec. The problem is computation power is increasing much more slowly than bandwidth, and is likely to continue to. 8K PNGs (by my napkin calculations) already take around 1 second of CPU time to decode. There are other solutions, and certainly better options today, but it's plausible that in the next 10-15 years, this will become the best option for fast-loading, high-resolution, 2D images. (Or texture data for 3D graphics. Or 360, I guess - not sure why that'd be different.)

I don't get it, you're complaining that an image codex is not a video codec? Well... duh.

2D images are used for their own sake, not as a compromise because you'd rather show a moving 3D object rendered at 8K but you can't. So obviously this is about the problem of compressing 2D images, it's off-topic to argue that it can't do video or whatever.

Wouldn't fast decompression become very relevant for viewing lightfield data? If these media assets are indeed very big as you suggest, then doesn't it make sense to stream the relevant view data based on view orientation?

For something like that you'd low latency and thus fast decompression.

Very little, you can check by downloading the images as the LZ4 images has been reencoded into PNG so the browsers can show them.

Orig PNG: 1443667 bytes. Lossy LZ4I = 1433231, delta = -10436 bytes. A more lossy LZ4I = 1344183, delta = -99484 bytes from PNG.

Maybe I’m missing something, but isn’t that how JPEG works? It preprocesses the DCT blocks to throw away data (or add noise if you want to read it that way). By doing so, it lends itself to shorter encoding of the data (compression of a sort).

I can't see this being used for games, those need to deliver textures in GPU-friendly formats like BC/ASTC, or a close analogue like Basis for performance reasons.

In this day and age you can buy a 2TB M.2 SSD for $140. Even if you only gamed AAA that’s a ton of games. How often would you really make use of all of that space, to the extent where keeping extras on an external HDD would be too much of a hassle?

Perhaps my lived experience is too different. I typically keep a small handful of games that I play frequently, like Halo Infinite or Cyberpunk 2077. Both have huge memory footprints, but not much when HD space is measured in terabytes…

Lz4 decompresses with very little CPU load - a fraction of Zstd's load. When bottlenecked by CPU time rather than I/O it could make a large difference.

Because SPARC is making a comeback any day now!

The computational cost of an endian swap rounds to zero, and file format implementers expect serialised ints to be big-endian by default. It's just the Principle of Least Astonishment at this point.

Also no room for a version number or extensions.

You know that sooner or later something else is going to be needed. Eg, HDR.

network byte order

The site (Blogspot) from Google, something from the earlier days of the Internet, are not really updated since a very very long time.

I have set my browser to default to Reader View Mode for Blogspot, PG's website, DaringFireball and few others.

Orrr, the grid is moved towards renewable energy so that the fear of global warming doesn't stunt technological advancements?

Not a bad question, though a difficult concept to measure in reality.

We can estimate the emissions caused by crypto-currency as that is almost entirely CPU-bound by about the same amount (or an easy to estimate average) per unit. The costs of transmission and storage are going to be relatively minor.

For images the compression cost, while the process is similarly CPU-bound, is pretty small unless you are doing something small like compressing to hundreds/thousands of algorithm/parameter combinations and picking the best result (perhaps by some heuristic more complex than final file size). Also transmission and strorage costs are difficult to estimate because for any given image how do you guestimate how many times it will move and be copied?

Having said that, the article's statement that: “[as] bandwidth savings from overly lossy image codecs will become meaningless, [] the CPU/user time and battery or grid energy spent on complex decompression steps will be wasted.” implies that energy use is at least a coincidental consideration here. Though image quality is by far the primary concern, and that despite being subjective is probably an easier metric to measure in this case, with compression time likely to be ahead of energy use too.

Isn't that more or less the same thing in the end?

Storage at this point is plentiful. Most people can live just fine with 1 TB of it, which is trivially found in both HDD and SSD forms. Using less saves you nothing, as the media is still there.

Bandwidth is also plentiful and available 24/7. It's also very bursty -- you only download your game once, while you may play it a lot. So saving 50% download time makes no difference energy-wise.

What we can control a lot is CPU power usage. We now have 16 core consumer CPUs that are designed to sleep when not in use, so difference in power usage at runtime can be dramatic.

So an algorithm that trades disk space for decoding efficiency is probably the best way to save power.

> Why not optimize for CO2 emission instead ?

How if you don’t know where the electricity comes from powering the hardware your image decoder runs on.

What is with this new trend of eco-virtue-signaling?

The best way to "optimize for CO2 emission" is to just stop living.

The best way to optimize for CO2 is to ask the user to do it at the right time of day for their grid. Otherwise it's the same thing as using less energy which is ~ the same thing as being faster (assuming fixed resources).

But watch out for induced demand (the user turning the graphics settings up because you made the FPS better.)

It already exists - progressive jpeg is interlaced.

That's basically how PNG interlacing works:

http://www.libpng.org/pub/png/pngpic2.html

The problem is that it destroys compression rates. PNG filtering and compression depends on being able to make predictions based on adjacent pixels; interlacing ruins that.