Google and Binomial Open-Source Basis Universal Texture Format | Hacker News Reader

Google and Binomial Open-Source Basis Universal Texture Format

opensource.googleblog.com

265 pointsjasondavies20 comments

Compressed texture formats are underused. I think few people outside of game development understand the benefits. The linked post explains it but I'd like to take a stab at it too.

Regular image compression formats like PNG or JPEG save network bandwidth and disk storage but not RAM. They must be completely decompressed and stored uncompressed in RAM as long as they are displayed. Compressed texture formats do save RAM. They remain compressed in RAM while they are being displayed, typically 1/6 the size of regular images or even smaller. That saves not only RAM but memory bandwidth and power too; especially important on mobile. Switching to compressed textures can make applications significantly faster.

Compressed texture formats have two main disadvantages that Basis solves. One is that their compression ratios are fixed and not as good as JPEG. Basis adds extra compression to make them more comparable to JPEG on disk.

The other, larger issue is that compressed texture formats require hardware support on the GPU, and GPUs don't agree on what formats to support. There is no single format with guaranteed support on every platform, which means that cross platform applications must encode each image multiple times and ship different formats to different users. Basis solves this by efficient runtime transcoding to most common GPU formats. This capability is unique and a really big deal.

Anyone with images in their application should take a look at Basis. Now that you can ship a single compressed texture file that's supported everywhere, I hope that this technology can expand beyond games.

Part of the issue is they have been patented to the hilt by parties like S3. It hasn't been an issue in video games because companies just agree to implement other S3 apis in return for a patent license, but in cases like this that obviously isn't an option.

https://en.wikipedia.org/wiki/S3_Texture_Compression

It has been an obstacle for open source drivers too, but luckily this specific patent expired.

> beyond games

Accept: image/basis

Could this be used for convolutional neural networks?

I'm not sure where it would help. The encoding process is slow so it wouldn't make sense to encode anything that changes like the inputs or activations, or the weights during training. You could maybe encode the weights after training but you'd really have to massage the data to fit into these formats, and the lossy perceptual tricks used here are designed for the human visual system, not high-dimensional convolution filters.

is it lossless? what is the trade off?

No, compressed texture formats are lossy. Also the fixed bitrate that allows random access into the compressed pixels constrains the encoder a little more than the variable bitrate of JPEG or other image formats.

From this [1] it sounds like the texture is a modified variant of ETC1, which gets loaded up from disk into memory as such, and transcoded on the CPU into target GPU format (DXT, PVRTC, etc), and then uploaded to VRAM.

If so that sounds similar to their previous Crunch library [2], in that it does transcoding on the CPU before moving the texture to the GPU - but most likely much faster than Crunch because it starts with ETC as shared basis. (And artifacts are a common with transcoding between compressed formats, especially when transparency is present, but I have high hopes for this one considering the authors! :) )

Authors suggest that a version that transcodes on the GPU would be possible - that would be really interesting, I would imagine that would be even faster, although at the cost of higher texture memory usage.

[1] https://github.com/binomialLLC/basis_universal#quick-basis-f... [2] https://github.com/BinomialLLC/crunch

Crunch only supports one family of formats, DXT. (Unity has a fork that supports the ETC family, but it does not support decoding ETC and DXT from the same compressed file). Efficiently transcoding one compressed file to multiple compressed texture formats at load time is the key new feature that only Basis provides.

Congratulations to https://twitter.com/sehurlburt ! ...who is also a font of interesting and insightful business advice and life/growth thoughts

Seconded. Very friendly too :)

This is amazing to me from an outsider's point of view; I thought Binomial was a two-person shop (idk if they expanded recently). To be able to partner with Google and Chrome Media on an equal level just doesn't seem possible to me. Binomial must have some pretty nice tech. Congratulations on the partnership!

They are still two persons. https://twitter.com/sehurlburt/status/1131371251790835712

Thanks for the link!

Well, they fixed a problem none of the GPU makers is interesting in solving directly: providing a common image compression format for 3D rendering. This is all a patent-encumbered mess, and the reason we are in this mess are the GPU vendors alone.

When Microsoft still had power over desktop computing, it pretty much dictated GPU vendors to use the DXT image compression format which is supported on all desktop GPUs, but nothing similar has happened on mobile.

In the current situation, this problem couldn't have been solved from the "inside". It isn't a trivial problem by far to solve (probably one of the few topics in programming that deserve the name "rocket science"), but it also doesn't require a huge workforce or standardization committee (and thank goodness this hasn't been "designed-by-committee"), "just" many years of focus and dedication.

There is a paper describing the technique here: http://gamma.cs.unc.edu/GST/gst.pdf

That paper described the inspiration for Basis, but Basis itself is slightly different.

https://github.com/binomialLLC/basis_universal#quick-basis-f...

> Internally, Basis files are composed of a non-uniform texture array of one or more 2D ETC1S texture "slices". ETC1S is a simple subset of the ETC1 texture format popular on Android.

Three.js already supports it: https://github.com/mrdoob/three.js/pull/16522

Is this going to lead to a new general image format? What happens with webp? That seemed to have never taken off much because of legacy support.

Is anyone thinking of adapting this create a new GPU-based video format ? Like HAP https://hap.video/

Reply on news.ycombinator.com