Hacker News Reader

Comment by IshKebab | Hacker News Reader

We must be reaching the limit at which video codecs can only achieve better quality by synthesizing details. That's already pretty prevalent in still images - phone cameras do it, and there are lots of AI resizing algorithms that do it.

It doesn't look like AV2 does any of that yet though fortunately (except film grain synthesis but I think that's fine).

View on HN

Arguably that's already happening with film grain — you have to extrapolate _what the original probably was_, encode it because it's smaller, then add the noise back to be more faithful to the original despite your image being better.

I imagine e.g. a picture of an 8x8 circle actually takes more bits to encode than a mathematical description of the same circle

>I imagine e.g. a picture of an 8x8 circle actually takes more bits to encode than a mathematical description of the same circle

I wonder if there are codecs with provisions for storing common shapes. Text comes to mind - I imagine having a bank of 10 most popular fonts an encoding just the difference between source and text + distortion could save quite a lot of data on text heavy material. Add circles, lines, basic face shapes.

Outside of AV1/2 (and linear media in general) that's already well and truly developed tech. Nvidia DLSS, AMD FSR and Intel XeSS all provide spatial/temporal super sampling to process lower fidelity base renders [0].

There also seems to be a fair bit of attention on that problem space from the real-time comms vendors with Cisco [1], Microsoft [2] and Google [3] already leaning on model based audio codecs. With the advantages that provides both around packet loss mitigation and shifting costs to end user (aka free) compute and away from central infra I can't see that not extending to the video channel too.

[0]: https://mtisoftware.com/understanding-ai-upscaling-how-dlss-...

[1]: https://www.webex.com/gp/webex-ai-codec.html

[2]: https://techcommunity.microsoft.com/blog/microsoftteamsblog/...

[3]: https://research.google/blog/lyra-a-new-very-low-bitrate-cod...

Please no. This is what jbig2 does for images and it’s a nightmare in my view, you can’t trust the result is not something totally different from the original [1]

[1]https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...

>We must be reaching the limit at which video codecs can only achieve better quality by synthesizing details.

Not quite yet as shown in H.267. But at some point the computational requirement vs bandwidth saving benefits would no longer make sense.

I think Fabrice Bellard have been experimenting with neural codecs like that.

The one I recall done by him was NNCP[1], a lossless compressor.

[1]: https://bellard.org/nncp/

Computings own Da Vinci.

Reply on news.ycombinator.com