Hacker News Reader

Comment by kragen | Hacker News Reader

The XVideo extension, commonly used by MPlayer (at least in the past, maybe still?), has supported sending YUV images to your video card for 20½ years, since XFree86 4.0.2: https://www.cs.ait.ac.th/~on/mplayer/en/xv.html

There are various problems in XVideo that make it hard to use for general application display, but the fact that this was a thing they included in it makes me think that a substantial number of video cards have included YUV decoding capability in hardware since the previous millennium.

View on HN

Since I just learned the hard way that "XVideo" can produce unrelated results if you put it into a search engine, here's a direct Wikipedia link for those who want to know more about it: https://en.wikipedia.org/wiki/X_video_extension

Oops. Sorry. I didn't think of that. My search was [yuv xvideo driver].

YUV has been the standard video Color encoding on TV. It is in fact a backwards compatible extension to black and white as the wikipedia page puts it: https://en.wikipedia.org/wiki/YUV

while not standard, writing a shader to decode is not hard(admittedly if you know shaders)

The time I spent dealing with video decode/encode was not pleasurable but interesting.

The really hard part is the actual encoding/decoding algorhtims

YUV was the standard in PAL, NTSC used something called YIQ.

Is that why the industry joke was the NTSC acronym meant “Never The Same Color Twice”?

I don't think so. I think NTSC's color issues (compared to PAL) were due more to how PAL alternates the phase of the lines (PAL = phase alternating lines, btw). The alternating phases or PAL had the effect of canceling out transmission errors (in the colors), providing more stable color than NTSC. This is somewhat similar to how balanced pairs in ethernet cables cancel out transmission errors. (see also: balanced lines in long pro audio cables or differential pairs in PCB routing)

I'd add the relative lack of calibration on NTSC as another huge factor. When you have TV manufacturers putting out sets with completely different defaults on top of broadcasters not being able to calibrate to any sort of standard TV appearance, you'd see a lot of different results per-station AND per-show.

It's similar, but note that what differential-voltage signaling cancels (as in pro audio cables, PCB routing, or ethernet cables) is common-mode EMI; but, as I understand it, the differential-phase signaling in PAL instead cancels out errors introduced by dispersion (where the phase delay varies significantly over the 6MHz bandwidth of a TV channel).

So it's analogous, but there's a critically important difference. (Not that you claimed otherwise.)

> while not standard, writing a shader to decode is not hard(admittedly if you know shaders)

Ah, the old "Step 2: Draw the rest of the owl" scenario.

We'll leave that as an exercise for the reader

I may be conjuring things, but I think many early video cards actually outputted a TV or component video signal, which uses YUV (=YCbCr?) color, so it is RGB that would have required conversion, not YUV.

Video codecs generally use one or another type of YUV format so lots of display hardware can already use it, on something like a mobile SoC or settop box no one can afford the memory bandwidth to do the conversion anyhow.

Carmack is giving an extremely simplified version of the world as it exists today. Your hardware can already use YUV formats, maybe it is even outputting YUV because you chose a resolution and bit depth that forced it to for bandwidth or clock constraints. Complex apps like browsers that do their own compositing can already choose to use YCbCr formats when supported for textures.

Overlay support tends to be very limited - it's effectively blitting to the final framebuffer after compositing, which means no transparency and a limited number of surfaces. The desire is for the compositor or any other shaders to be able to read YCbCr surfaces natively, converting to RGB as a part of the sample.

Namely, VK_KHR_sampler_ycbcr_conversion. I'm actually not sure if other APIs provide the equivalent yet, the HW capability is pretty recent...

Yeah, those are some of the problems I was thinking of.

There’s GL_MESA_ycbcr_texture for vanilla GL and GL_EXT_YUV_target for GLES, but I’ve no idea how well-supported these are. (Notably, the Mesa extension is ancient—the changelog says it’s been published in 2003.)

IIRC GL_MESA_ycbcr_texture is basically GL_APPLE_ycbcr_422, only supports packed 422, and was primarily intended for letting a compositor pass a 2vuy/yuvs surface to the overlay HW without actually converting it in GL.

GL_EXT_YUV_target is close, but it doesn't transparently include the specific step of YUV -> RGB conversion, just the sampling multiple planes + upsampling for chroma (and convenience functions for shader conversion.) (I think? actually not certain, since I can't find definition of how chroma is upsampled)

Well, I guess Vulkan's isn't completely transparent, since the conversion sampler has additional restrictions as to how it can be used. But at least with it shaders can be written to not care whether the source is RGB or not.

Overlays and GL/Vulkan extensions aren't strictly necessary, though support ultimately depends on the driver. For example, glamor does the conversion for some formats in a shader: https://gitlab.freedesktop.org/xorg/xserver/-/blob/8274dd664...

Android has had ColorMatrixColorFilter[1] since the beginning and YUV -> RGB conversion is actually an example use case they specify. Problem is, however, that many YUV formats do chroma subsampling, and you can't exactly put that into a regular bitmap without shuffling a lot of bytes around, which kinda defeats the purpose.

[1] https://developer.android.com/reference/android/graphics/Col...

> makes me think that a substantial number of video cards have included YUV decoding capability in hardware since the previous millennium.

You are correct, and you can find details on that by reading the card documentation (as far as it is available). That was in particular the case when cards had a fixed-function rendering pipeline.

With the advent of more general purpose cores on the GPU for the rendering pipeline, YUV decoding has been offloaded to them.

And emulators like Mame too I think.

Yeah, you need YUV to emulate NTSC or PAL.

You need it for a CRT shader, but games pretty much universally were in RGB internally and got converted on the way out. Especially for MAME, which is emulating games that would use RGB monitors.

For arcade, yes, but MAME does do consoles as well in both NTSC and PAL formats. MAME has a pretty high-quality MegaDrive/Genesis driver, for instance.

Genesis was RGB internally and supports RGB out, like I think every console generation after the Atari. PAL conversions were just about framerate.

https://www.retrorgb.com/genesis.html

I think MAME emulates some hardware for which NTSC artifact colors were important:

https://wiki.mamedev.org/index.php/Driver:Apple_II (which had no RGB at all)

https://old.reddit.com/r/OpenEmu/comments/hhxnry/sharing_my_... (showing Sonic, among others, though I'm not sure if this is really about NTSC artifacting or just dithering)

http://forum.arcadecontrols.com/index.php/topic,152465.msg15...

https://emulation.gametechwiki.com/index.php/NTSC_Filters

So, there are some exceptions, including a few extremely popular platforms, but certainly you are correct that emulating the majority of game platforms does not need NTSC artifacting.

Does Wayland support this?

Reply on news.ycombinator.com