Hacker News Reader

Comment by barrkel | Hacker News Reader

Video compression is not understood well enough throughout the whole stack yet.

I recently got a 1080p projector for home use, so now movies / TV series in my home are viewed on a 100" screen. Content is mostly from Netflix and Amazon Prime Video.

Netflix does a really good job with encoding. I cannot say the same for Amazon Prime Video; even with their exclusive (in UK) offerings, like American Gods or Mr Robot, the quality of the encode is quite poor when viewed on a big screen. Banding, shimmering blocky artifacts on subtle gradients, insufficient bit budget for dimly lit scenes - once you become aware of the issues, it becomes really distracting.

OTOH a really big screen is a fantastic ad for high quality high bitrate content. Anything less than 2GB/hour is noticeably poor.

View on HN

A nice comparison I often do: I take a plain black image full #000, and put it side by side with a video.

Keep in mind that for historical reasons related to the way analog TVs used to work, black in video is almost never "#000". We call that setup, or superblack. In the analog days NTSC black was 7.5 IRE, not 0, which is why you could always tell when a television was turned on even with nothing on the screen. More than one digital standard kept this custom, setting black at (roughly, in your terms) "#111". So if you're seeing encoding artifacts or grayness on black video, that's more a legacy of video itself and not the encoding work done by whoever made the media.

Say I sit down at a switcher and push the Black button. You'd think nothing would come out, right? Not quite: that switcher will almost always emit broadcast black, not "null," whether it's analog or digital. Sure, you can probably get Photoshop to spit out 0% then encode in some codec that lets you keep 0% and successfully display it on a modern digital television, but the more equipment and encodes you add to a video production the more likely the signal will be pulled up to broadcast black, so it's better to just think of that as black.

There's a similar exercise with white on the upper end. Remember when analog TVs would buzz when you wired up your computer and put RGB white on the screen? That's superwhite.

Well you should still be outputting full contrast. If it's not displaying black as fully black on your screen, there's a problem somewhere, probably in your playback software.

In MPC there's an option Render Settings > Output Range > 0-255 or 16-235. When set to 0-255 (default) my videos play with full black.

That setting sounds like it's for clamping your output to drive analog televisions in case your media has supercolors in it, given its default. I mean, putting a lens cap on a camera will record 16, and a white title on black in most editors will use 235 white on 16 black. I'm not sure what you think that setting is doing in that circumstance, when playing "typical" media. It definitely shouldn't be scaling, because that would distort intended color.

A unique pleasure of HN is watching programmers try to argue with domain experts on the topic of their domain expertise.

Domain experts sometimes don't realize when their knowledge is out of date.

To wit: HDMI does allow for 0-255 (or 1023, etc) "full range" quantization instead of the default "limited range" of 16-235 for most CEA-861 formats, if the sink's EDID allows a selectable YCC quantization range and the source sends AVI YQ=1.

Or even getting to the "plain black image full #000" remark that prompted this thread, images are basically always full range where black is 0 and there is no "superblack" concept. If it's being transmitted over HDMI, it's up to the GPU to do range conversion and RGB/YCbCr conversion as necessary, and up to the OS to do ICC colorspace conversion as necessary.

> Domain experts sometimes don't realize when their knowledge is out of date.

Sigh. Another unique pleasure of HN.

Sometimes there's a difference between practical and possible. I did not say it was not possible. While you're technically correct, the worst kind of correct, very little content and few delivery systems actually utilize that capability -- look how much you had to type just to get there. Then how are you going to deliver that alternate colorspace content? To wit: I note you've overlooked ATSC and other delivery mechanisms, as well as HDMI being a specification designed to handle all cases including non-NTSC-legacy regions which had a different colorspace all along (notably PAL, which defined "blanking" and "black" as equivalent).

Practically speaking, pretty much no content you'll ever play makes use of wide gamut in North American contexts. If you're cooking a file for yourself, that's one thing, but once you're producing something that's mass consumed there's a lot of compatibility issues to worry about. We still have fractional frame rates and picture safe zones, for crying out loud, despite drop frames being pointless for decades and overscan being a word nobody under 30 has heard. What's the point of leveraging wide gamut in HDMI if all your capture equipment, editing tech, broadcast delivery, playback equipment, and displays cannot handle it? For the general case you're counting on Vizio adding that ability into a $200 TV and Comcast handling it in their CPE. Good luck.

Why does that matter? When producing video content handling a wide array of distribution media matters. Except for circumstances where you control the entire system end to end, such as a planetarium or some other kind of venue, you always have to plan for your content potentially being sent over lame, legacy-style HDTV broadcasting. That's the floor. Sometimes you even have to think about SDTV transmission; God help you, and have fun with chyrons...

Not even Rec. 2020, which is designed for UHD and is way better, changed this[0]. It's not a concept that's in dispute, despite whatever HDMI -- one standard of several hundred that are involved in delivering you a single TV broadcast -- can do.

[0]: https://en.wikipedia.org/wiki/Rec._2020#Digital_representati...

Well...

1. You asserted that 0/0% in image codecs was superblack. The most common codec for which that is actually true is WebP, which I'll bet no one in this thread actually cares about.

2. In digital colorimetry, gamut and range are orthogonal. Gamut is the space of real colors a given colorspace encompasses, range is whether the digital signal is quantized to 85% of the theoretical bitdepth and is what we've been talking about. Rec 2020 very much does increase gamut, as mentioned in your link.

3. Studios are actually starting to master in full-range 48-bit RGB (or XYZ technically, I think...) But yeah, that's irrelevant to end-users and this discussion.

4. Yes, everyone has been talking about PCs and how they render / output to TVs. That's why images and MPC were mentioned; broadcast doesn't deal with images. Computers deal with full-range RGB; that's literally the framebuffer format everyone renders everything to. As such, they convert video's YCbCr to full-range RGB before HDMI or TVs come into the picture; MPC's setting mentioned earlier is to ignore what the file says about its YCbCr range when it performs that conversion. Because while there are video files on the internet that set the wrong range, there are also internet video files that are legitimately full-range. And I'm not even talking about ancient codecs using palettes or VQ.

And since computers have traditionally outputted full-range digital RGB signals (over DVI, DisplayPort, LVDS, etc), they do indeed negotiate full-range RGB over HDMI when possible. And yes, even $200 TVs generally support that nowadays.

Heck, if you created a full-range H.264 file, tossed it on a thumb drive, and stuck it in your TV, I'd be willing to bet most if not all modern TVs play it correctly.

And absolutely, positively none of that matters to what the last episode of Scandal looks like as delivered over Netflix alongside an image of RGB black, which was the entire point of the thread. My perspective was the production of the average content that a user would be doing that comparison against, not your damned thumb drive that absolutely works and therefore, obviously, invalidates everything I'm saying. I even said, repeatedly, that the content just doesn't exist. That's all I said. Let me repeat that, just so we're abundantly clear: yes, video engineer, it is technically possible, but it is pretty much never used.

I did not assert that 0% in image codecs was superblack. I specifically mentioned Photoshop to avoid pedantry with someone coming along and saying "but nonlinear editors won't emit superblack unless you force them to!", not to talk about image codecs. You did that, and I ended up with more pedantry than I had bargained for in this entire thread, so that was a pointless exercise. As was sharing my perspective at all, honestly; I'm a professional video editor with credits (where "gamut" is a common term to discuss this phenomenon, despite your doctorate in colorimetry), not a video engineer. I apologize for speaking out of turn, and will defer to your expertise in the future.

The irony is that you responded to someone making a joke about the exact thing that you're doing. Please, just stop.

Sorry for thinking you were talking about image codecs and thinking you were confusing RGB black with superblack.

And other people who do different things have different perspectives and discussion to offer! Amazing! Are you satisfied with your correctness?

You are the reason I hate this community. "Oh, neat, a thread I know something about. Let me try contributing. Oh, look, now I'm in a slapfight with a guy who develops codecs for a living who completely missed the point of what I was saying and the context thereof, and wants to correct my usage of an incorrect term because it's his area of expertise. Let me rush to contribute more."

HN: The Game of Being More Right Than Everyone Else. Congratulations on winning. I'll go play something productive.

Edit: This was left when the parent comment said "Sorry, I only develop video codecs for a living," and it has now been rewritten. I'm leaving mine and I'd rather my whole subthread just be detached and deleted at this point

I am always reminded of this xkcd https://www.xkcd.com/1831/

You're right I don't really know exactly what the setting does.

But if a video cannot play with the black in the video being the darkest black my monitor is capable of, there is a problem there. Maybe it's some inherent problem with standards or formats or something. But if a video wants to show black, it should display on my monitor as dark as the monitor supports.

A camera module's optical black level isn't the same as Rec.709 color spec using a range of [16..235] for RGB color. https://en.m.wikipedia.org/wiki/Rec._709

I'm sorry, I'm not sure what you're explaining to me. I'm just saying recording "black" will yield broadcast black in general, without switching colorspaces and producing an entire content pipeline designed to handle an alternate colorspace. Nothing to do with the camera.

Depends on the content - HDR 4K content on both Netflix and Amazon is encoded in horribly low bitrate - about 12-15Mbit/s which makes it barely look any better than 1080p content (for comparison - 4K BluRay discs are about 60-100Mbit/s and look infinitely better).

Really? I find that their 4K content looks a lot better than their 1080p content, not necessarily because of resolution but because of H.265.

It looks better, but it's not even remotely close to how BR looks (and how 4K actually should look when bringing full resolution to the table) - even with H.265/VP9 it's quite bit starved at that bitrate.

This is probably not an accident.

A good broadband connection is limited to a sustained rate around 13-18 Mbps.

in US maybe, Im getting my advertised Mbps 24/7 as long as the source is able to keep up.

Could you share a link to the projector you bought? And what do you project it onto?

So maybe a more useful comment than video-range/full-range pedantry over historical brokenness:

Banding (which I think is the root cause of all your complaints) is effectively due to overly coarse quantization. This is amplified in H.264 (which Netflix and Amazon generally use) since the signal has multiple 8bit -> 10+bit -> 8bit trips, with each 8bit reduction contributing additional errors that can effectively result in small changes between blocks in the input signal amplified into noticeable differences in the output. And it's more noticeable in low-light scenes since current standards' gamma transfer functions compress a bit too much on the lower end (maybe my personal opinion there...)

The easiest solution is to have more discrete steps at each intermediate point. But for that to really help you'd need to change codecs from 8-bit H.264, which means not being playable on many devices. I mean, you could get 15% finer quantization with full-range 0-255 over video-range 16-235, but that doesn't help that much; you really need the 4x that 10bit gives you to be noticeable.

The second easiest solution is to have a debanding postfilter. But between the wide variety of platforms Netflix/Amazon have to run on, and modern DRM requirements, that's also not really an option.

So ultimately, you're left with developing prefilters to try to shape where bits are needed for given scenes, and careful tuning of your encoder. Which Netflix (presumably) has a several year head start over Amazon on.

Reply on news.ycombinator.com