Hacker News Reader

Comment by carbonatedmilk | Hacker News Reader

This makes entire sense to me: When you're going to stream the episode of Stranger things to 15million people, who really cares what the one-time cost to encode is? Surely you'd take algorithmic complexity that was 100x or even 1000x the baseline if it provided you with even a few kB of bandwidth savings?

View on HN

You still care, since at some point the cost of the extra encoder-hours used is going to outpace the bandwidth savings. I'm guessing she mentally assumed 'at no bandwidth savings, how much margin do we have?', since the amount saved wasn't ever quantified.

It's not bandwidth savings. It's growing the user base. There are lots of people on low bandwidth connections. Maybe now they don't find Netflix watchable, because the video quality is to crappy. But if you can fit higher quality video into the same bandwidth, they'd become users.

I agree with your overall point though, that at some point the extra cost for encode isn't worth it.

Netflix is in fact working very hard to optimize the quality per bandwidth at low bitrates. Megha Manohara from Netflix had a conference talk about it last year:

https://www.youtube.com/watch?v=S9E9xUbcYAM

Their goal is to have video of acceptable quality at 250 kbit/s, which makes Netflix an option for a lot more people worldwide.

That's part of it, but I think you'd be astonished at how much IP transit can end up costing when you're distributing high-quality video. It's pretty staggering how much bandwidth can cost. I would not be surprised in the slightest if at least 1/4 of Netflix's total engineering spend is on network transit, so any savings that are per-video can have a huge impact.

Source: I work at a company that distributes a lot of high-quality video :)

There are a couple of things that make Netflix somewhat unique.

1) They have a fairly limited catalog (contrast with the constant ingestion of new content by Youtube, for example)

2) The cheapest way for them to ensure they have sufficient capacity to handle peak loads leaves them with a lot of extra compute during non-peak times. That excess compute is essentially free.

Re #1: It's not catalog size that matters; it's more like revenue per minute of video, or views per minute of video

Re #2: This is a fairly standard batch-vs-online compute mix tradeoff faced by large enterprises.

>That excess compute is essentially free.

Since they host on AWS, wouldn't they just scale down during non-peak times and save money?

The AWS-hosted part is not the CDN afaik, which will probably make up the vast majority of their infrastructure.

If you're talking about OpenConnect, then I think they are really geared to just be a CDN and not much else.

Speaking as an ISP: Netflix traffic comes from the Netflix ASN and peering session. Their globally distributed CDN/caching/video file storage servers are not on AWS to the best of my knowledge. Netflix traffic does not come from our peering with Amazon's ASN.

They do, but as any large AWS customer you want to reserve a lot of capacity, so that it's there when you need it and also it's a lot cheaper than spot usage. So, scaling down below the reserved capacity doesn't save you much if anything.

Netflix has an internal spot market for compute, letting them use their reserved instances somewhat optimally.

For what it's worth, I don't know what YouTube's actual constraints look like, but it doesn't seem implausible that they might still be open to reencoding their top 10k video. After a billion views, you might get some substantial savings.

EDIT: After all, they did implement VP8/9 ;)

Exactly this. I don't know how many times this need to be repeated. If your original encoding cost were 1 Million, 5x or even 10x is only 9 million more. At 100x that is 99 million. 1000x is a billion. Are you really willing to spend a billion on a codec that saves you 50% transfer and a fee million of royalty cost?

Especially when Netflix already has the Open Alliance in many ISP DC. Bandwidth cost saving are much less that most estimate.

However I do understand her point, she should ask I give you 100x complexity headroom, can you get me 80% reduction in bitrate compare to HEVC.

Lowering bandwidth requirements not only lowers bandwidth bills, but also opens new markets. You suddenly have subscribers that you otherwise woudn't have. There's a long tail of people with slow connections.

Netflix is probably at the extreme end. Remember: the same codecs are used to compress the videos your cell phone takes.

Hardware implementations of codecs can be made that perform encode at low power if a certain level of quality/efficiency can be given up.

This is already the case for AVC/HEVC on mobile devices where storage and power considerations overwhelm the possible quality and coding efficiency advances that could be available from a highly-intensive, highly-tunable CPU-based encode.

In comparison, if you look at digital cinema, video is dumped as raw, unencoded pixels onto high-speed, high-power SSDs without any compression; this way, the original image can be manipulated losslessly before going through an intensive encode process for an optimal quality/space balance for end-user media delivery.

Right, but designing a codec that is scalable in complexity to accommodate real-time hardware encoding on one end, and 100x Netflix encoding on the other end probably isn't easy.

Why does it have to be the same codec?

Use the right tool for the job.

The encoding part may well differ, but having the same bitstream format and thus being able to use the same decoders for a very wide range of efficiency/time trade-offs has huge advantages. The necessity of using hardware implementations of video decoders in many scenarios requires a big initial investment into a new codecs' decoding part before even the first frame of video may be displayed by it.

This is why pretty much anyone has standardized on h264, and now on h265, as the video codec of choice, despite using very different encoder software or hardware to realize hugely differing trade-offs on the encoding side of things.

The very popular RED digital cinema cameras have always shot in compressed form using a low-latency wavelet-based codec (presumably with some ASIC or GPU acceleration on board).

http://www.red.com/learn/red-101/redcode-file-format

http://www.theblackandblue.com/2012/05/02/shooting-red-epic-...

What cameras shoot uncompressed 8K HDR video? That doesn’t even seem possible given current SSD bus speeds. Note that many high-quality lossless formats are actually still compressed (they subtract current frame data from the previous frame and then apply LZ4 for example).

> What cameras shoot uncompressed 8K HDR video? That doesn’t even seem possible given current SSD bus speeds.

12-bit bayer RGB matrix 8k @24 fps: 7680 * 4320 * 12bit * 24/s = 1.2 GB/s uncompressed. Currently SSDs go up to 3 GB/s. And you could have a RAID 0 array made out of multiple SSDs.

So it is possible. Entirely another matter whether it makes sense.

One hour of video at this bitrate requires 4.3 TB.

Did you forget that there are multiple color channels? 12 bit should be 36 bit so even more is needed.

No. That's how practically all of the cameras work. Google RGB bayer matrix.

There's just one color component per pixel, thus 12-16 bits per pixel is enough.

Netflix also encodes to hundreds of combinations of bit rates and resolutions. If improving the quality for a given bitrate lets them reduce some of the complexity (by shrinking the combinations needed to serve users), then there are pretty nice advantages there.

Bandwidth is extremely cheap, so 15 million by a few kB is just 15+ GB, which costs pennies in terms of bandwidth. That doesn't buy you that much encoding compute time.

Bandwidth can be cheap, all depending on the connection you're using. It can also be extremely expensive:

[ Dutch ] https://www.telegraaf.nl/vrij/2167192/zoontje-duitsers-kijkt...

If your Dutch isn't up to this let me summarise: A German family took a boat trip from Kiel to Oslo. Their 12yo son watched a few movies (or clips, no idea) while on board. Total data consumption was 470 MB. They got a bill for €12.000,- which comes down to just over €25 per MB.

Bandwidth can be very expensive as well. Of course this is an extreme case but any byte shaved off an encoded stream can end up saving some people a lot of money.

For you, perhaps, it's cheap. For the use cases they are talking about, it's expensive and limited.

Agreed that, if all they can do is shave a "few kB" off a video, it's probably not worth the investment. But what if they can buy a 5–10% bandwidth improvement that actually looks better on the device they're targeting?

Netflix isn't necessarily trying to reduce the bandwidth on _their_ side. They want to be able to deliver higher quality to users on lower bandwidth connections. The better that goes, the larger their prospective customer base.

That's a big issue in rural areas with microwave links. Everyone who has one of these links is limited by bandwidth and they all want to watch Netflix.

I wonder if there's a market for a "Netflix ISP" - essentially setting up an antenna for local WAN and a microwave antenna, and then asking Netflix for one of their caching appliances.

Reply on news.ycombinator.com