I got a Netflix cache server. 262TB | Hacker News Reader

I got a Netflix cache server. 262TB

old.reddit.com

67 pointswebmobdev32 comments

It would be interesting to save every single thing you ever see online to a local cache and index it using machine learning so you can search by topic. This is the kind of storage I imagine you could use for something like that.

> ever see online

Or ever seen. Screen record desktop at 1080p on codec optimized for text, maybe even grayscale with small samples of color data. Then have system index content (text, label graphics). When viewing, system will upscale, colorize, interpolate frames. Can probably throw in neural audio codec to store speech/audio.

I think there are at least a dozens of comments on HN suggesting the same and 10x more upvote. It would be a great feature to add to a NAS.

I have an idea to implement this technology ,it will just need a dns server ,routing techniques used in cdn and SSL decryption with custom cert

I do this by printing-to-PDF every web page I've ever read for longer than 30 seconds. 70,000+ files since 1997 .. its my own private internet. I take great delight in being able to grep for subjects through this directory ..

What do you use to do that?

Yeah, seconding the question as to what you use to do this?

Also I’m curious how large the collection is in total file size.

You can also run Recoll on it .. awesome full text searchable personal history and knowledge base.

grep is nice, but Recoll gives you stemming and such.

That looks suspiciously like a back blaze storage pod.

https://www.backblaze.com/blog/petabytes-on-a-budget-how-to-...

It does, but the Backblaze pod uses a backplane whereas this looks like it uses breakout cables which honestly kind of surprised me with the amount of drives in this.

Backplanes are tricky and costly and in this case you need a custom one with sufficient power delivery.

If you making a custom case for a fire-and-forget (and very rare HDD replacement, and in Netflix case - probably redundant on the node level) operation it makes sense to use breakout instead backplane.

backblaze did used cables in at least one version of their storage pod, and they surely experimented with it before that..

and since netflix box known to have being inspired by earlier backblaze boxes it is not surprising they look similar..

One thing to add.. the version that blackblaze used direct cables the reason was that the backplanes they were using where the most unreliable part.

this is something you can handle when all the servers are in your own DC like in blackblaze case.

For netflix that need to built the server and ship it to someone else it makes more sense to go with direct cables as it would be more reliable and a cable problem would only affect a single driver, while the backplane will take 5 drivers out.

also an regular sata cable is a easier part to source then a specific backplane..

Everyone’s made fair points! I stand corrected.

I would expect that fitting large number of HDD in similar sized enclosure would lead to pretty similar looking solutions, with airflow and such considered.

It is rather common that same design constraints lead to similar solutions.

This looks similar to the servers they use for their home grown CDN: https://youtu.be/tbqcsHg-Q_o

Its a cache server from netflix. So I am not suprised it looks like a cache server from netflix.

How many of these would be needed to hold their entire library?

Assuming 10-20k titles probably just one

Do they really have so few titles?

It's been a while since I last subscribed so my recollection my be fuzzy, but I remember they had easily over 500 unique sci-fi tv shows and movies alone.

They lost a bunch due to licensing over the years. Data online suggests it’s 20k globally so you probably wouldn’t need to cache all of them at a particular edge

Would that allow them to cache the full collection at multiple resolutions? I expect not.

They likely store the same video repeatedly with different resolutions/codecs/bitrates, so it might be fewer than expected

I was factoring that and error coding in my ballpark estimate

How many episodes per show?

Things to consider:

- These are cache servers they don't need to handle the whole library, just what's most hit.

- They cache, video and audio in mutiple formats

- This thing lives inside telecom infrastrure, near their users. A content available in South America should be to no use in an european location.

I wonder what CPU and motherboard it's using. Probably a 2012 or 2013 era Xeon, but would be cool to get a definitive answer on exactly which internal components were selected.

the specs are in the thread (well, not the exact motherboard, but pretty much everything else)

It’s a SuperMicro X9SRL-F and Xeon E5 2650L:

https://www.supermicro.com/products/archive/motherboard/x9sr...

Wow, thank you kindly for the info.

It's also got 4x LSI SAS9211-8I HBAs, so it's running 32 drives from the HBAs and the remaining 10 drives from the motherboard. I saw the original thread and couldn't figure out from the pictures how all those drives were wired up because I didn't know SuperMicro made motherboards with 10 SATA ports onboard. That's what got me curious about the exact MB. There's a link to imgur with the POST screen showing the model:

https://old.reddit.com/r/homelab/comments/yddi1t/what_would_...

I want one so flipping bad...

Reply on news.ycombinator.com