Netflix uses FreeBSD 9.0 | Hacker News Reader

Netflix uses FreeBSD 9.0

signup.netflix.com

161 pointsauvi94 comments

Yet you can't actually use Netflix on FreeBSD.

So much of what is celebrated as "innovation" these days involves taking something open and using it to build something closed that it almost feels unfair to call Netflix out for doing the same.

(Almost.)

This is where you usually see some silly comment like "Because the majority of users cannot use a command line and will never use FreeBSD."

Of course that majority will also never be given a choice.

Sony PlayStation users are using FreeBSD. I doubt they will complain.

Imagine, for the purposes of a thought experiment, a majority of users would be receptive to using FreeBSD. Imagine that instead of staring at a screen that says "Buffering...", the user could transfer files to the her home computer with something as simple as:

tnftp -o /home/netflix/nextmovieinqueue.mp4 ftp://netflixftpserverlocatedatisp.netflix.com/requestedmovie.mp4

And when the download completes, the user could serve the video to the devices that are connected to her WiFi router:

tcpserver -vRHDl0 192.168.2.0 80 bozohttpd -snX /home/netflix

And then she accesses the file as she would any video on the www, by pointing her web browser/http client at http://192.168.2.0/nextmovieinqueue.mp4

In this imaginary scenario, how could "DRM" be implemented without forcing the customer to use a particular OS, browser or video player, or some other closed source program?

Could it be done?

Um, if PS4 has netflix support and PS4 runs FreeBSD then ... you can, in fact, access Netflix on FreeBSD? ;-)

Except that's just a fantasy scenario and not how Netflix actually works or would work on FreeBSD.

The problem is there is no port of Silverlight with the necessary DRM. The latter is unethical in the context of free software.

However, from what I've read, Netflix intend on switching fully to HTML5 in the near future. This will make their platform OS-agnostic, while still benefiting from things like Encrypted Media Extensions (EME).

How is an open source browser like Firefox supposed to implement Encrypted Media Extensions without leaking the keys to anyone who decides to grab the source code?

In the same way Firefox implements Silverlight today: It's still a binary plugin that talks to the OS for some "secure path", but it's only the decryption in the closed path, instead of the entire playback experience. I don't think HTML5/EME will get Linux/BSD et al users watching Netflix any more than they were before.

Though breaking EME is probably more interesting for a certain class of developer than implementing all of Silverlight with DRM is.

they can't. Firefox has themselves said so.

they cannot implement eme because they are open source.

Between the two, I'd rather use FreeBSD than Netflix.

That's what I think Netflix is not even a global thing. While projects like FreeBSD and Linux will keep the platform open for next big thing.

You may be a bit confused. GNU keeps things open. FreeBSD and/or Linux ... well, I don't mean to troll. :)

I don't consider that trolling, just statement of fact. But I guess it depends on what each person considers free. GNU has a concept of freedom that it tries to protect via the GPL. Licenses like the BSD or MIT do not.

Personally, I think this licensing difference is what makes companies use code from one but not the other. E.g. Apple's Darwin and it's kernel (XNU) contains code from BSD. While Darwin is "open source", it acts as the core/foundation of a closed source OS. Had they used a GNU kernel like Linux, or GNU libraries (glibc) then they'd have had an issue or two with certain uses in conjunction with closed source software (e.g. statically linking glibc). I reckon that influences decisions on what to use.

Perhaps the best example to fit your comment is JunOS, a closed source, FreeBSD based OS.

In Netflix's case, it does seem that they modify some of FreeBSD to fit their use case, and they boast about how they will contribute back to the FreeBSD project any changes that they make. The BSD license does _not_ require them to do so, and who are we to know if they're actually contributing everything back. If they used a GPL'd OS, they might be obliged to do it, but I'm not a lawyer.

I don't believe, however, that the license played any part in Netflix's choosing of FreeBSD, and I don't believe that what your comment points out should be considered trolling :)

The GPL wouldn't require Netflix to contribute changes they make to operating systems running on their own servers, either. The GPL says you can't distribute binaries built from modified source without distributing your source modifications, but if you're not distributing binaries, you're really under no obligation to contribute your changes to anyone. If Netflix was using Linux as its server OS, it could keep its changes to itself simply by not distributing Netflix Linux to anyone outside Netflix.

To bring things a little bit further, this specificity (not having to redistribute the code when using it as a service) is precisely why AGPL[0] exists: if Netflix were to use AGPL code, and I were to use Netflix services, then I could claim the source code of the service under AGPL.

[0] https://en.wikipedia.org/wiki/Affero_General_Public_License

GPL is open for a self-selected group of developers, BSD is open for all developers and a self-selected group passes their changes back.

What is open for all vs open for some will always come down to the definition of the word "open".

If we are talking about open as in permission to use, modify and distribute, then both licenses are equally "open".

If we are talking about open as in permission to close down the permission for others to modify or distribute newer version, and sue anyone who acts different - Then we do have one license that is more "open".

One could also argue that a standard business model of today rely on being able to threaten users with lawsuit if they modify or share the software. The "openness" to use such model is only available for one kind of licenses and not by others.

I might not go as far as calling it more open, but then I have a distinct opinion on the word open.

Aw, geez, and here I tried to avoid these debates. I used "open" to try and imply that GNU's license will keep the source code "in the open", as "open" implies something different from "free". I'm not going to argue that one is better than another, or that one gives more rights to derivative users or not. That way lies politics and madness :)

GPL is open for all users. BSD is open for users of a self-selected group of developers.

GPL is only open to users who have a developer willing to distribute the code in a workable form. Given the GPL is often used in the "pay us if you want to ship" mode, I don't see much to recommend it versus BSD. GPL can die just as easily as BSD without developer support.

The morale of the society and demand makes one or the other successful. You and I feel that companies today, that means 2013, have finally decided to try OSS seriously and this is something what other companies imitate. Monkey style, the first adopter's success is giving clue for those afraid of trying new things. The corporate culture is not too big to fail too and I believe that hirings of new or younger managers like Marissa Meyer will help to kill corporate culture to some degree, even if it's not where we would like to have it. To be precise, making corporate culture an open culture. One that shares, despite the cost and risk of monkey style adaption by their opponents. This is what I'm waiting for. (Yes I know what she did, but it's nonetheless getting better for Yahoo, or hopefully so)

That means, for developers who build a startup using the XaaS business model, only MIT or BSD works really well. While LGPL and GPL may work, but require legal steps, where there is no money for lawyers and such. These startups cannot compete against the corporate enterprise market, like Google for example. And according to the theory of "The rich get richer", this will not change, unless someone clever enough, uses system theory to subvert and defeat the corporate culture, or the whole society backs this idea, ie. to stop the inevitable destruction of the remaining planetary resources. Guess why SpaceX wants to mine asteroids. It's not that resources are rare or something, they are simply rare, because someone with the budget reserved a large amount for himself, to control the market. Yes this is virtually a power Google also has, it could make the barrier of entry for any startup so hard, that those who want succeed need to be financed by them (through acquiring) or by visionary investors. Please don't get this as Google hate, it's just a logical observation and I am also doing human errors from time to time. When you find a logical flaw, I'm curious to discuss that with you.

Netflix is declining. Their selection of movies available for streaming has been steadily losing mainstream, popular titles. Now it's mostly B movies and kids shows. Probably the main thing keeping it going are the few popular television series that they still have (Breaking Bad, etc.); I can't remember the last time I actually watched one of their movies.

Their forays into original programming are pretty amazing (House of Cards, et al).

The documentaries on both Amazon prime and Netflix instant are awesome though - the only thing I watch instantly that's part of the free collection.

A lot of kids shows had a mass exodus to Amazon a few months ago. So they don't even have those anymore.

They are trying really hard to push HTML5 forward. And that will let them do such things.

They are stuck, the movie people want DRM, but nothing really exists on Linux for such. So they had to go with Silvershit.

they are pressing for html with drm so Linux and friends will still be left out with no viable legal options.

BSD vs GPL was already mentioned. Perhaps now we can discuss how DRM only hurts paying customers of protected content? Oh, you bought access to some licensed content? Great! Thank you. As a reward we will now severely limit how you consume the good or service.

From June 2012, more information at: http://lists.freebsd.org/pipermail/freebsd-stable/2012-June/...

And on the other end of the spectrum, they require you have Microsoft Silverlight available to your browser.

Thankfully, there are easily installable Wine ports so it isn't really a problem.

Netflix does not seem interested in enforcing their "Linux (/*BSD) ban"[1], I can even use a desktop recorder to rip straight from Netflix if I wanted which essentially makes it a "take what you want" service.

[1] http://www.omgubuntu.co.uk/2012/02/netflix-no-plans-to-suppo...

When are people gonna shake the false impression that *BSD are outdated and their flavor of Debuntu is far superior?

Who says BSD is outdated? I've never heard anyone say that, and I thought I'd heard all the OS whinges going.

When people also acknowledge that Gentoo is effectively the best of both worlds?

Actually, Arch[1] is the best of both worlds.

1.https://www.archlinux.org

Comparison to others: https://wiki.archlinux.org/index.php/Arch_Compared_to_Other_...

Arch is closer to Fedora than Gentoo or BSD.

Gentoo still uses a BSD Ports-like system called Portage. You can still have a sync'd directory of Ebuilds, which correspond to FreeBSD Makefiles, only you have the added benefit of USE flags, compilation options, and even binary packages.

You can run Portage on FreeBSD.

Gentoo still uses, by default, OpenRC, which is a BSD-like init system that also handles dependencies. It is compatible with other FreeBSD and NetBSD scripts as well.

You can run OpenRC on FreeBSD and NetBSD.

Arch, on the other hand, has Fedora's systemd, which is completely incompatible with any of the BSD's thanks to its use of cgroups, which are a Linux kernel only feature, and the developers of systemd have dismissed, openly, the BSDs as outdated and unimportant.

Arch also uses binary packages for most of its packages, which is closer to rpm/dpkg than it is to Portage/ports.

The AUR is a limited system designed for pulling in alternative and source packages. It lacks USE flags and compile-time and runtime end user control. The AUR is mostly managed by the community, and many AUR packages have a lower quality than the base pacman available packages. The less popular a particular AUR package, the less people will be involved in it, and the lower quality it will be.

On the flipside, the BSDs have impeccable quality packages. They're not configurable without tweaking stuff manually, but the defaults are rock solid.

Gentoo's core Portage tree packages are managed by official developers who are assigned duties. And like the AUR, the community is able to apply any number of "overlays" on top of the Portage tree, giving the community direct ease of modification of not only community packages, but core ones as well.

So, how is Arch the "best of BSD and Linux" when it has absolutely nothing to do with BSD?

> So, how is Arch the "best of BSD and Linux" when it has absolutely nothing to do with BSD?

You're being a little biased here.

Systemd has been default since 2012, yet Archlinux started in 2002. Before systemd, the default system (initscripts[0]) was very much BSD-like by design. Not anymore, that's true.

Regarding packages, you don't have to use binary packages; you can very well use the ABS [1], which AFAIK really is what ports are about:

- You get a tree where you can download recipes. They're called PKGBUILD in Arch, ebuild in Gentoo, but essentially do the same thing. Take a look at nginx's PKGBUILD [2], and you will recognize a logic similar to the corresponding ebuild [3], yet far simpler (because less configurable, I hear you coming)

- You have a set of tools to create a fake root, calculate dependencies, build the package according to global properties (there are compile flags [4], but I have no idea how they compares to the famous USE flags) and install the package.

- Pre-built packages are just packages that were built by the Archlinux team with sensible defaults and have proven to be stable enough (by Arch's standards), so I as a user don't have to bother re-compiling. But the process behind is completely done with PKGBUILDs.

The AUR is "just" a staging zone where packages get to be tested by the community at large. When it's popular enough and a Trusted User commits to maintaining the package, the PKGBUILD can be part of the [extra] repository, where all packages are automatically built and proposed to arch users... but still, with the same build process.

To me, Arch sits between Gentoo/*BSD and "higher-level" distros: it keeps the process as close as possible to upstream applications by providing a minimal framework to install and update them, so that users can get understand how gluing packages together works, and what they have to do to have a functional system that does what they want, and not spend time on understanding how the packages work inside. Arch is more about understanding and using the system as a whole, than understanding every brick.

[0] https://wiki.archlinux.org/index.php/Initscripts [1] https://wiki.archlinux.org/index.php/ABS#What_is_the_Arch_Bu... [2] https://projects.archlinux.org/svntogit/community.git/tree/t... [3] http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/www-... [4] https://wiki.archlinux.org/index.php/Makepkg#Architecture.2C...

Even on Arch, ZFS is awkward (btrfs is most of the way there but maturity is important in a filesystem), sound is still the mess that it always is on linux, and the system keeps changing underfoot.

ain'tnobody got time for that. Gentoo is cool and all but I don't want to compile everything from source.

I used to use Gentoo back in the day, then I got a laptop and was like "I won't have time to compile everything!" Then I distro hopped for a few years, never really being happy, I am now happily back on Gentoo.

Didn't gentoo do away with the stage 1/2/3 stuff a couple Years ago?

They did, but then you have to spend all of your time fixing broken packages whenever you do a world update. :P

When people will understand that the best of both worlds is what works for them?

Pshh. Gentoo is so last decade. Arch is where it's at.

FreeBSD package server was compromised quite a long time ago. Have they restored it yet? It not being there is sort of a deal-breaker for me, since I really cannot be bothered to compile all the 3rd party packages I use from source.

Yes, it's restored: http://www.freebsd.org/news/2012-compromise.html

Too bad those mods/upgrades/updates are not free software?

They are. Most companies using BSD-licensed software realize that it just makes sense to not release contributions back to the community. Having to backport whenever a new release comes out just to hide a couple shiny features quickly becomes very much not worth it. Netflix in particular is a major contributor in code, cash, and community, so saying the mods aren't free software is utter ignorance. The meme that only restrictively-licensed free software gets people to contribute back has frankly been debunked from here to mars and back.

How many companies have contributed to help OpenSSH??? Damn few.

How many companies have made changes worth contributing back, but decided to keep to themselves? That'd be interesting to know.

RedHat contributed their ssh-ldap-helper to allow ssh to effectively use an LDAP server as its authorized_keys source.

... except that they are. Scott Long and Adrian Chadd are working full-time for Netflix, and other developers (e.g., Kirk McKusick, Gleb Smirnoff) have been hired from time to time for other work. Aside from short time periods while Netflix is testing kernel patches (on live traffic!) all their FreeBSD code is in the SVN repository.

If the company is willing to contribute it back, it is probably good software and the company will probably be a good partner. If the only reason the contribute the code back is because of the license, I would imagine they will suck as a partner and the code probably won't be of good quality.

I look at BSD / MIT as foundational licenses that anyone can use. Those companies that don't understand the business case for getting their changes put back into the mainline of a project just aren't worth your time. It is a nice self selection process by companies and a good indicator if they will get the whole open source thing. Its like a spam filter for source code.

For example, Netflix is contributing source and money back.

I'm curious as to what you mean by this since it does not compute.

Ummm, it's a DRM, not technical issue. The DRM is imposed upon them by content creators. In theory, FreeBSD could get a moonlight port, then try to license the keys used for the DRM, but unfortunately, this is likely a deal breaker for Netflix. Also the FreeBSD project has to do the work, Netflix can't, since they can negotiate a contract on behalf of FreeBSD.

Moonlight is open source. Netflix could port the DRM if they wanted to; FreeBSD would not need to do it. by my read of the CC license they'd have to contribute that back to the project, but I'd guess they could then negotiate the licensing of the keys; keys are not code so I don't see any more problem with that than using your own keys in e.g. GnuPG.

So few people use *BSD as a desktop that the demand for it is not going to make it happen, however.

> Netflix could port the DRM if they wanted to

I'm pretty sure the DRM used in the Netflix silverlight player is Microsoft's "PlayReady", so no... Netflix can't port it if they want.

At vBSDCon Scott Long announced they'd been tracking -CURRENT for over a month, so assume they're already on 10-STABLE in a lot of places.

Some good details on the OCA http://people.freebsd.org/~scottl/Netflix-BSDCan-20130515.pd...

I'm confused, doesn't Netflix use AWS for all their servers? AFAIK you can't run FreeBSD on EC2 without significant challenges.

There are a few AMIs available: http://www.daemonology.net/freebsd-on-ec2/

Yes, this if for their CDN/edge system. They run their cloud architecture on EC2. There was a very interesting talk about this on EuroBSDCon (http://2013.eurobsdcon.org/eurobsdcon-2013/talks/#AlistairCr...), but I can't seem to find the video.

I believe these are the boxes you can install if you're an ISP to cache the most popular content - hence the talk of 100TB of hard drives etc.

The first time I've scrolled through the title I've read: Netflix uses NetBSD and I thought that's strange. When I had to re-read the title and realized it was using FreeBSD I wasn't that surprised.

FreeBSD is a natural choice if you want stability, even over Linux. I enjoy their update scheme although the lack of an official 'pkg' repository gives in to my nerves.

So, why is this news, is beyond me...

ps. I see a lot of people complaining that they use an open platform (FreeBSD in this case) to create a closed product. If I'm not mistaken (I'm not a US citizen and Netflix doesn't work in my country) it's a media-content rental service. So what exactly is the open source alternative, www.PirateFlixbay.org? :-)

I read it as "Freeflix uses NetBSD 9.0" (and wondered how they got at 9.0 so quickly) but that's probably just me.

"... and I thought that's strange."

Why? I believe the leader of their core group works for Netflix.

Because is the unpopular choice. I almost never came across someone using NetBSD, except myself 10 years ago.

[s]I may have misinterpreted this, but to me it seems as though all this provides is two things: 1) Netflix get some technology for free from which they can profit and 2) Netflix get to boast about using free/open software.

I think instead of a page thanking FreeBSD for software, Netflix, alongside many other large companies who use free software, could make more of an effort to actually contribute to the software proportionally to their use of it.[/s]

I apologise, clearly I had misinterpreted it and was wrong!

They do contribute money, and developers to FreeBSD. They send patches, they sponsor the FreeBSD Foundation directly, they sponsor developer conferences, and they work with companies like Intel and Supermicro and LSI to make FreeBSD more awesome. This is a page for their content delivery appliance package, and it's not really an appropriate place to discuss all the stuff that Netflix does for FreeBSD.

Page 18 of the pdf kev009 linked to in his post[1] lists Netflix's contributions to FreeBSD.

  • Camcontrol mods to download SATA firmware
  • IPv6 ref counting fixes
  • ixgbe interrupt mitigation, RX optimizations
  • Fixes for isci driver for firmware download
  • Collaboration with FF, Isilon on Unmapped I/O
  • VM/VFS Tuning: vfs.read_min

1) https://news.ycombinator.com/item?id=6949244

Netflix does contribute quite a bit:

https://www.freebsdfoundation.org/donate/sponsors

Or search for "Sponsored by: Netflix" in the commit messages.

http://freshbsd.org/search?project=freebsd&q="Sponsored+by%3...

I thought it funny that NetBSD is a donor to the FreeBSD foundation.

It is pragmatic. NetBSD will grab keys bits from FreeBSD, and will want to periodically merge changes. If they don't commit their changes "upstream"... but really more "sidestream", they will have to re-merge their local changes each time too.

I'm sure Netflix is contributing to FreeBSD. From the link posted above it states:

>We've been working hard on stabilizing the FreeBSD IPv6 stack for production-level traffic, and I recommend that all users of IPv6 update to the latest patches in 9-stable and 8-stable. Contact me directly if you have questions about this.

Surely that work has gone back into FreeBSD and is not just for Netflix alone.

yes it has. they were the first very high traffic ipv6 freebsd service. they discussed this at eurobsdcon.

I presume you mean monetary contributions? Netflix of course will not have developers with OS expertise.

Edit: wow, I'm really getting reamed for this comment.

From the link:

"We will contribute changes we make as part of our project to the community through the FreeBSD committers on our team."

They could hire a FreeBSD hacker and let him/her code all day long on the OS.

They actually have several developers on staff and most of the work they are doing ends up getting committed back to head

They have done that, more than once.

> GNU has a concept of freedom that it tries to protect via the GPL. Licenses like the BSD or MIT do not.

BSD and MIT are trying to protect their own concept of freedom.

That's interesting. Only about a year and a half ago when there was the leap second fun that disrupted so many people they were running on CentOS 5 (and so weren't affected): https://twitter.com/adrianco/status/219238574552387586

The Centos part is the control layer that runs on AWS that people talk about mostly. The BSD part is the actual video streaming.

There was a good talk about this at EuroBSDcon but the video does not seem to be available or the slides. Running the Netflix Video CDN on FreeBSD and Handling 30% of all US Internet Traffic by Alistair Crooks.

I am curious on what kind of software they are running with nginx on freeBSD or is all the video content just raw video files that their client software directs to?

And they were sponsors at EuroBSD conference this year too.

Juniper routers, the most used routing equipment after cisco, runs on FreeBSD. So everything on the internet effectively uses FreeBSD.

The same goes for VxWorks, and IOS (the cisco one).

The company that owns VxWorks, Wind River System, did involve in BSD circa 2000, and the most can be said is early VxWorks did share some lineage with BSD Unix, but ever since VxWorks has been switched to WIND kernel, I doubt we can really put these two OSes in the same group.

Now owned by Intel.

Actually the internet runs on Tcl:

http://en.wikipedia.org/wiki/Cisco_IOS

Newer Cisco IOS versions (IOS XE for example) are running as a process in linux. Old IOSes were linux-based too. Don't mix up Cisco IOS with Apple iOS (which has roots in BSD).

huh. I thought they were using a QNX based one on the new stuff.

Plain old IOS is a monolithic program and tends to be used in older routers and current small-to-medium routers & modern spins of older product lines.

IOS XR runs on QNX and tends to be used on routers in a carrier role.

IOS XE runs on Linux and tends to be used on service/aggregation routers.

Good to know. Thanks for the info.

Not sure what you mean, but VxWorks is not based on FreeBSD.

Reply on news.ycombinator.com