Would be interesting to know what business considerations lead to the decision to open source MoonRay.
Over the years they've bankrolled a lot of development around Linux on the desktop and have one of the most astonishing NFS infrastructures I've ever directly layed hands on.
They've talked about their love of open source over the years, especially at Red Hat summit.
(I spent quite a bit of time working with them and their former CTO is currently a teammate of mine).
If you can't, legally, nbd.
https://www.usenix.org/legacy/event/lisa10/tech/slides/kamat...
Now, switching gears to talk about that presentation. ;)
I'd call out a few things.
1) Note on slide 17 the call out to _multiple sites_. Yup. You got it. There is a single view of the file store. At a practical level to the end user the sync time between sites is trivial to non-existent.
2) automount, automount, automount (Slide 25). The automount configuration is stored in LDAP providing a convenient way to correlate and sync data which is already integrated with the name service switch/NSS (man 5 nss).
3) Caching (Slides 22-24). This is where a lot of the magic happens. It's this cache tier which _presents_ as if it's a single file store.
One conceit I will make (which is glossed over a bit) is that the use of DNS and suffix searching is critical in a situation like this. It allows each site to use configuration management to push out relevant config files (e.g. /etc/resolv.conf) which can then present a search order like "site1.us.example.com us.example.com example.com". Thus, when you do a query for the unqualified name "ldap" (or even better "IN SRV _ldap._tcp") NSS will hand that off to the resolver which will dutifully check for the following (in order) until it gets a positive match or fails: ldap.site1.us.example.com, ldap.us.example.com ldap.example.com. This allows for a DNS based fall back as well as the added benefit of always hitting the local system (instead of a remote site).
Also, make sure you check out the stats on Slide 28.... 4 PB... in 2008... before they had to render a movie for each eye... (3D animation!)
I speculate that having this open source increases its popularity, which makes more likely that any given potential hire or VFX company will have experience with it, which makes it easier for DreamWorks to ramp up new hires or contract out to third party companies.
When I was at EA, we switched from a proprietary UI tool (which was quite well suited for consoles) to Flash (which wasn't) entirely because it eased staffing problems even though the technology itself was a worse fit. The best tool is often the one that people you can hire already know.
It might even still be in use at Cartoon Network, it's been a while since I last heard from my former co-worker in the dialup-delivered Flash cartoon mines who ended up as the animation director for Foster's and Titans but he sure was still using an old version of Flash/Animate long after the browser plugin was retired.
There's probably a lot of lessons in Flash's status as a tool that did a ton of different things half-assedly.
I remember someone at EA talked to Adobe about getting access to the Flash (.fla) file format so that our UI tools could read it directly to export into the in game format instead of reading the already compiled and compressed .swf format. And they were like, "Uh... you really don't want to do that. It's like a memory dump of the entire application state. It's a giant mess. We're really sorry."
By all accounts, it seemed like the Flash codebase was a dumpster fire. But so many amazing little games and UIs were built using it.
But there's a huge advantage to being the first high-end renderer to go open source, so lots of small studios will adopt it if it's as solid as it looks.
It'll become available in Blender.
Maintenance will then get extended to the community, including (most likely) support for alternate shading schemes (MaterialX/Lama has decent momentum right now, converging with Sony's OpenShadingLanguage).
Apache 2.0 is an ideal license for this kind of project.
At the same time however, does their renderer give them that many advantages? As someone who works on a (sort of) competing proprietary renderer, it's a lot of work and effort to do it and support it, and maybe they want to build a community around that from smaller studios and compete with Renderman a bit for mindshare?
If every animation studio started using MoonRay because of this, they'd have won.
Surely the win for DreamWorks is the competitive advantage of having aspiring creators be familiar with their pipeline. If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null. (The only way this isn't true is if you subscribe to the belief that more MoonRay use → more MoonRay experts → more DreamWorks movies → more profit. That's probably not sound. This probably won't impact how many movies they put out, just how good they look (after benefiting from open source work) and how easy it is to hire for the movies that they put out.)
They're in the movie business after all - speed of production saves a lot of money.
Other studios could also contribute improvements, which the original studio could benefit from. That's one of my motivations for open-source: I'm selfish and I enjoy the benefits from the efforts of others making my software better, should they find it useful.
(Note that this could mean that if you thought your software was putting you at an advantage, you might not want to open-source it, but you could also believe that even if it is better, your competitors won't / can't switch to it).
But if the improvements to MoonRay drive down production costs for movies, the entire industry wins.
When I was on the design team for the Lucas Presidio Campus, we were collapsing all the various divisions to the new campus, aside from Skywalker...
I was the LV/Datacenter/network designer for the project.
One of the early design requirements was to 1) have fiber to the desktop and 2) have every machine on an interconnect matrix which brought the edge desktop connections as close to the core as possible such that every workstation would become a part of the ender cluster when idle/at night/on-demand etc...
This didnt happen but it was a goal back in ~2004
--
I recall them telling us when Jobs came to visit and they had the "Death Star" <-- render cluster they were working on... Lucas apparently saw no future and sold it to Jobs for ~$10mm?? and thats where Pixar got it start.
That was an amazing fun project to work on.
---
Oh - and during the design sessions, Cisco borked the design review for their (joke of a response) to the RFP, the then CIO for LucasArts in this design meeting with ILM, Lucas, Cisco others -- he says to the entire team in dead seriousness
"When can I have power-over-fiber to the desktop" with a look like "Ha! GotchA!
And everyone just blinked..
Studios don't really have idle hours on their render farms.
EDIT: I thought you were replying to my comment about the Lucas Presidio Campus
They also weren't really set up for the kind of strict isolation and security that offering resources to external parties would require.
Translation: we think making this open source will bring in significant outside contribution so we can get a better renderer with less investment ourselves while not having to pay external providers
Open source is a good strategy for the runners up trying to compete, you get the good will from giving something away in exchange for other people improving your product for free.
They open something up, let people use it, and hope that some people might contribute back. Cool, sounds great to me. They aren't forcing me to contribute back to the project. Seems pretty win-win.
You're not going to get a free work force just by open sourcing your very context-specific code. Most companies/people learned this ~22 years ago.
MoonRay likely has the best volume rendering in the world, certainly that's open source.
Only Weta's internal renderer is in the same league.
That fact might have allowed them to take short-cuts.
How do you know what Manuka's capable of? :)