Hacker News Reader

Comment by jsheard | Hacker News Reader

Most in-house game engines built after a certain point use a non-trivial amount of third party code, including console stuff which is under strict NDA, so it's a huge hassle to open source them. Most iterations of the Source engine use Havok physics for example.

IdTech probably was only open sourced because Carmack pushed for it, but it helps that IdTech of that vintage was all in-house code exclusively targeting the PC. I think the only thing they had to cut out for legal reasons was the patented shadowing algorithm in Doom 3.

View on HN

Id were using a 3rd party sound library for Doom to handle the nitty-gritty aspects of DOS sound hardware, so the open source release was based on the linux port of Doom. People had to port it back to DOS.

They couldn't release the windows port of Doom either, as that had been done by Microsoft, and would therefore include Microsoft copyrighted code.

With Quake, Id did their own windows port, so it was possible to release the source code for winquake.

> as that had been done by Microsoft, and would therefore include Microsoft copyrighted code.

How does that follow? Normally copyright would transfer to the company paying for it.

It all depends on how the contract between Microsoft and Id was worded.

If Id had paid Microsoft to do the port, it would automatically be work-for-hire and the copyrights would belong to Id, unless the contract said otherwise.

But Microsoft essentially licensed the Doom source code and ported it themselves, it's not work-for-hire, so the copyright on any changes would default to Microsoft. Though, I'm really not sure if Microsoft paid anything for this licence, or if there even was a contract.

This is a comprehensively incorrect explanation of copyright and contract law.

> If Id had paid Microsoft to do the port, it would automatically be work-for-hire and the copyrights would belong to Id, unless the contract said otherwise.

Nope, not true. Work by non employees is only work-for-hire if it falls into one of the qualifying categories and the contract explicitly says that it is work-for-hire.

It's definitely a topic for a CSI: HN, but I think it's a was most likely a part of a larger agreement between a relatively small shop and a behemoth.

It's not automatically work-for-hire; that has to be stipulated in the contract. Recently there was a lawsuit about this involving, of all things, Sonic the Hedgehog. An artist and writer named Ken Penders, working for Archie Comics on Sonic comic books, sued his former employer claiming ownership over certain characters created for the comic, most notably those connected to Knuckles the Echidna. Because Archie couldn't prove that Penders's comic work was work for hire (they lost the original contract), they settled out of court in favor of Penders. This ended up losing the Sonic license and had various effects on the Sonic universe. For example, he threatened to sue Paramount over the idea of Knuckles having a father, because Knuckles's Archie Comic universe father was a Penders creation. So rather than depict his father directly for the Knuckles series, they represented him with a puppet in an abstract sequence -- a puppet bearing a suspiciously Penders-like moustache.

Microsoft weren't hired by Id to port Doom. They did so exclusively for their own purposes: to show off Windows 95 with DirectX as a performance gaming platform. They licensed the Doom engine, made their own changes, and released Doom 95 usable with a retail copy of the game assets.

Oddly enough, the project of "porting Doom to Windows" was started by Gabe Newell, who worked at Microsoft at the time,

The earliest(? - who knows) release of the port is from 1994, ran on 3.x and used the WinG and Win32s API:

- https://www.doomworld.com/idgames/historic/wdoom

- https://www.doomworld.com/idgames/historic/wdoom2

- https://virtuallyfun.com/2011/03/29/windoom-wing-win32s-on-w...

I knew Gaben's initial project was for 3.x and WinG, but I didn't know it was released!

Ah, that makes sense, thanks.

The wording used in the release notes was:

> The bad news: this code only compiles and runs on linux. We couldn't release the dos code because of a copyrighted sound library we used (wow, was that a mistake -- I write my own sound code now), and I honestly don't even know what happened to the port that microsoft did to windows.

> Still, the code is quite portable, and it should be straightforward to bring it up on just about any platform.

It seems like they just didn't have immediate access to the code for the Windows version. The DOS source code eventually leaked a couple years ago along with the code to the Mac port of Doom. https://archive.org/details/doom-mac-source

Fun fact: Gabe Newell built the original WinDoom port, and led the team that built Doom95!

That IS a fun fact.

To be clear, Robert Hess was the primary engineer on WinDoom, Newell was the team lead.

I thought Monolith worked on doom95.

Even the original Doom source code release was the Linux source code because the Dos version of the game uses a 3rd party audio library, DMX.

I believe DMX was also used for Descent, and the company successfully shut down open source projects to reverse engineer the file format. Interplay was also extremely protective of their full-motion video codec, which only got well-documented after the release of Baldur’s Gate II.

I recall Ton Roosendaal saying something similar about the origins of Blender: convincing his then-employers to let him open source the original abandoned engine was a huge hassle, but the lack of third party code blocking it helped a lot.

I had similar experience working for BigCo.

On one hand, at one point we had CEO and CTO publicly stating in townhalls that they want us to be a technological forefront and looking for ways to showcase our capabilities. At the same time it was virtually impossible to open source even something as trivial as a string formatting library, because once you started talking to stakeholders about approval their POV was that it's part of a project that took X amount of employees Y amount of time and that equation is in millions, so that has to be really valuable.

Middle managers look at that XKCD comic[1] and say "we need to buy out that guy in Nebraska and monetize it".

[1] https://xkcd.com/2347/

strong leftPad vibes here

hell, when I joined the frontend circus in 2013 I was flabbergasted that most people needed to install 3rd party library to perform "isEven(num)" operation

... not to mention that aforementioned library had at least 3 dependencies itself.

> non-trivial amount of third party code, including console stuff which is under strict NDA

This just seems like a lazy excuse. Ok so some of the code can’t be released. Fine, what stops you from open sourcing the rest of it that isn’t licensed? The OS community will surely fill in The blanks.

exactly.

I do think that the source code of source was leaked. Its just that valve has to themselves give the code of source. FLOSS community would really do it. Trust me.

Well for the FLOSS community to do anything Valve would not just need to release the coede but also put it under a free software license, which isn't the case even for the game code released here.

Modders in general won't care though.

Well in a sense you are right that it is lazy, in that it would be quite a lot of work to disentangle those parts.

So they could probably do it, but it would be costly, while just releasing the TF2 code is basically free.

Don't bother with subtle disentanglement. Just take an axe to it and release a broken mess. It will be fixed and running within a month.

It depends on how the code is structured. It might be that the licensed parts are sprinkled throughout the functions, or that they would have to reorganize things to not reveal too much about a specific NDA'd technology (like a console SDK)... The resulting code might end up being just as useless as no code at all, and I assume it would be very hard to get the management to allocate resources for that.

Perhaps, but a multiplatform codebase won't call the console SDK directly from hundreds of random functions. And even if the developers were that crazy those calls need to be excluded when building for other platforms so can also be stripped form the source code in an automated fashion.

> Perhaps, but a multiplatform codebase won't call the console SDK directly from hundreds of random functions.

You’d think so, but some developers have a very adversarial relationship with abstractions, such as abstracting away platform-specific code behind a common interface. It happens more often than you’d think.

> And even if the developers were that crazy those calls need to be excluded when building for other platforms so can also be stripped form the source code in an automated fashion.

So they’re going to leave people code that either doesn’t compile or, if it does, doesn’t work? And what’s stripped out may have very important technical considerations that might only be known based on which function calls were yanked out.

> So they’re going to leave people code that either doesn’t compile or, if it does, doesn’t work?

I think that in general, potential open source developers would not care at all about that. Some code (that doesn't yet work) is better than no code. As someone else mentioned, a motivated open source group could probably easily fill in all the missing gaps within a month.

There is always the risk that some of the code (non 3rd party) may look a lot similar to code of a previous employer of the person that committed it

It's all legal risk with little commercial upside

If a team had access to some licensed source code related the third party component (e.g. restrictively licensed/NDA'd sample code), clearing the engine source for release may involve a thorough code audit. Who is going to put in that work to prepare a freebie?

Source does, although within the last year they replaced Miles for MP3 and Rad Games Tools for Bink with tinymp3 and free WebM library.

Probably other stuff there but not sure off the top of my head.

You see WebM in quite a few projects that have had Linux ports. Apparently Bink requires separate licenses per platform. OGG is also popular instead of MP3 but with the MP3 patens expired that isn't a legal concern anymore, just a technical superiority one.

Oh, I didn't realise separate platforms required separate licenses so that makes sense.

And yeah too much effort to re-encode all the game audio most likely.

> Most in-house game engines built after a certain point use a non-trivial amount of third party code, including console stuff which is under strict NDA, so it's a huge hassle to open source them.

reminds of the whole "opensourcing solaris" drama that Bryan Cantrill talked about in his speech at usenix some ten years ago.

i wonder if the culprit is i18n again, "lol"

Not just that, they had specific renderer backends, one for GeForce, one for GeForce3, one for Radeon 8500 that they had to cut out as they used proprietary information or code perhaps.

Correction, I didnt remember correctly as those are actually there in the code release, here is the r200 (radeon 8500) renderer specific code:

https://github.com/id-Software/DOOM-3/blob/master/neo/render...

Even for the source release we have gotten it's not too uncommon that some proprietary middleware is missing and needs to be replaced before being able to build the code.

> I think the only thing they had to cut out for legal reasons was the patented shadowing algorithm in Doom 3.

I remember that being somewhere in the neighborhood of 1 LOC to fix, and exactly where to change it was “common knowledge” the moment the source came out

It's quite silly that depth fail shadow volumes were ever granted a patent.

ah, that NDA stuff is interesting, that probably explains it.

I wonder how UE manages it. It compiles for consoles but you can download and modify the engine source. The license isn’t open but you can still get the source.

Not sure about UE but I’ve worked on other engines and usually you just have an interface that abstracts platform APIs and each platform implements the same interface

The implementations get compiled in based on whatever platform you’re compiling for using a bunch of #ifdef statements

In that case it’s just a matter of not including certain .h/.cpp files

The source you can easily get access to doesn't include any of the console stuff, you have to prove you're licensed to develop on those consoles to get access to that code.

Makes sense. Definitely something valve would have to deliberately set up and it probably would be more complicated than it seems at the outset.

that and the hodgepodge structure of the codebase is probably embarrassing

Reply on news.ycombinator.com