Hacker News Reader

Comment by yokoprime | Hacker News Reader

There’s a slide saying few developers could build a whole instance of win2000, but when working at this scale you rarely build everything from scratch locally. You work on your module and the build system will have to build everything nightly. I don’t read this as devs throwing completely unchecked code into source control.

View on HN

I was a dev on the Visual C++ team at this time (1990s). Part of making largish compiler changes would involve throwing a test compiler at all of WinNT and seeing what broke in the build process. It was challenging to do so, to say the least. Just getting the whole mess set up was a struggle for me.

Of course, things got better as time went on thanks to process improvement. I started in 1991, and I remember driving over to the NT team's building with a large (for the time) hard drive to grab a physical copy of the source tree. This was before NT was first released - when you tried running your build and went to shut it down, you had to watch the activity LED on the drive flash a few times to be sure the cache had synced to disk and powering down was safe. Fast forward a few years, and building all of WinNT was more routine, to the point it was just another component built by the automated VC++ checkin procedure (we called it submitting code to The Gauntlet), along with Excel and other Office components.

I might be misremembering if NT was part of Gauntlet, but it was definitely something we could and would build as desired.

Certainly building everything from scratch is time consuming, but why should it be difficult? Windows 2000 was developed before I became a programmer, but not too many years later (2003) I was a teenager running Gentoo, and while it was time consuming to build everything from scratch, it was easy and reliable. It was also quite easy to switch out or hack on individual parts. In particular I remember fiddling endlessly with the kernel.

Windows 2000 was certainly complex, but was it really substantially more complex than a full Linux distribution (including compilers, desktop environment, office suite, etc)? Why was it so difficult to build from scratch?

I have no information here, but I wouldn’t be surprised if in 2003 Windows had an order of magnitude more code in it than a Linux distribution. Don’t forget, back then, Windows NT had support for these runtimes: Win32, Win16, DOS, and POSIX. Also, Windows had drivers for a lot more hardware than Linux did. Add in all the management stuff (Active Directory came about around this time), and I think it almost had to have substantially more code than Linux.

Why would those runtimes be particularly large? I'd expect the Win32 runtime to be large, but the others should be tiny by late 90s standards. Also, I remember that Windows didn't embed that many drivers itself, but that it had a stable ABI that hardware vendors could target (I remember driver CDs). Further, while the Linux kernel is certainly smaller than all of Windows, my point of comparison was compiling an entire desktop Linux system, which also included a full set of compilers, a desktop environment like KDE, office suites, multiple browsers, etc. Someone working on Windows 2000 shouldn't have to compile all that stuff.

My main question is also not so much why compilation of Windows should be time consuming, but why it should be difficult.

> Also, Windows had drivers for a lot more hardware than Linux did.

Did it? It had more third party drivers, but did NT itself build in that many?

NT had a HAL and that's it, with VERY few drivers on its own.

I would think/guess you weren’t fully building from scratch, but from intermediate snapshots. If Linus pushed a commit, would your full build, minutes later, use it?

Also, the first beta of NT 5.0 shipped in September 1997 and it was renamed to Windows 2000 in October 1998 (https://en.wikipedia.org/wiki/Windows_2000#History), and 1998 vs 2003 is about 3 times 1½ years, so, at the time, about three performance doublings.

Chances are your hardware was at least 5 times as powerful as what the early Windows 2000 engineers used.

Back in 2005 or so i was building freebsd, x11 and kde from scratch without issue. But these os’ are better engineered as there are no managers getting in the way.

> I would think/guess you weren’t fully building from scratch, but from intermediate snapshots. If Linus pushed a commit, would your full build, minutes later, use it?

Yes, I would frequently download e.g. new kernel release tarballs (this was before Git) and slot it into the system. This didn't require recompiling anything but the kernel. Actually installing Stage 1 Gentoo required compiling everything (although it was on top of a compiler binary for bootstrapping.)

My hardware was cheap 2001 era consumer hardware, so I doubt it was that much faster than what the Windows developers had available. Besides, my question is more about why Windows (or anything else) would be difficult to compile, rather than just time-consuming. The nice thing about recompiling an entire operating system from scratch is that there are no external dependencies, because you're building everything! (Except the bootstrapping compiler, but for the Windows operating system there's no reason to rebuild that.)

Imagine you're a teenager building gentoo, except every single component of the system is a random development snapshot instead of a tested release.

Linux distributions put together the products of a a huge and very loosely organised community, and I never heard it was anywhere as horrible as these (old) horror stories of Windows development. Is it perhaps the case that the (enforced) loose coupling of open source development prevents the aggregation of build complexity and intermingled dependencies?

> Is it perhaps the case that the (enforced) loose coupling of open source development prevents the aggregation of build complexity and intermingled dependencies?

I think that's a huge help. I think that it's also helpful that the system is intended to be compiled by a bunch of other people and the code is released with that in mind.

The biggest reason is that you need to pull in code you don't have access to (i.e. the DRM media modules, or PatchGuard, which is restricted to only engineers that work on it), as well as the sheer size of all of the localization content to build every language of the OS (remember i18n content isn't just text, it's images that contain text too).

There's no reason to build everything from scratch, it's like working on a patch to e.g. KWrite, and deciding to build the kernel in order to do it. If you're working on a Windows component, you install a daily build so it's close to your equivalent of main/master, write your code, and overwrite the binaries on your test machine / test VM. Your development loop is pretty fast in practice

Reply on news.ycombinator.com