Ask HN: Is C# suitable for creating a modern OS like Windows 7? | Hacker News Reader

Ask HN: Is C# suitable for creating a modern OS like Windows 7?

37 pointssshykes55 comments

Are C# and the .Net framework suitable to create a modern OS like Windows 7?

This is a followup question from [the now deleted] thread:

http://web.archive.org/web/20100812071342/http://stackoverflow.com/questions/783238/why-windows-7-isnt-written-in-c

What kind of problems might one expect to encounter by choosing C# over the tried-and-true strategy of using C? What kind of problems in this domain might be better solved by a high-level language like C# compared to C?

View on HN

Microsoft had this as their aim with "Windows Longhorn", announced ten years ago this month, but failed in their task and released instead Vista.

Here's some relevant snippets from http://arstechnica.com/information-technology/2011/06/window... ...

* ... Longhorn would integrate .NET into the core Windows platform. .NET FX, as it was known in the company (with "FX" standing for "framework") would give way to WinFX, a Windows Framework based on the same technology. Among other things, this would include a brand new way for writing user interfaces, codenamed Avalon, that would be thoroughly modern, vector-based, and hardware-accelerated. Windows itself would be written to use WinFX for its user-visible programs—Explorer, calculator, and so on—and, going forward, .NET would be the way to write Windows applications. Win32 would still exist for backwards compatibility, but it would be frozen and left static.

* Longhorn would have been the end of the old ways of writing Windows programs, and the dawn of a new era of modern Windows development, one that was not hamstrung by design decisions made ten or fifteen years prior.

* As we all know now, Longhorn never shipped as such. The project grew enormous, unmanageable, and broken. Around the same time, Windows XP, on which Longhorn was based, was being ravaged by hackers. Microsoft poured its resources into making Windows XP and Windows Server 2003 tolerably secure—an effort that culminated in Windows XP Service Pack 2 and Windows Server 2003 Service Pack 1—then, for the most part, started development of its next operating system, the one eventually to ship as Windows Vista, all over again.

* One of the biggest losers in this developmental reboot was .NET. Windows Vista, though radical in some ways, abandoned the entire "WinFX" concept. Avalon did ship—it's now known as Windows Presentation Foundation (WPF)—but as an add-on to the operating system, not a core part.

In many ways Microsoft in the early 2000's was a bit ahead of the trends in operating systems and mobile devices.

They probably failed because they were competing against themselves and were just too big and compartmentalized.

Very good post, thank you. I wasn't aware of the Avalon -> WPF relationship.

I'm a fan of .NET and it would have been pretty awesome to see it take such a large role in the OS itself.

I suppose WinFX's spiritual successor is WinRT.

Yes, there is no question that C# is sufficient to write an OS. However, you would need to ignore/replace the libraries that are normally used with C# applications. Instead you would write your own low-level libraries that provide only the kind of services that an OS requires. But the biggest hurdle will be the compiler. A normal C# compiler only generates bytecodes that will be run on the .NET or the Mono virtual machine engines. You will need an additional stage of compiler that either produces machine code, or something that can easily be converted to machine code such as C code or GCC's intermediate representation. It is a hurdle but not a massive one since there is a lot of support out there for writing compilers. People have recently been writing compilers for a number of languages that emit Javascript and your hurdle is no more complex.

When you write your compiler you will probably end up leaving out some C# features such as reflection, because they are too intertwined with .NET. This is common with other languages when people have tried to write a compiler that emits machine code and it does not usually cause a problem because the problem of interfacing with hardware does not demand the same features from a language as building a modern web app does.

Do your research before starting this because you will likely find a lot of projects that have already built bits and pieces that you can reuse.

Yes, it's already been done. http://en.wikipedia.org/wiki/Singularity_(operating_system)

To clarify, part of what makes this possible is that they extended C# with some attributes to specify that regions of code should run in a sort of high-integrity mode, where they can't be suspended for GC, etc. This allows them to do things like handle interrupts safely from C#. They call it 'Constrained Execution Regions'.

http://msdn.microsoft.com/en-us/library/ms228973.aspx

CER is part of the hosting work done for SQL server integration (i.e., SQL server hosting the CLR), not Singularity.

For Singularity, they used an extended version of C# called Sing# (based on an earlier research project called Spec#)

http://en.wikipedia.org/wiki/Sing_Sharp

Is it the case that Sing# doesn't leverage CERs at all, then? I thought it did.

Cosmos is more interesting. It is a toolkit to create operating systems in C# - http://en.wikipedia.org/wiki/Cosmos_(operating_system)

Looking verbatim at the starting question, it's stll not comparable to Windows 7, in a sense of all built-in functionality in Windows 7. But it is a nice and smart research project.

The Lisp machine operating system was written in Lisp. In fact, it was Lisp all the way down to the microcode, or at least, it would present the microcode in Emacs as if it were Lisp, and you could modify it there. The Lisp machine OS was similar to other OSes of the age, more capable in many ways.

A modern OS is a huge amount of work. It's so vast, your choice of language isn't going to make or break you. Moving a mountain with a tablespoon should proceed three times faster than with a teaspoon. You're still looking at an eternity if you're the only guy.

For this reason, your best bet is to narrow in. Decide what it is you want to write an OS for. Do you want to explore novel user experiences? C# is probably fine for that. You don't need to write the whole OS there, just the "shell" and the window manager. You can do that now, and not spend any time dealing with drivers. Do you want to experiment with microkernel or exokernel architecture? You'll be breaking new ground. Is C# OK for that? What you're really asking is, what are the limitations of C# that you'll run into? The GC pause is one, another is dealing with raw bytes and making it perform. If make the CLR self-hosting, you're going to find performance problems like crazy. If you make fixing that your mission, you could be a hero like the PyPy guys, and everything gets better, or you might just be throwing away a lot of work.

I happen to think a lot of interesting problems would benefit from a high-level approach. Security is a declarative process. You want to say what you want secured and not how. Only one system I know of has ever used Prolog for this (MeeGo, IIRC?) but it was probably a perfect fit. There are lots of opportunities for this kind of thing in OSes and they're never really exploited. If your whole kernel is C and you want to add a declarative security module, well, you'll be writing C. You can surely accomplish the same tasks in C as C#, but if it's a pain in the ass you'll either wind up with a weaker or more limited version in C, or you'll spend more time writing and polishing it. But the world is ripe for a few people to take the unpopular tradeoff and see how well it works out, rather than taking their predecessor's word for it.

To sum up, they're not suitable for the whole job from start to finish. However, if you tried it, you'd make some interesting stuff, and learn a lot, maybe even make things better for other people. Take a crack at it. Worst thing that could happen is you fail. :)

Look at fleitz's answer. Maybe you could download Singularity and use that as your base. If you get really good at it, there might even be a chance at a job on the Midori team.

> The Lisp machine operating system was written in Lisp. In fact, it was Lisp all the way down to the microcode, or at least, it would present the microcode in Emacs as if it were Lisp, and you could modify it there. The Lisp machine OS was similar to other OSes of the age, more capable in many ways.

On the other hand, IIRC early machines at least used custom hardware with direct support for many runtime features and implementation details, e.g. GC and tagged pointers.

I'm far from an expert, but I've definitely heard mention of those things.

I think what makes us non-Lispers romanticize the Lisp machine isn't that it ran Lisp per se or even a high level language. It's the Aston Martin flavor of a machine built to be one comprehensive whole by people who understood all the layers and wanted a coherent machine in toto. We still crave these things, and there's something vaguely upsetting about knowing that (for instance) one's Smalltalk image is running within a Unix environment and not natively, even though in practice it's better this way.

Yes but why would you want to?

I can't think of a single long term advantage of using a language that high level to build system software. Sure it makes a cute research project but that's about it.

As for application level stuff, a big chunk of windows 8 is c#, mainly in the metro space as is windows phone.

I'm going to put the equivalent effort into answering this question that you did into researching it: maybe, maybe not.

That you didn't mention Singularity or Cosmos tells me that you haven't cared enough about this to even do a cursory Google search on the subject.

Anyway, Google those above OSes, browse through their white papers and you'll have your answer (as given above). You might also check out JavaOS, Inferno, JNode, etc which have similar goals.

To get you started with less effort:

https://en.wikipedia.org/wiki/Singularity_(operating_system)

https://en.wikipedia.org/wiki/Cosmos_(operating_system)

In fact a few people at MSR had that exact idea (http://research.microsoft.com/en-us/projects/singularity/).

To write an OS you need to generate native executable for the hardware that you are running on. The mainstream implementations of C# require CIL + CLR. i.e. you need a virtual machine + libraries and framework. Not exactly "bare metal" environment.

But C# is more advanced language than C, so that might be beneficial in some areas. As a minimum you would need a C# to native compiler and facilities to include assembly language code in some low level areas.

To get a better idea of what it takes to implement an OS, take a look at the Linux kernel source code. Or if that is too daunting, you could look at Minix 3 sources.

C# generates native code just like many other languages. In fact with a few minor tweaks MS has written an OS in C# called Singularity.

I looked it up on Wikipedia. You may have overlooked the fact that MS abandoned this project and that it used assembler, C and C++ to build the foundations. So the exercise has shown that it is possible, albeit with a considerable amount of non-C# code. But I think they also learnt that it isn't a particularly effective. There isn't enough information on Midori, the "commercial" successor to Singularity to make any assessment.

C# by default doesn't generate native code. C# generate CIL (Common Intermediate Language), which only gets turned into native code at runtime by the JIT (Just in time compiler).

That being said, it is also possible to generate a native image (using ngen) to transform CIL into native code for a specific machine. But that approach is the exception more than it is the norm.

The process you outlined language -> bytecode -> native code is exactly how FreeBSD works when compiled by clang.

That approach is also the norm on iOS for both Obj-C and C#.

I am pretty sure ngen doesn't create native code in the sense that it doesn't create a windows executable. All it does is the same as the JIT but does it up front (for the target platform). In other words it still generates something the runs inside the CLR.

You're right, it's basically skipping the JIT step for better performance for managed apps at the cost of higher initial load time (e.g. more to read from disk).

It does generate native code. What else do you think is what runs at runtime? .NET never has been interpreted (apart from Micro) so you either have the JIT or ngen and both result in the bytecode translated to native code prior to running. That it still calls into the CLR for things like object allocation, garbage collection, etc. doesn't change anything. You can do the same from C++ if you want.

There is a reason they call it managed code. It isn't native windows code.

What makes C suitable for low level programming is not only lack of GC, but lack of global heap at all. Global heap is just a library thing. You can decide to alloc on a stack or use a particular memory region for allocation. Also you routinely copy between user and kernel spaces for security. This is totally different then global GC and global heap approach of c#. All works around this problem is basically getting away from C# so could you still call it OS written in C# when this problems were solved?

You might want to wait for Rust to be ready, because this way you can build an OS that truly made for the future (very optimized for multi-core).

If you don't want to appeal to the general market, but more for a specialized market (like in enterprise or whatever), then you should probably optimize it for AMD's Mantle API, too, and take full advantage of the performance for both the CPU and GPU that you get from that. Also don't even bother with 32-bit.

Elementary OS is built on Vala, a language syntactically similar to C#, but compiles to C. Not trying to detour you from C#, but it might be a better option as even Singularity is not entire C# (the lower level stuff).

http://elementaryos.org/

Elementary OS is a Linux distribution. Linux is primarily written in C.

> Elementary OS is a Linux distribution. Linux is primarily written in C.

Yes, the Linux kernel is, but not all of the userland, which is what makes Elementary OS, "Elementary OS." ElementaryOS outside of the kernel is primarily Vala[1].

One of the comments by the ElementaryOS developers the other month on HN about their usage of Vala and its performance:

> aroman 51 days ago | link

Vala is, all in all, fantastic. The way I describe it to people is "it feels like C# but runs at C speed", and that's pretty much accurate. The only issues we have are binding bugs and Vala bugs, both of which are harder and harder to find these days. And when you do find one, the development community is very active and you can find the help you need.

For concurrency, I have personally written async vala code (which has some excellent support, including closures), but there is full thread support via GObject. The thread safety stuff is the same as the underlying library, and lots of work goes into GLib/GTK+ to make them thread safe where appropriate.

[1] https://news.ycombinator.com/item?id=6194028

This makes your point about Vala being a better option than C# because it can be lower-level a little moot if the core is still C just like in Singularity.

> because it can be lower-level a little moot if the core is still C just like in Singularity.

Not exactly, as it's more likely they didn't want to roll their own kernel when there's a perfectly good open source one. Microsoft and Windows didn't really have that luxury and based on the project's likely goals with its commericial leaning successor, Midori, they wouldn't want to use something under the GPL either. I'm not a hard core kernel developer, but I'd wager one could be written in Vala if one was so inclined.

Microsoft carried out a research project looking into this, but an open source group is actively developing another implementation.

What they've achieved is really quite impressive. http://cosmos.codeplex.com/

Haha, not to be confused with this: http://blogs.msdn.com/b/seliot/archive/2010/11/05/cosmos-pet...

No, but Rust is a contender.

Do you want to offer some reasoning for your enlightening statement?

From my cursory review of Rust, they're getting a lot of things right. One small example: in C#, you can't properly stack-allocate things like strings, because "reference types" get alloc'd on the heap, end of story. A function might decide to store a reference later, so stack alloc isn't possible. The CLR implementation doesn't do escape analysis.

In Rust, the type system encompasses things like who owns a pointer, so you can keep memory management simple, while choosing performance-oriented paths when needed.

The CLR itself is actually fine with a lot of such things (nothing stops you from stack allocating a string "by hand"), it just doesn't help you out. IL is pretty expressive and can translate C in a fairly straightforward fashion. So "C#" by itself has some drawbacks, but they could be patched over with some interop libraries or language extensions.

If by string you mean array of char then it's perfectly possible to stack allocate an array of char in C#.

http://msdn.microsoft.com/en-us/library/aa664785(v=vs.71).as...

And seriously, it's not like everything in the linux kernel is stack allocated. The stack runs out of memory as well so it's not as if stack allocation is some magical place with out resource constraints. Since we deal with processors with caches and lots of other variable timing issues you don't even really get deterministic timing either.

No, I mean to allocate a System.String instance that you can pass to other functions. Or a managed array (say, the char array you pass to String.Split). Or any other reference type.

> The stack runs out of memory as well

I agree with the rest of what you're saying, but I want to point out that Rust stacks are growable (at least in theory), so you won't run out of stack until you're OOM.

I'm a fan of Rust, but the practice of allocating strings on the stack in fixed-sized buffers is one of the biggest mistakes of C culture. Perhaps you could just picked a poor example?

Stack allocation is kind of like inlining, the more you use it the faster your program gets.

I know this may have been tongue-in-cheek, but on the CLR, inlining functions makes a massive difference. That is, using F#'s forced inlining capabilities to generate massive functions (many KB of IL) resulted in much better runtimes than having even a couple of small functions called (which "should" be inlined by the CLR; MS tells us we should just rely on the JIT for these cases).

That's more a problem of unsafe manipulation. Allocating a safe length-prefixed string should not be an issue. Or, arrays. Take .NET's String.Split function which takes a char array. If you pass in a new array each time, there's a whole heap-alloc for no reason. Or Tuples. Or closure objects. Things like that do add up (GC cost, even though small short-lived objects are fast, it isn't zero), and escape analysis/stack alloc can make a huge difference, sometimes.

I've come to this conclusion while writing a "high-speed" indexer program where manually avoiding tiny allocations in certain areas provided double-digit percent improvements. I wish I had kept proper notes.

Yes, absolutely, allocate everything possible on the stack.

But strings are the poster child for dynamic memory allocation. We only very rarely know a good upper limit on the size of the actual strings at the time the code is written, yet C programmers have shipped endless numbers of bugs thinking they did.

Stack allocation needn't be unprincipled hackery. You can keep a buffer of static size on the stack (resp. in the struct) and fall back to dynamic allocation only if there is insufficient space.

Yeah I've written that code a bunch of times. A real pain in the butt to get right and to test.

I bet. All the more reason for the implementation to provide it for you - possibly transparently.

Rust doesn't have stack allocated strings, I have no idea what you're talking about.

We have string slices, but there's no way to stack allocate those, they don't own their allocation.

I must have misread then. I thought in Rust there were various pointer types denoting ownership, and one of them was callstack-scoped (borrowed pointer?), meaning it could be backed by stack allocated memory.

Well, yes, borrowed pointers can point onto the stack, but we don't have string values that can live on the stack, not really. String slices, denoted as `&str`, can point into the stack if you construct them from a `[u8, ..N]` (a fixed-length vector).

String literals are stored in the ro section, as one might expect.

Right, literals, but for example, suppose I'm creating a ton of temporary strings (e.g. appending a filename to get a full path and calling touch on it) - in C#, I'm going to heap-alloc a new string for each call. With Rust, I _could_ compose it into a stack-alloc'd buffer, and pass it any function taking a string it doesn't need to own, correct?

Yes, you can put a fixed-length buffer of bytes (e.g. `[u8, .. 1024]` is 1kb) on the stack and use that as the backing vector for the string.

Reply on news.ycombinator.com