First Rust code in the Windows 11 kernel
thurrott.com
thurrott.com
https://azure.microsoft.com/en-us/blog/microsoft-azure-secur...
> Rust as the path forward over C/C++
> Decades of vulnerabilities have proven how difficult it is to prevent memory-corrupting bugs when using C/C++. While garbage-collected languages like C# or Java have proven more resilient to these issues, there are scenarios where they cannot be used. For such cases, we’re betting on Rust as the alternative to C/C++. Rust is a modern language designed to compete with the performance C/C++, but with memory safety and thread safety guarantees built into the language. While we are not able to rewrite everything in Rust overnight, we’ve already adopted Rust in some of the most critical components of Azure’s infrastructure. We expect our adoption of Rust to expand substantially over time.
It's not just that it doesn't have a GC. It also doesn't have some of the worst features of OOP, it also has a sane std library, it also has an expressive type system, etc. Rust helps you write better code, even when compared to languages like Java or Python.
It would be cool if there was a "relaxed rust" which provided interop with standard rust, but used some designated smart pointer for everything automatically (at the crate level, for example). I'm not sure how/if this could work, but I think it would make transitioning from GC languages -> Rust much easier.
Hmm I think for 95% of the people sticking to something like Swift would work out better for them
> It also doesn't have some of the worst features of OOP, it also has a sane std library, it also has an expressive type system, etc.
You made a baseless assertion, I at least gave examples of features/antifeatures that helped me draw my conclusion.
Recently I had a vexing silent bug in a prototype, where I was intermingling strings and ints as keys in a dictionary, and thus not writing to the correct keys. Normally I would be more careful than that, but it was a quick and dirty proof of concept. Because this involved JSON deserialization, Python's modest static type checking couldn't catch it.
That just wouldn't have happened in Rust. (But of course, I wouldn't have had the particular ML libraries I needed, either, so Python remains the best option for this project.)
If you cap the number of tokens, the are fewer Rust programs. If you prefer, things that may be done implicitly in Python must be done explicitly in Rust - there are fewer valid choices when constructing a program; or equivalently, as you write a Python program, the options for your to proceed fan out faster.
Really we aren't searching the whole space of programs though. We understand that adding no-ops to a program doesn't move us out of the equivalency class of that program (unless we're actually using those for timing or another side effect, of course, but then they are no-ops in name only). We're only searching through programs that reasonably solve the problem at hand. Maybe that space is infinite, I don't think so.
I think of programming as being a fuzzy beam search (among other ways I sometimes think about programming). Your salience may vary, feel free to drop in a metaphor of your choosing there.
I don’t sit in front of the screen as candidate programs flash by until I see the one that I want.
I also don’t edit by AST transformations from one valid program to another, like edges in a graph.
I really don’t see the possible program space as all that relevant.
I don't really see how else you would edit a program? That's what you're doing when you type at your IDE, right?
But I think you're understanding how I think of it, feel free to take it or leave it. I'm not gunnuh die on a hill of insisting my way of thinking of it is right, it's just right for me.
> I really don’t see the possible program space as all that relevant.
In my example, I fell into a silently invalid state. In a more constrained environment, I wouldn't have. It's easier to go bowling with the bumpers turned on, and if your goal was to make as many strikes as possible (rather than to be sporting), surely you'd only ever bowl with the bumpers.
It's a Murphy's law thing. The more invalid states you have, the easier it is to get mired there.
Edit: I cannot reply, but there are structured program editors out there you may find interesting!
(You can reply to comments when the reply button is hidden by clicking the link to go directly to the post. I consider it a polite request by dang to consider whether a conversation is getting too heated, which I don't think it is here at all. Cheers, I'll try to check those editors out.)
People don’t write programs by scanning through the space of possible ones, so I don’t see why would that not be a hindrance, let alone help one.
Also, static types, or at least optional typing is at this point pretty much more common than dynamic typing, so I fail to see your example making Rust a better choice than “most other languages” for this reason.
Especially that the real strictness of Rust is from the ownership system, which allows basically only tree-shaped lifetimes, and that’s not a restriction like a program being type-safe, where non-type safe programs don’t make sense. It is an “arbitrary” constraint that has obvious advantages for not memory-managed languages, but is absolutely not a tradeoff I would take when not necessary, as random lifetimes are just as valid.
I definitely search when I write programs. Do you write a program from start to finish and it works, or do you iterate and debug and sometimes find you've made a misstep & need to change your approach?
Python's optional typing wasn't capable of spotting my issue, that was my point. Note I didn't say it was a better choice. I said, "strictly less complex." I wouldn't make a claim that Rust is "better." That only has meaning relative to a set of requirements. (Furthermore, I mentioned Python was a better choice for my project.)
Tree shaped lifetimes are a very robust and easy to work with set of lifetimes. Of course you can add more degrees of freedom and write programs with graph lifetimes. That's fine, but it's pretty clearly more complex. I view allowing arbitrary lifetimes as something I don't want unless it's, well, absolutely necessary is too strong, but unless there's a compelling reason. It's not something I want to spend my complexity budget on, because it makes it more difficult to reason about and debug a program.
How do you measure the complexity you're talking about here?
What if a program using graph shaped lifetimes was 10 times shorter than a program that had to use tree shaped lifetimes? Would you still say that the tree shaped program is less complex?
With Python for instance, it's just really difficult to say, because you may have arbitrary exceptions at any point. When writing long lived services, this is very significant; you need to handle all the exceptions that may be raised by you can't actually determine what that set is. I've had to read the source code of my dependencies on more than one occasion to figure this out, and still, you never get all of them before production.
With Rust this is much more manageable. Technically you could have an arbitrary panic at each step, but it's much less common to catch and handle panics than it is exceptions, and recoverable errors are handed via return values. Relative to Python, it's a breeze to figure out when errors may happen and what to do about them.
Similarly with my example before, what type are the keys in a Python dictionary? Probably the ones you meant to put there, but who really knows. What types are in the keys of a Rust dictionary? The ones you put there explicitly, full stop. There's no way for that to get screwed up or change from under you.
But I'm still not sure whether this idea is sufficient to explain away the situation where you have to write a far longer program in order to work around those restrictions.
For instance, safe Rust cannot express linked lists in the usual way, i.e using pointers. So in order to write a linked list in safe Rust, you would have to reimplement some of the functionality that pointers provide using array indices (or similar).
This program would be very long indeed, plus it would cause many of the same safety issues that safe Rust was supposed to avoid. Essentially, what this program would do is create an unsafe memory management system in safe Rust.
I believe that the dominant factor for complexity of a program is its length. If you have two programs that produce the same output for the same input and one is 10 tokens long while the other one is 10,000 tokens long then the first program will always be less complex.
I say "I believe" because I'm a bit out of my depth here. These claims (mine and yours) clearly need formal proofs and some formal definition of complexity.
But the thing is, you don't write C programs in Rust and you don't generally use doubly linked lists. (Linked lists turn out to be a very niche data structure that doesn't work well with modern CPUs anyway, but I digress.) You'd probably use a Vec or a BTree from the stdlib anywhere you're thinking of using a linked list.
So I don't think it's really the case that programs are significantly longer in Rust. Rust is more explicit, so you'll end up moving some things that existed as comments or documentation or just in your own head into code - that's a win that doesn't increase complexity, only exposes it. That program may look larger, but it's only because you can see it better.
It really depends on what those 10 tokens are doing. If I have a token that creates a new universe, seeds it with life, and creates the conditions for life to develop an optimal linked list - it might solve the problem in one step, but our atomic unit here is absolutely massive.
Similarly, if I compile a program to assembly I'll generally get many more tokens. But I can't really buy that I've increased the complexity of the program here.
I'm pretty satisfied with this understanding but I understand your desire for greater rigor.
The usefulness of linked lists is entirely beside the point. They just serve as an example for situations where additional restrictions can cause a program to be longer or more difficult to understand.
>It really depends on what those 10 tokens are doing. If I have a token that creates a new universe, seeds it with life, and creates the conditions for life to develop an optimal linked list - it might solve the problem in one step, but our atomic unit here is absolutely massive.
This example shows why I think that your claims lack a definition of complexity. You're now saying that the output of a program determines its complexity. I don't think that's a useful measure of complexity because it doesn't allow us to compare the complexity of two programs that produce the same output.
Rust's tree-shaped lifetimes are a big benefit for me, as someone who values not wasting RAM.
(Seriously. Why did I have to upgrade my computer to 32GiB of RAM when I haven't appreciably changed what I use it for since I bought it wth 4GiB of RAM and sized that amount based on intent to run a VirtualBox VM or two? ...JavaScript bloat. ...though I will admit that I also upgraded the CPU, RAM, and motherboard in the transition from 4GiB to 16GiB... but that was because a RAM socket went bad.)
I think of borrow-checker errors as "Wait a minute. Can you clarify what you intended here? Did you want multiple ownership? A copy? Was accessing it this far down a mistake?"
About certain few properties only. It is impossible in the general case.
I agree regarding “change anxiety” compared to dynamic languages, but that’s just static typing. Due to lifetime annotations leaking into API boundaries, the rest is not true though.
I can't say much about the ecosystem besides it lacks something in the desktop GUI department (I use egui, it's really good, but just it's not strong enough). So, yeah, sometimes you miss a library and that can hurt productivity.
But the protections against memory and thread issues really help productivity though: I tend to make lots of mistake in these area and rust helps to to prevent many of them.
Well, it makes sense as both are low-level languages. Hell, one might even argue that Rust is “nothing more” than C++’s RAII made into a compiler-enforced primitive.
But C++ is not considered a high-productivity language to begin with, at least not many would choose it for a regular old CRUD app.
While I am a big fan and all my personal projects are written in Rust, introducing Rust will likely paralyse entire teams for months, until they get used to the borrow checker. ORMs are quite clunky (at least Diesel) and web frameworks are not as user-friendly as ASP.NET MVC, and that’s 80% of what the average developer needs today.
As we build vending machines and such things, our products usually contain multiple processors that communicate over various connections and there's a lot of interaction with real-world, physical things. Also, as there are a lot of microcontrollers involved, C and C++ are the dominant languages in our development.
So, a lot of our development time actually goes into solving the general unpredictiveness of the real world and humans interacting with actual machines.
Rust enables us to focus on solving those problems without getting held up by stupid memory issues. In our case, iterating over new software versions often is slow because it may involve restarting a big machine and doing actual user interaction with it – the time Rust takes before compiling is easily saved for us.
With prototyping it's a double-edged sword: it can be a bit more cumbersome to change types, but the same type-checks are helpful for pointing out details during refactoring. For me it helps keeping things simple at first, expanding types later on when needed is easy.
It is still a low-level language, and while it is expressive, that doesn’t stop the low-level constructs from leaking to high level abstractions — which is not a negative in itself, that’s the purpose of the language. It’s just a hard tradeoff when it comes to anything not needing low level considerations.
Pair that with minimal memory issues with Rust, and guaranteed none if the framework avoids unsafe blocks.
For example, lots of people trip on `append` in Go because it tries to hide pointer semantics.
Having to write `&` sometimes is just not burdensome and it's frankly sad that so many professional developers, many with so-called computer science degrees, seem to think that a basic understanding of computers is too high a burden.
1. Rust is low level
2. Low level details leak through abstractions
3. This creates a difficult trade off for use cases that do not require manipulation of low level constructs
I am making these assertions:
1. Pointers are not particularly "low level"
2. The tradeoffs when pointers are abstracted away are worse
3. It is sad that trained professionals believe that pointers are difficult to understand
What I want:
* Powerful, strict typesystem (typeclasses, sum types, type-inference, closures)
* Constrained mutation, no 'spooky action at a distance'
* Safe multithreading
* Fast
* Memory-safe
* Native binaries
* Large, deep ecosystem
* Great tooling
* Jobs
Rust comes closer than most.What other option is there? I thought the same and wanted a higher-level GC'd language than rust, and played some with Ocaml, Haskell and F#. In theory they should be more productive and I'm sure are for things like compilers and parsers where they have top-quality libraries, but for most things I got the impression that even if I took the time to learn well one of those languages, the quality of libraries and documentation and tooling is so far ahead in rust that it more than makes up for the productivity hit of rust making you concern yourself more with low-level details
It's not just Rust's type system, but:
1. Unlike with Haskell, the Rust compatibility promise has set the tone for how the ecosystem approaches API breakage.
2. Go-like "statically link everything into a single binary" compilation means that "just keep using the old build until things are fixed" is a valid answer to "an upgrade broke the buld process".
3. Rust's type system enables design patterns like the typestate pattern, for encoding as many invariants in the type system for proving at compile time as possible.
4. Rust's design prioritizes removing the need for global reasoning about program behaviour.
Same reason I recently spent $10 on a used copy of the O'Reilly lex & yacc book to learn the warts of LR parsing. I plan to write my parsers via something like LALRPOP or grmtools instead of using nom (parser combinators) or pest (PEG parsing), which are currently more popular in the Rust world. (As with borrow-checking errors, shift/reduce and reduce/reduce conflicts aren't the bug, they're the feature. I read an article about how LR parsing allows the most detection of ambiguities in the grammar at compile time. Cry in the dojo, laugh on the battlefield.)
I'd rather pay up-front to avoid the stress of having a Sword of Damocles hanging over my head and feeling satisfied with Rust takes FAR less time than either burning out trying to replicate its type system in unit tests or playing bug whac-a-mole over the lifetime of the project.
I think that is also why rust ecosystem will keep expanding and improving a lot. They said about python that it's not the best language for anything but 2nd best for everything, which isn't really true as python can't be used when you'd need C or C++ but rust can really be used for anything.
I sort of understand the distrust of .NET, but why the JVM? It was/is pretty much the epitome of open-source.
Not the OP, but basically two things:
First, for Java's formative years, https://www.gnu.org/philosophy/java-trap.html applied. (On a related note, see also http://endsoftpatents.org/2014/11/ms-net/ )
(TL;DR: The JVM was NOT "the epitome of open-source" for many years and it's still struggling with the knock-on effects of spending so many years being one of the only things that you couldn't install through your package manager for purely legal reasons.)
Second, I can't remember which blog post it was, but Eric S. Raymond has mentioned that the reason he found Python much more appealing than Java is that Java's standard library embodied an attempt to push people to write portable code in ways that made life difficult when your goal was explicitly to do POSIX-specific things.
As a Linux user who saw C# for the "Microsoft's take on Java after the Visual J++ lawsuit" that it was and, thus, never really saw any interest in it, I can say that C# gives that same impression of "ill-fitted for POSIX-native stuff" (packing its CLR bytecode into .EXE files doesn't help, even before you get to the assumption that you'll have Wine and Mono fighting over who should open .EXE files when double-clicked.)
Rust, by contrast, produces truly native binaries, has Cargo come standard and makes `cargo add libc` or `cargo add nix` trivial and reliable, etc. etc. etc.
Third, the JVM's tunings, optimized for long-running processes, and the start-up time and approach to memory allocation that resulted, gave it a reputation for being slow and bloated. The POSIX ecosystem has a history of encouraging composition of short-lived processes via shell scripts.
Fourth, Java has always let the quality of the GUI experience on X11 languish.
I still remember how you needed to set environment variables to work around "Java applications produce empty grey windows under non-reparenting window managers" for years and years.
I still remember when you had to open part of the JVM in a hex editor and replace XINERAMA with some nonsense string that doesn't match anything to un-break Java applications on multi-monitor systems.
TO THIS DAY, I still can't find a Java GUI widget toolkit that doesn't have a noticeable responsiveness/input latency problem under X11.
(Swing, SWT, JavaFX... that annoying fraction-of-a-second sluggishness is one reason I'm planning to write my own replacement for the parts of the new JavaFX-based version of PlayOnLinux that I actually use once I stop using the old Python version.)
I haven't tried QtJambi, but given that SWT, which should be using GTK on the backend, exhibits the problem, I don't hold out much hope.
(In essence, in the same way that Swift is ill-suited for stuff outside Apple devices, .NET has gained a stigma of "ill-suited for anything outside Microsoft platforms" (Is Unity's Linux support still only viable for targeting it, not developing on it?), and Java similar, but for JavaEE servers... and since your average POSIX developer sees AbstractThingFactoryBeans as a tired but too-accurate joke about what hell it is to write Java... you do the math.)
“Since this article was first published, Sun (now part of Oracle) has relicensed most of its Java platform reference implementation under the GNU General Public License, and there is now a free development environment for Java. Thus, the Java language as such is no longer a trap. You must be careful, however, because not every Java platform is free. Sun continues distributing an executable Java platform which is nonfree, and other companies do so too”
Also, Stallman is quite an “extremist” and he is often himself the enemy of open-source by his gatekeeping, so there is that.
Re the GUIs: I think it says a bit also about the state of GUIs on linux, then the reverse - and I say that as someone who has been using linux on every computer I own since forever.
You made some great points, but I think the real reason is even simpler, old graybeard linux users are very conservative in their technology takes. The overabundance of C software (and the unavoidable memory vulnerabilities that come with that) even at places where it makes zero sense is clear sign of that, but so are the “systems/pulse/wayland hater fangroups”.
People who don't care about Stallman's zealotry do still care about that and it presented a similar lasting problem to Java's Linux uptake as the problem D had more broadly with two competing and mutually incompatible standard libraries (Phobos and Tango) in the early years.
As for the GUIs, I listed three different ways unique to Java that Java GUIs were subpar on Linux:
1. Needing weird workarounds for applications to not display blank grey windows on non-reparenting window managers. (Basically, if the WM was something like a tiling WM that didn't reparent it from the root window to a WM-provided "titlebar and borders" parent window, the app would refuse to render anything without enabling the relevant hack.)
2. Caring so little about solving known bugs that slipped through QA that, for a shamefully long window of time, users with more than one monitor literally had to hex-edit their JVM to intentionally no-op the XINERAMA detection to make applications work. (i.e. XINERAMA being the X11 extension that allows Windows/Mac-style "one desktop stretched across multiple monitors" multi-head instead of the older "Behaves like a software KVM and applications are trapped on the monitors they opened on" Zaphod mode.)
3. Having some kind of input-response latency that I've never seen in another language or toolkit, which is apparently caused by something so fundamentally Java that it's present in every Java toolkit I've checked.
As far as greybeards go, I think we'd need more data points. Eric S. Raymond is quite an old-guard guy and even had some "jumps to conclusions"-y, "it's not enough like C"-ish reasons for choosing Go over Rust (eg. No `select` as part of the standard library, ignoring how Rust intentionally keeps the standard library minimal), but had no problem with using Python where it was suitable, while he didn't like Java because it was was more of the ChromeOS or Android school of doing stuff on top of a POSIX base.
And also the cost of FP is quite high on JVM. Paying 10-20x performance price for functional transformation chains over similar ones in Rust is a bit too much for me.
List<Something> list = ...;
for (...) {
Something item = new Something();
list.add(item);
item.setFoo(...);
item.setBar(...);
}I hope not, but it's the popular memory safe language we have now, and ironically Stroustrup's quote applies "There are only two kinds of languages: the ones people complain about and the ones nobody uses".
I would like Val to succeed: https://www.val-lang.dev/ With a Swift/Kotlin style syntax, the learning curve is likely easier as well.
Even though I primarily am a Linux user and I prefer using Linux on a daily basis, dabbling in Linux kernel development failed to create this feeling of awe and infinite possibilities (and exploitability) in my mind.
I guess my point is, Windows Kernel development in Rust excites me. Same with Linux would be very cool but it doesn't create the same feeling of wonder in me. Perhaps that's why this is happening first.
We have been playing with Rust and minifilter for Windows. Its almost there. Microsoft has been working on rust driver kit.
It’s kind of a worst case scenario for adding in a new language to as a result, since it can’t be added in a very isolated way.
This is an amusing claim, given that Microsoft's first use of Rust in the NT kernel is in win32k.sys, which is the in-kernel code that used to live in userspace back when NT was actually a microkernel. So pre-NT4, which was released in 1996.
Then there are the whole set of userspace drivers, including graphics.
How are those X Windows driver crashes holding on?
Syscalls powered by MSNBC
Don’t get my wrong, but me and I bleed Unix-like systems and have done for decades, but what you’re saying actually makes sense for MS as a design decision…
Relevant:
Why the Windows Registry sucks … technically https://news.ycombinator.com/item?id=32275078
https://rwmj.wordpress.com/2010/02/18/why-the-windows-regist...
The problems with the windows registry are orthogonal to that.
(eg. Changing their save icon to an "arrow pointing at hard drive" one that's harder to visually distinguish from their download icon because they don't understand that you shouldn't privilege affordances (which only give benefit while learning a new system of symbols) over consistency with existing established iconography like the "save icon" (diskette) used by every other system.)
...or GNOME 3 ruining the "Cancel/OK in the bottom-right corner of the dialog, in that order for LTR languages" they borrowed from Apple with their "action buttons in dialog header bars" idea. (TL;DR: Dialog boxes are laid out to read like paper forms, following the writing order of prose text in the user's native language. Action buttons go in the reading-order-terminal corner for a similar reason to why the signature field on a paper form is at the bottom and why business letters are supposed to end with what you want the reader to do.)
Likewise, for dialogs where they stretch OK and cancel to full width. Now you've required the user to move the cursor much further during normal operation.
See also https://uxmovement.com/buttons/why-ok-buttons-in-dialog-boxe...
This is quite literally stuff that was covered in my introduction to Human-Computer Interaction course at university.
There's also the advantage of being able to add Rust to the build pipeline without someone complaining that you broke the build for their TI-83+ calculator because there's no Rust compiler for that platform. Windows deals with amd64, x86, aarch64, and maybe some simple 32 bit ARM, but that's about it. Nobody is going to care about support for MIPS or Power9 or Motorola 6800 or SPARC or Xtensa.
I see Microsoft has done some work and there’s a Windows crate, but I’m not seeing IDL support for COM and it looks like you end up with a ton of unsafe blocks. It makes me wonder if a programmer who sticks with smart pointers in C++ is all that much worse off than a Rust developer on Windows?
On top of that, it should still be possible to write the application logic in safe Rust, even if you need to use unsafe for FFI with Windows stuff.
My criticism was directed towards the article, not the submission.
Edit: Ah, the guidelines say unless it is misleading.
Well, I would not have called it truly misleading, just typical headline style you can often see. I did not like it, but I did not feel misled.
One can do a Google, and push it no matter what, or give up and take the adoption path that is easier to bring the safety luddites along for the ride.
We can wait the usual progress rate of one funeral at a time until a new generation buys into fully managed OS, or another Google/Apple big spender forces them into developers no matter what, which most likely I will not live through.
So we're left with the compromise of allowing only userspace for managed languages, while adopting something like Rust for the "only over my dead body folks" anti any form of automatic memory management languages.
Given that even the success stories of bare metal deployments of managed runtimes doesn't change their minds anyway.
2. It is heavily pinned by unnecessary metadata. Well, actually some maybe necesdary to facilitate GC but it is bloated anyway
3. RyuJIT is not as optimized as LLVM, and RyuJIT produced the AOT code. There in an abandoned project called LILAC attemping to use LLVM as a second AOT generator but it was never heard from again after 2019.
4. Native interoperability in .net requires JIT as well, to generate things like call sites and runtime thunks (function pointers to restore CLR context). Not only that wastes more resources but also that not all P/Invoke calls can be AOT compiled.
- Midori
- Oberon
- Mesa/Cedar
- Topaz
It is a matter of actually wanting to make it happen, or giving up to the voices that think otherwise.
Joe Duffy himself said with Midori they "started with C# and .NET [but] were forced to radically depart in the name of security, reliability, and performance" and that at one point they had 11 different garbage collectors. The jury is still out (and has been deliberating for an awfully long time) on whether you can build or even contribute to a successful, mainstream, general-purpose OS kernel in a garbage-collected language but a distinctive selling point of Rust is that you can start adding memory safety without having to. And the fact that this is actually starting to happen to more than one such kernel while Rust is still a fairly new language could be interpreted as evidence that maybe, just maybe, the GC really was the problem all along.
Maybe, just maybe, the problem isn't technical, rather human, and that can only be tackled one funeral at a time.
By contrast newer generations don't think C and Assembly is the only way to program a mobile device, or embedded system, see MicroPyton and JavaScript in the maker community among school kids.
1. Well technically speaking, you need a unified heap space. Just like in Linux kernel you have vmalloc and kmalloc which does virtual memory and real, page-aligned physical memory. So this is technically means I will have to pre-allocate everything. But it is not gonna work, how do you save space?
2. "necessary". I should have enabled autocorrect on my phone, it does more good than harm this time if I had it on.
3. It's actually LLILC (https://github.com/dotnet/llilc). I just remembered how to pronounce it. But if the name is wrong, then you have to correct it.
Tested and Failed. There are lot of interesting learnings and blogposts from this project and lot of them are actually landed in different products of Microsoft
If anything, it failed at the Windows business unit politics.
Unfortunely it lacked the kind of management that has pushed Java no matter what on Android, or either Swift or the high road on iOS.
It seems like you're confusing a few different things there.
The 'redo' step that you're talking about happened because in the early 2000s Microsoft was getting blasted by security breaches left and right.
Pretty much every senior developer in the Windows org was pulled to work on security patches. An in between update to the consumer version of Windows (codename Longhorn) was planned to hold customers over until the team had the time to do a proper iteration on the next OS.
As the senior devs were wrapping up their security fixes and coming back to see about shipping Longhorn they decided that it wasn't up to their standards and more importantly they didn't think it was moving in the right direction because it didn't include all the security fixes they had been doing.
Longhorn had essentially 'forked' XP pre-security fix and it was decided that it would be easier to throw out all of the Longhorn code and start fresh then to try and port all the security fixes over.
So they just that: they threw out everything that had been worked on, 'reset' the project to be based on the latest code with all the security fixes present, and then started the development of Vista fresh.
Microsoft internal emails from Jim Allchin (who ran Windows at the time) to Bill Gates reveal that the issue was performance.
> LH is a pig and I don’t see any solution to this problem.
https://web.archive.org/web/20210427171552/http://blog.seatt...
Eventually, the solution was to drop the use of a garbage collected runtime (desired for memory safety) and go back to C/C++ (necessary for performance).
Now they are moving to a language that gives you the memory safety without taking the performance hit.
Note teams could have private source code repositories and Windows builds. Changes did not have to be shared with the entire Windows team. The better teams shared their changes when they worked and were ready. On Vista, a lot of teams shared things which were far from ready. Note that this was unusual and did not happen on Windows XP.
Longhorn was refereed to as Cairo.net, as a callback to a previous Cairo project from the 90's that failed to ship and as a reference the drive to improve security through the use of memory safe garbage collected code.
If you are going to claim garbage collection caused performance problems, you need to state what part of the operating system used .NET or GC and why using those technologies caused problems.
Several of the tentpole features of Longhorn were based on .NET and managed code.
> First and foremost, while Windows Server 2003™ embraced managed code by being the first operating system to ship with the .NET Framework preinstalled, Longhorn is the first operating system whose major new features are actually based on the .NET Framework.
... Microsoft appears to have concentrated their development effort in Vista on native code development. In contrast to PDC03LH, Vista has no services implemented in .NET and Windows Explorer does not host the runtime, which means that the Vista desktop shell is not based on the .NET runtime. The only conclusion that can be made from these results is that between PDC 2003 and the release of Vista Beta 1 Microsoft has decided that it is better to use native code for the operating system, than to use the .NET framework.
https://web.archive.org/web/20051212045841/http://www.grimes...
What you are talking about was called the Windows Security Push and this is how it worked. Basically, every developer (not just the senior ones) had to help review every line of code in Windows. First, we were shown presentations which explained why security was important, where Microsoft products had failings, what were common security bugs and how to look for them. I think the presentations were done my Michael Howard. They were very good.
Then we were given a copy of Writing Secure Code to read. It enumerated all of the know types of security vulnerabilities and told us how to fix them. It also taught us how to write a threat model, validate input from untrusted sources, reduce our attack surface, use the principal of least privilege, etc.
Finally, we spent three months reviewing Vista's code. Each team was responsible for reviewing its own code. We filed bugs as we found them and then they were fixed.
Note that porting the security fixes to Longhorn from Windows XP took very little work. Windows had one code base and you could move changes from release A to release B (Windows XP and Windows Vista in this case).
Also, Longhorn was reset at some point but all of the work was not thrown out. Basically, some teams reduce the scope of their work (i.e. cut features) and some projects were cancelled. The reset did not occur because the security fixes were missing. It ocucred because Longhorn was an out of control project which had been going on for 2 to 3 years and was not close to shipping.
Vista did include some new C# APIs and Frameworks. The best ones were Windows Presentation Foundation and PowerShell (it uses a lot of .NET technologies).
Vista took a long time for a lot of reasons. I do not know all of them but my observation was there were a few basic problems:
1. Teams were allowed to check in buggy code. This led Vista to be unstable while it was being developed. Note the Kernel code was usually very good. The shell was frequently difficult to use because of the bugs and because of poor performance.
Note that when it shipped, it worked much better than it did during development. I remember being shocked at how well it worked. I also remember going back to Windows XP and realizing I actually liked Vista better (it had a better user interface, and I liked the improved Windows Update app).
2. Very poor project management. Basically, no one knew what needed to be done or how long it would take.
3. A lot of overly ambitious projects and features. Some teams really tried to do revolutionary things. Sometimes they succeeded (The Windows Display Driver Model is an example of this). Often, they failed (Media Foundation is a great example of a mediocre API which came out of Vista).
4. Some teams took dependencies on immature APIs or frameworks. The problem was the framework and the applications using it were being developed at the same time. This led to a lot of reworked because applications kept on having to be updated because their dependencies changed or were cancelled.
5. Poor leadership - Will Poole led the Window Client team (AKA the desktop version of Windows). Wille Poole previously led the Windows Media player team. My impression was he valued political skill, politics and empire building over competence and technical excellence.
After Windows Vista shipped, he was "promoted" to working on Windows for Emerging Markets. After that, he "managed" his administrative assistant. He then "retired".
I am sure I missed a lot of things and I certainly do not know everything because I worked on a small part of a huge project. Windows had thousands of software engineers, program managers, testers and managers and they worked on a lot of different things.
Vista SP1 fixed most problems -- but you needed 2GB RAM.
Dave Plummer from Dave's Garage channel on YouTube is a great example of this. Sure, just creating Task Manager alone would get you on the books for being a kickass programmer, but everything else he did, and the fun YouTube stuff he's doing now, you can tell he was really a force behind the scenes in the early days.
(Fine meaning as well as it ran anywhere else, which was not well at all)
Mainly because it has different tradeoffs and its high level abstractions and other features like LINQ, interfaces, non-struct generics and async/await are very much not zero-cost, unlike in Rust.
In addition, it does not offer crucial features one would want in kernel development: deterministic memory usage, compile-time safety guarantees for writing concurrent data structures and general systems-programming-first language design.
While C# is a perfectly viable choice for writing userspace OS components, applications, UI and etc., it is a bad choice for kernelspace when C++ or Rust exist.