Microsoft exec says Windows 11 kernel will soon be booting with Rust inside
neowin.net
neowin.net
Rust is a game changer in that it is built from the ground up to offer memory safety without garbage collection and still aims for zero overhead abstractions.
It would be ironic, after their notorious security issues of the early 2000’s, if due to efforts like this, Microsoft Windows ends up being the most secure general operating system.
It already is. What, exactly, is better than Windows at security features on desktop computers? Linux? There is nothing in there that comes even close to the defensive features of windows, like HVCI, a subsystem that checks for driver signatures and the likes isolated by virtualization mechanisms, which completely prevents tempering with the kernel. Linux's support for secure boot only exists to make it convenient to dual boot with windows, it doesn't do enough to prevent kernel level rootkits, it's a total placebo and it's even worse if you use a distro that doesn't have signed kernels, like Archlinux. If you're self signing on the same computer, how exactly are you stopping malware?
Since Vista, the OS also gained some serious resilience against crashes that I have never seen on other operating systems. For example, it is possible for your desktop session to survive a GPU driver crash. On linux this is a guaranteed freeze or kernel panic. This is, fortunately, a rare event, but the last times I've seen my computer freeze on linux, it was always because of the graphic stack.
openBSD's slogan for having few remotely exploitable exploits out of the box doesn't mention that it's because it has literally no features enabled out of the box.
macOS and iOS are the systems with the greatest amount of privilege escalation fails by far. In fact, what do people think jailbreaks are? Some of which are truly frightening when you think about what could have been. Multiple jailbreaks were made that could be run just by browsing a webpage on safari. This means they punched through the browser, punched through privilege escalation and had the potential to install a rootkit on your phone. Just by visiting. A. Webpage.
How many times such a thing has happened on Windows in the recent years? visiting a webpage installed a rootkit on your computer?
Ive tested a two or three years old Chrome version with JIT compiler vulnerability and guess what - on empty Linux vm it managed to escape chrome and execute code
Meanwhile on Windows with Crowdstrike software installed Chrome just showed some error message about mem. access
Im not sure who handled that attack - was it Windows or Crowdstrike, but eitherway Ive been impressed
I would think otherwise if Windows used virtualisation and sandboxing to run old Win32 apps from XP and below. Because lots of enterprise software depends on proper compatibility modes, and there the security gets thrown out of the window.
Both Windows and iOS(I can’t speak to macOS) are becoming incredibly security mature operating systems via these security mechanisms that get stacked on top of one another. Saying on is better than the other in terms of security is hard to quantify.
Windows still does have some issues with user mode logical exploitation through DLL hijacking, or issues with credential relaying, although relaying targets are generally known and mitigated by enterprises.
iOS still has issues with remote attack surface, however it has gotten better with iOS 16 and Blastdoor +Lockdown
https://www.microsoftpressstore.com/store/windows-internals-...
https://www.microsoftpressstore.com/store/windows-internals-...
Some of the content is available for free,
https://learn.microsoft.com/en-us/sysinternals/resources/win...
One of the main differences from Windows 11 to its predecessors is that all those defaults are now turned on,
Here are some of them,
https://learn.microsoft.com/en-us/windows/security/threat-pr...
https://learn.microsoft.com/en-us/windows/security/threat-pr...
https://techcommunity.microsoft.com/t5/windows-hardware-cert...
https://www.microsoft.com/en-us/security/blog/2020/07/08/int...
https://learn.microsoft.com/en-us/windows/security/informati...
The macOS app sandbox actually works. On Windows nothing uses the app sandbox due to serious bugs and performance regressions. Chrome rolls its own sandbox for example.
SIP successfully stops macOS getting screwed up. The number of Windows installs out there in some bizarre half-broken state is incredible. It's routinely the case that API calls which work on one Windows system don't work on others even at the same patch level for no clear reason at all, which trace back to weird configuration differences to the OS.
Windows still relies heavily on client side virus scanning. Apple do malware scanning server side and then lean on their code signing and integrity systems instead, which is one reason Macs have great battery life.
And then there's all the other more well known security things Apple do with secure processors and the like.
Windows is just so far behind and they're so drowning in tech debt it's unlikely they'll ever catch up.
- Windows 11: 498 reported CVEs in 2022. [1] - MacOS: 379 CVEs [2] - iOS: 242 [3] - Android: 897 [4]
Linux isn't as well-comparable or categorized (especially given its just the kernel, and there are dozens of other "products" which make up an equivalent to what Microsoft would call "Windows 11"). Nonetheless: 306 [5]
You should check your preconceptions and susceptibility to Apple's marketing. No one is substantially far ahead or far behind (except maybe Android, but again, these are hard to compare apples-to-apples). Everyone still experiences roughly the same class and magnitude of vulnerabilities. But, everyone is also getting better at it.
[1] https://www.cvedetails.com/product/102217/Microsoft-Windows-...
[2] https://www.cvedetails.com/product/70318/Apple-Macos.html?ve...
[3] https://www.cvedetails.com/product/15556/Apple-Iphone-Os.htm...
[4] https://www.cvedetails.com/product/19997/Google-Android.html...
[5] https://www.cvedetails.com/product/47/Linux-Linux-Kernel.htm...
I'll believe that Apple's operating systems are significantly and measurably more secure when they can make it a few years without a maliciously formatted iMessage crashing the kernel. Until then; its arguing minutia. Everyone has security issues. Everyone is taking steps toward improving their security. No one is so far ahead that they're worth white knighting on HackerNews.
More than 75% of Windows CVEs isn't exactly "a much lower number of CVEs", even without considering its actually much lower market share.
This is a live issue in the Rust community, which does appear to care a great deal about security, as to how to deal with minor/theoretical vulnerabilities perhaps unworthy of a CVE.
Apple is a consumer electronics company. For serious tools, use Windows.
Any Android 12 and later has a little bit of Rust taking care of the bluetooth communications driver.
https://security.googleblog.com/2022/12/memory-safe-language...
https://source.android.com/docs/setup/build/rust/building-ru...
Safe Rust cannot represent circular data structures which makes entire classes of algorithms and architectures unimplementable. You have to workaround these limitations by creating auxiliary structures for tracking references or use reference counting; neither are "zero overhead abstractions." Rust is only zero overhead if all you do is pass values up and down the call stack. Its false appeal says more about the simplistic types of applications folks are writing than anything else.
As for overall performance in real programs, Rust seems to consistently do as well or better than C++ programs as the cases from Microsoft here show.
In safe Rust. So just don't use safe Rust for those parts. Unsafe isn't evil, it's there for a reason. You should avoid it when possible and when there are no downsides, but sometimes there are, and that's ok.
Make of that what you will.
That's not accurate. Safe Rust can represent circular data structures. It cannot represent circular data structures with 0 overhead. But you can use Rc/Arc/Weak + RefCell for that and it is not very hard. Or use one of crates that expose ways of building self-referential structures in a safe way (of course they do use unsafe underneath, but the API is safe) like ouroboros.
There is also Project Verona.
> Has driven changes in upstream Rust: more try_ methods for Vec that don't panic in OOM: https://github.com/rust-lang/rust/pull/95051
I was curious to have a look at that PR, but it seems it was closed after a long discussion (mainly because it would add ~30% more methods to Vec?). So which changes landing in upstream Rust is the bullet point referring to? Was the Keyword Generics Initiative born out of this?
It doesn't always mean that your app has no memory, it just means that your chosen allocator has no free memory. That's not always an unrecoverable situation.
I usually write my programs to only grab a little more memory than is actually needed so I might play around with this at home. I wonder if this has lead to a culture of grabbing more memory than is actually needed since mallocs only fail at large values if everything is set the traditional way.
Defaulting to overcommit seems risky. I'd much rather the system tell me no more memory is available than just having something segfault. I could always wait a bit and try again or something or at the very least shut down the program in a controlled manner.
* exec
* some tools or even libs map gigantic area of anon mem but only touch few bits of it.
However, I stopped using vm.overcommit_memory=2 because the i915 driver has something called the GEM shrinker, and that never runs in that mode. That means all memory ends up going to the driver over time, and other allocations fail eventually. Other parts of the graphics stack do not handle malloc failures gracefully, either. In my experience, that meant I got many more desktop crashes in mode 2 than in the default mode with the usual kernel OOM handler and its forced process termination.
But the entire value proposition of Rust is reliability and predictability. Use this in critical applocations. And this is the first time this language is being used in a major Os.
The fact that these changes weren't accepted is not a good sign.
In addition, Rust doesn't require you to use allocation, ever. It was originally expected that users who can't handle allocation failures would eschew libstd in favor of libcore (a subset of libstd with all the allocating parts removed).
Sorry to be pedantic, but that's not really the case: https://en.wikipedia.org/wiki/Rust_for_Linux
1. It's not always possible to detect memory allocation failure (e.g., Linux overcommit). So many applications will have to design their operation around the possibility that out-of-memory means someone is going to axe their process unwillingly anyways to support those platforms.
2. Memory allocations tend to be pretty close to omnipresent. If you consider a stack overflow to be a memory allocation failure, then literally every call is a chance for memory allocation failure. But even before then, there's often the use of small amounts of heap memory (things like String in particular).
3. Recovery from OOM is challenging, as you can't do anything that might allocate memory in the recovery path. Want to use Box<dyn Error> for your application's error type? Oops, can't do that, since the allocation of the error for OOM might itself cause an allocation failure!
https://rust-for-linux.github.io/docs/alloc/vec/struct.Vec.h...
NB The prose for Vec here, including examples, is copied from the "real" Vec in Rust's standard library, so it talks about features like push but those are not actually provided in Rust for Linux.
If you're writing a high-performance userspace application, there's a good chance you don't want to deal with handling an error in every single place where your code allocates. I think Rust made the right choice, even though it means some growing pains as it starts being used in kernels
The thing is that ironically, a browser engine is only marginally inside the Rust's niche. (Or maybe it's even marginally outside, at this point I don't think anybody knows.) And for most things things that fit squarely at the focus of the language, it is a bad choice.
The alloc crate came later, and the custom allocator too and is not even stable yet.
https://doc.rust-lang.org/std/alloc/index.html#the-global_al...
Sort of. Rather than bolting on fallible methods adhoc to an existing type, it was felt it would be better to take a step back and actually design this properly. This includes third party crates experimenting with different options.
Maybe we should have a FallibleVec type? Maybe common vec-like methods could be abstracted out in to a `RawVec` type? Maybe both? Maybe the (unstable) `Allocator` API could be adapted to better suite all these cases? Whatever the case it's not great to be adding on a ton of methods in the heat of the moment.
The speaker covers a bunch of areas and the final part of the talk (around 10 minutes) is about Microsoft introducing Rust in some self-contained areas in Windows.
Some highlights:
- Their focus is on "killing bug classes". More context in this post by Microsoft Research from 2019 - A proactive approach to more secure code.
- They want to do this with memory safe languages, CPU architectural changes and safer language subsets. This talk focussed on memory safe languages, specifically Rust.
- First area they've introduced Rust in - a cross platform rewrite of a font parser called DWriteCore. The team reported that parsing was "incredibly easy". Font shaping performance increased by 5-15% compared to the C++ version.
- It took about 2 devs working for half a year to complete this. The speaker says this is pretty good value for an area that is notorious for security bugs.
- Second area is the REGION data type in Win32k GDI. Currently in consumer Windows, disabled by feature flag. Will be enabled in insider builds soon. Performance has been good, some small wins for the Rust version.
- There is now a Windows SysCall implemented in completed safe Rust.
TLDR - Rust is inside the Windows Kernel, will be enabled widely soon.
Personally, I wouldn't link it directly to rust, but to rewriting. When you develop something, you usually can't account for all future changes that affect performance, design, LOC, robustness, and so on. But with rewrite, you take them all into account. So there is a big chance that rewrite will be superior in many areas. It will probably have the same effect as if they had rewritten it in C++ again.
For example:
"Too many cooks spoil the broth" vs. "Many hands make light work"
"Look before you leap" vs. "He who hesitates is lost"
'It's just a few bad apples" vs "one bad apple spoils the barrel"
Or "blood is thicker than water" can mean you prioritise family.
But the full saying is actually "blood of the covenant is thicker than water of the womb" and means opposite
e.g. https://lemire.me/blog/2023/04/27/hotspot-performance-engine... which was on HN today.
Andrew Kelley (zig) did a whole talk on data oriented design a while back talking about the same thing. And Casey Muratori is also talking about it a lot right now, and with good reason.
A C rewrite promising a moderate speedup wouldn't so much be skipped because it was not worth the effort, they are not done because the speedup isn't worth the risk of having to go through all that again.
However, there are some ways that I'd expect similarly capable programmers are likely to often produce faster Rust code than C++ and much fewer ways I'd expect the opposite, so it's not a surprise when this work results in a small unlooked-for performance improvement.
What I think this note bucks is the concept choosing a secure language to re-implement something in means performance overhead compared to C/C++ code that has had years or decades of work putting into it. It doesn't necessarily argue A or safe B is inherently faster, just that a safe B doesn't imply you should expect it to be safe and slower, just safe.
Microsoft is busy rewriting core Windows library code in memory-safe Rust (theregister.com)
147 points by mikece 9 hours ago | flag | hide | past | favorite | 106 comments
The primary source material is this talk: https://www.youtube.com/watch?v=8T6ClX-y2AE
Though there is some learning curve..
Then it also has a borrow checker so if you refer to the content of an other object the langage ensures the (non-owning) pointer does not outlive the pointee, at compile-time. Though this concept is lexical so it’s quite restrictive.
And then it encodes some forms of thread-safety in the langage directly.
It also removes things like nullable pointers.
Essentially all the stuff that makes std::move questionable "just works" in Rust. It doesn't even exist, values are moved by default and clones must be explicit (equivalent of a copy constructor for non-trivially copyable types).
The other giant advantage is that use-after-free doesn't exist in safe Rust, since you can't have a reference to a value outlive the value.
* It is also a compile error for a closure that captures a value by reference to outlive the value.
I can't tell if one if us is confused here or if this is just confusing wording, but the check I referred to is not bug-prone. The pattern of using something after it's been moved from is bug-prone (even though it'd work fine for things like integers, in either language), and it's checking for that bug-prone pattern, hence the name.
I didn't find the syntax very ergonomic but then I'm the kinda guy that likes Python because it's so loose
If you hate writing cmake/make/vcpkg/conan bs, and want to be able to git clone and build (almost) any project, without installing anything beyond rust+cargo... rust will be nice to use.
If you hate the idea of class hierarchies to try and describe behavior and would prefer to attach behavior to any type through traits... rust will be nice to use.
If you like the idea of having generics checking on said traits at compile time with sensible messages rather than the duck typed macros also termed templates with their horrendous error messages... rust will be nice to use
If you like the idea that the compiler verifies for you at compile time the concept of ownership while giving out references, ensuring 1 mutable reference and 0 immutable references, or N immutable references are allowed, while also ensuring the variable being referenced lives longer or as long as the references... rust will be nice to use
If you love spending time debugging invalid references/pointers, races, and more then rust isn't going to nice to use.
While C++ will never be as safe as Rust, mastering it is still a must in many domains, including contributing to Rust's compiler backends.
Such as say in 20 years when you want to be able to run custom code in a then old console.
All of this will become e-waste without the ability to run unsigned code. And not only, by allowing custom code, which some do not allow, the usefulness or even purpose of the device can be extended or altered.
I dont' want to throw away a perfectly usable device just because the company made it obsolete.
Check "Windows 11 Security", "App Isolation", "Sandbox", "Standard User", "Pluton".
Or skim the presentation slides,
https://github.com/dwizzzle/Presentations/blob/master/David%...
To name just a few examples: they want all code to be signed but Windows code signing certs are more expensive than the Apple developer programme membership, much harder to obtain, the rules actually make it "impossible" in some cases like if your country issues ID cards without an address on them, they're now forcing mandatory HSMs and if your CA decides to change your subject name you're just SOL because only Windows 11 anticipates that problem but Microsoft can't be bothered maintaining Windows 10 anymore so their solution was never backported. Yet, the Windows 11 security hardware requirements mean many people won't upgrade.
So whilst building a theoretical strategy around sandboxing apps, they aren't even able to get the basics right. If the people making these decisions were actually writing Windows apps themselves, they might realize this and Microsoft would be able to get its teams marching in the same direction but there's not much sign of that from the outside.
Compare to how Apple does it: they run their own code signing CA, assign stable arbitrary identifiers to companies and people, and still manage to sell these certs for less than a Windows certificate whilst also throwing a couple of support incidents into the mix too, something you can't even get from Microsoft at all as far as I can see (and I've tried!).
By the way, some of this stuff is already on Window 11 Previews and can be enabled.
Even if they botch this, like it happened to UWP, the alternative will be moving everything to Azure OS with thin clients, so one way or the other, it will happen.
The issue is that a lot of their security strategy is inherited from UWP. So it's already botched.
[1] - https://learn.microsoft.com/en-us/windows/package-manager/wi...
That doesn't mean we should just ignore security elsewhere. Many users know not to trust mysterious executables from the internet, but don't expect a PDF or font file to be able to infect their machine.
I think being able to trust that non-executable files like these won't compromise your system could be a big deal.
Microsoft is so confident in their security model that they openly explained how it works: https://www.youtube.com/watch?v=U7VwtOrwceo
Citation needed. Log4j comes to mind as a logic bug disaster, but all the data I've seen is that >65% of the high severity bugs come from memory unsafety in most software projects.
https://security.googleblog.com/2022/12/memory-safe-language... https://alexgaynor.net/2020/may/27/science-on-memory-unsafet...
1. When it's a common recurring source of bugs/problems/pain.
2. When it's no longer fit for purpose. Often either because the purpose has changed in some way or because the purpose was misunderstood when it was created.
In this case at least some of the rewrites were prompted by subsystems being a common source of memory safety bugs continually popping up. Rewriting with technologies and techniques that eliminate some classes of memory safety bugs immediately and in the future is a pretty large payoff for a kernel.
To you know catch those pesky ad cheaters.
96 KLOC of C++ is now 152 KLOC of Rust.
What causes the increase, and is that 1.5x ratio typical?
I mean, sure, it's useful anyway but still quite a niche product and was an oddly sudden dive into Rust from Microsoft at the time.
Eventually they got tired of C++/WinRT, left it half-done (see CppCon 2016 talk about future roadmap plans), and started Rust/winRT.
I'd say I'm quite the opposite. I'd much rather hear Microsoft working on core technical improvements rather than adjusting the goddamn roundedness of their UI.
And unfortunately my new computer does not allow rollback
Do you remember when Windows 10 was the last version of Windows? I wish we could go back.
https://www.theverge.com/2015/5/7/8568473/windows-10-last-ve...
The Windows team is so huge that I wouldn't be surprised if the people writing the shell and the people writing the kernel barely know eachother's names.
(I talk of my work machines over which I have absolutely no control. Every personal machine I own, that isn't a phone, runs some variation of linux.)
I use Windows daily and think it's fine. However today I turned on a laptop in a coffee shop that has been offline for a few weeks, needing to get a few things done. It saturated the small amount of wifi bandwidth allocated to me by the AP for at least a half hour doing stuff, making the work I wanted to do take longer. Under normal circumstances if I used this laptop daily I probably wouldn't have noticed.
It is a paradox that in our world of always-online software that the less you use it the worse it gets.
Like the non-removable link in my start menu for the MS "Mixed Reality Portal". Total waste of good pixels.
I totally get it though, they make it unreasonably difficult to get a usable OS. Default home or even """Professional""" windows makes my eyes water it's so stinky.
I just wish they would 'finish the job', rather than having hodgepodge of windows 2000 and 7 leftovers that don't blend well.
See how silly this looks.
> Click into comments
> Don't engage with TFA
> Rehash random pet peeves about hobby horse
> Become top comment thread above 100 other comments actually discussing TFA
best thing I've read all week.
All this becomes is the xkcd trope of "this next migration will really fix our problems".
Unless they can genuinely replace more than 1 sub system in one go, they just increase complexity.
edit: wrong kind of rust
not that I've experienced that on Win10, which I found to be great.
It has been possible to use Rust to write device drivers that run on Windows kernel space for years, already.
The Windows-rs crate (Microsoft's crate wrapping the Windows API) already has the WDK for a while (i.e.: the special sdk to interact with the kernel).
I welcome the news and agree it is important and meaningful but it is the kind of thing that was easy to see coming.
I would interpret "for a while" as more than ~7 weeks.
Version 0.46 added initial WDK support: https://github.com/microsoft/windows-rs/releases/tag/0.46.0
There are plenty of things in this industry that are hardly surprising but still newsworthy.
The real bummer is that this might eventually be a reason to switch to Win 11, which is something I’ve been trying to avoid for as long as possible.
/s ...unless..?
One day people will smoke on the streets and will comment "Wonder what happened to the tech wizards, we really could use some of them right now".
Even in today's age (during COVID) some small % of the populace actually moved out from the big cities and into rural -- or just smaller city -- communities.
I have hope. Sadly the positive societal changes move with the average speed of a glacier.
blink\nblinbknlibnklbink\nblnink
You know the browser used to be an integral and inseparable part of the OS right? They're just going back to their old ways :-)
It just didn't need to be their browser. But the more important thing, they were fighting the trend, and making sure their browser didn't work. That is what broke the Netscape plan.
Of course they would never, ever do that, but I can hope however hopeless that is.
It'd be based on a userland and kernel that is open source.
And the UI would be hackable (if it were opened up).
Those 3 reasons are huge imo
Windows NT is a remarkably modular kernel architecture, with design decisions far ahead of its time. Things like (acronym overload here) COM, (A)LPC, UMDF, Windows subsystems (for Linux and Android), and an absolutely immense media and utilities ecosystem: Direct3D, DirectWrite, WASAPI, etc.
Why not hope Microsoft open-sources the entire Windows stack including the NT kernel, NTFS drivers, and the Windows API, rather than hope it 'uses Linux'?
> Windows "experience"(sic) into a UI/Desktop Environment for Linux
And for the record, this more-or-less already exists: it's called KDE Plasma Desktop 5.
There's a LOT of weird stuff lurking in there, and a lot of features they've added in recent years just doesn't work properly at all (anything uwp related...). Even the basics of how you start another process have turned into a labyrinthine mess. Then you have things they never fixed like the totally unhelpful NT file locking semantics that regularly break apps that work fine on UNIX.
NTFS is supposedly a nightmare of tech debt that uses SEH for control flow, yet their attempt to move Windows to a new ReFS stalled and failed. Note: Apple managed this with APFS and Linux distros have routinely introduced new file systems.
COM isn't a part of the kernel but is the same situation - incredibly complicated and has not been evolved well. Lots of failed rebrands, attempts to rejuvinate it that actually made things worse.
If you want stable driver APIs that allow you to distribute hardware support with the hardware, keeping that investment proprietary, then obviously no it's crap, and that's a pretty important use case.
If your hardware has good driver support in-tree though, then Linux is hard to beat these days. It just gets so much more investment than the NT or Darwin kernels.
There are a few places where Darwin is ahead especially w.r.t. code signing and sandboxing. I can't think of anything that NT in particular excels at though.
IIRC the new gen graphics frameworks have their origins in work by AMD.
As for Oberon, there is enough material out there.
Suffice to say how Powershell integrates with .NET, DLLs and COM, enabling OS and applications scripting, pursue of safer approaches to OS system programming instead of yet another UNIX clone written in C.
Package that UI and those libraries into the windows experience desktop for Linux and call it a day.
---
If they open sourced windows that'd be good, too! Hackers could then remove all the garbage like telemtry, ads, etc., and it'd be dope as well. They could hack the UI to be even more tiling friendly. It'd be amazing.
> KDE Plasma Desktop 5
Personally, I use KDE because it allows me to replicate one particular Mac feature that's important to me: leaving Control alone for terminal (editor) control characters.
„TL;DR: Rust is named after a fungus that is robust, distributed, and parallel. And, Graydon is a biology nerd.“
https://www.reddit.com/r/rust/comments/27jvdt/internet_archa...
[1] https://www.reddit.com/r/rust/comments/27jvdt/internet_archa...
"Graydon Hoare named the language after the rust fungus because it is robust, distributed, and parallel. He also liked the play on the words from words such as robust, trust, frustrating, rustic, and thrust."
https://www.reddit.com/r/rust/comments/27jvdt/internet_archa...